I don't see what would prevent you from screen scraping the archive, and bring our history with us to the new site. url_pattern = "slashdot.org/archive.pl?op=bytime&keyword=&year=&page=" for year in (1998 to 2014):
thread_page = 1
while true:
url = url_pattern.replace(").with(year).replace("").with(thread_page)
threads = extract_threads( fetch(url) )
if no threads:
break
for thread in threads:
comment_page = 1
while true:
comments = extract_comments(thread + "?page=" + comment_page)
if no comments:
break
append_to_database(comments)
comment_page += 1
thread_page += 1
How long would it take?
The Slashdot archive spans 16 years. Let's say there is an average of 35 pages worth of threads each year. Each such page of thread links takes about 7 second to load. In an hour you should be able to get all thread URLs.
Let's further assume it takes 4 seconds to load a comment thread and that there are 270 threads URLs per page. That gives us a total of 16 x 35 x 270 ~ 150000 thread pages. Getting all those threads so you can extract the comment related data and meta data would then take about a week (7 days).
Finally, lets assume each discussion thread has two pages, so we double that time. Still it's only two weeks of linear scraping to get the entire slashdot archive. Now if you parallelize this, it'd naturally go much quicker. A few hours of scraping is all that's needed, or a bit longer if we don't want to put undue stress on the site (being good netizen).
Go for it I say, before the PHB DICE folk realize that they bought a platform for technical people by technical people and start removing useful features from the site (like the archive).
Please service enable the back-end. If you do you'll be free to experiment with *your* user interface as much as you like, and the rest of us can create the UIs we want, including recreating and maintaining Slashdot Classic, even after you no longer feel like doing so.
I share in the common sentiment that the current design is more than good. To me it's close to perfect. The few features I'm presently lacking I've easily layered on top myself via GreaseMonkey.
Any change you do will never agree with all readers, so IMHO the best thing you can do to cater for those who will hate your changes is to allow them to render the UI themselves. Seeing as large parts of the entire industry is moving to a REST based backend model, for channel independence (mobile & web using the same services) and UI evolution through competition (marketing agencies creating competing with skins / mobile apps using another company's backend services), why not follow suit and offer such services yourself?
If you provide me a REST interface to all the categories, threads and comments, then I can myself create the UI I want, on-the-fly (in my browser / mobile app) or on my laptop's local web server. With that option available I won't cry foul, since at least then I have an avenue of recourse.
Where did the PP state that apple gloss over new product models? I saw no such statement nor sentiment. What I read between the lines was rather a compliment to Apple, namely that the PP speculated the reported R&D sum for Apple in the report might have been too low compared to the other company numbers. If improvement in OSX, IOS etc. are not counted towards the R&D budget for Apple, while the same kind of spending would be at another company (E.g. Windows 8 for Microsoft), then obviously the reported Apple figure would be too low.
However, speculations and specifics aside, the interesting message in the PP's comment, I think, was "Think about what the numbers mean and whether they are comparable or not to the other numbers in the set". Are apples being compares to other apples (no pun intended) or is the report showing a mixed fruit comparison?
No modpoints today, but I'd have awarded you a "+1 insightful". Too seldom do people pause and reflect on what the numbers they see actually mean, myself included.
And yet developers are still treated like second class citizens in far too many organizations
Well IT is regarded as a supporting role, same as the girl servicing the coffee machine or the guy mopping the bathroom floor. They're all needed, just not viewed as part of the core business (which regardless of industry tend to be: selling more than last quarter, reducing expense compared to last year, coming up with new processes at least once a year and measuring KPIs for those processes).
I don't know why you would care as a linux user. You do realize don't you, that when average Joe & Jane get fed up with Microsoft, they tend to move to the other platform with a usable desktop, the walled Apple garden and not Linux?
Rendering faster is cool and all, but for me Firefox 3 and now 4.. I mean 5 is fast enough. What I really yearn for is tab isolation. Are you guys doing anything to remedy the problems which surface once you open many tabs ?
The only reason I started using Google Chrome is because FF would simply stall once 30-50 pages were opened in tabs. I'd love to continue using FF due to it being far superior to Chrome in all regards but two; scalability and memory management. That chrome allegedly renders pages faster than FF isn't even in my radar of important factors when choosing between the two browsers. Scalability however is.
And this helps how? If a site is overloaded, the service is denied to me. If *my* browser starts to "back off" it exacerbates the problem by increasing the outage I experience. A site is placed in the net to serve users content and if a user can't access it, then that person is per definition subject to Denial Of Service. A browser constructed with the described mechanism has a defect built in by design.
Then you have nothing to complain about, do you? Since you know the ad serving is a bottleneck, you must be valuing the advertisement important enough to warrant a slow internet experience...
Kickstarter seems like a very good idea and I'm glad I followed the links related to this particular post. There are some great minds out there and I hope this sort of venue can help those people pursue what they excel at for the betterment of us all. Thanks to the./ post, this "One man's quest" received some additional coin from me as well, since I consider his track-record impressive enough to warrant belief in his next enterprise.
Are you seriously questioning the ethics of allowing patents describing *new*, ahead-of-their-time (non obvious) kind of stuff ? That's precisely the kinds of patents I would like to see if we're to have a patent system at all, since ground breaking ideas pull civilization forward. Not that I'm saying *this* particular pattern is ground breaking, I haven't read it, but your sentiment seems to be that since it was filed looong before most of us had a clue, then its somehow unethical !?.
What we do not want are patents which attempt to monopolize the obvious and the stuff of yesterday since those are the kinds that pull society in the other direction.
A 'White Hat' hacker is someone who aims to improve security
That statement leaves the definition up to a point of view. From the US PoV this could well be seen as a white hat activity as the aim is to serve USGov interests, while from the targets PoV it would be deemed black hat. A Russian counterpart of this company would by your reasoning be a black-hat company from a US perspective but a white-hat (good) from Medvedev's, since it poses a threat to the USGov agenda and serves the RusGov's.
Can you please elaborate on the problem you see? I failed to see it (must be daft).
What is preventing you from sending in a patent application and then go talk to you colleagues / send papers to the academia press? Wouldn't it be a good thing to be able to just "fire and forget" a patent application and then go on with your life and do all the academic stuff you enjoy, without having to worry about who might next emerge from under the bushes and make your life miserable?
Equates to my building and one of our neighboring ones. With 1 Gbps per apartment I fail to see the awe aspiring in the "accomplishment" from that perspective. Assuming it wasn't my neighbors who got hacked and that the world's 500 million connected households have an average of 1Mbit/s uplink capacity, the feat might be interesting from another perspective than the consumed bandwidth; being able to orchestrate 100k drones without being traced. That's pretty cool since there must have been quite a couple of decoy routers and mechanisms in place to prevent tracing the origin of an orchestration like that. "Secret" command and control center of modern warfare, cloak and a 100k daggers.
Perhaps the USG should track these people down and recruit them in order to bolster the forces for their War On Internet (their goals seem aligned after all).
Unless you also use a proxy such as Tor or Relakks, Google et'al will typically be able to piece together that you're you by looking at your IP address or network. A VM by itself won't do squat for your privacy.
If you have a dynamic IP which changes all the time, then it will take a bit longer (more clicks) through the web before "Google" can associate your current surfing session to the "file" they have on you.
So Tor/Relakks + short surfing sessions - logging into any site should hopefully keep your surfing somewhat private.
Just to be clear, Google's customers are the advertisers. We the users are their products. But yes, if their products evaporate it will be a might challenge to sell anything to their customers and the nickle-and-dime folks at Google will feel that.
I put more faith in the article they referenced than the one in context.
From the first grade to 5th, I was a bit of the timid type that "played well with others" and tried to keep a somewhat low profile.
At 6th grade and through the rest of the school years I developed a "I don't give a rats *ss what anyone things"-attitude and I always said what was on my mind since I'd come to hate double speak and "political intrigue". The rationale was that it would be better if people knew that what I said was what I meant and thus know me based on that, rather than having to guess hidden agendas and conjecture their own ideas of who I was or what my intentions may had been.
Or as I would have then thought of it "Why would anyone say something they don't mean? That just makes stuff unnecessarily complex and it is therefore a stupid thing to do." I also had the impression that most people were fools because they couldn't see the obviousness and patterns in situations that I could (rather arrogant I admit) and said so when "stupid things" were said and done. Not appreciated by some, why, I didn't understand then.
The first four years after graduation, I came to realize that this "open and frank" attitude didn't really work too well in the corporate world and I tried hard to suppress my outer voice and instead try to find different angles to what people were saying and how they were saying it. This provided a new realization that there probably were less fools around than I had previously thought. After yet a couple of years, it had turned my view of the world and people in general completely 180 and since then I believe that most people have a lot of valuable ideas and valuable which I could learn and benefit from. My personality had returned to the personality which "plays well with others".
Now this evolvement from one type to another and the return to the original shows that personalities can indeed change with or without effort, as postulated by the article I linked. However the outcome of my little anecdote, if used in the study under discussion, would have supported that theory as well, which would obviously have been a false positive.
I don't know if it is common that people change personalities during their youth and then regress to an initial personality or if my example was just a fluke, but since the article in question claims that personalities would remain the same into adult life, I simply felt obliged to provide a counter example to negate that theory.
Why bother with trying to link a MAC to a public IP address and how would you even do that? Provided the GP uses some NAT box, like take-your-pick model of a home router/wifi-AP combo at Best Buy, all the MAC will tell the storm troopers is that some machine with a MAC got an internal IP address from the router. It does not tell the troopers that this NATed IP actually downloaded a given torrent.
However, if the troopers find a file with the same message digest on one of the drives confiscated from the GP's home, then it'd be pretty darn hard to sell the court the tale that "John Doe with his MadWiFi skillz passed by the window and is the culprit you should be looking for". MAC addresses have nothing to do with it.
I'd appreciate a 3:4 display such as my first monochrome monitor for my Atari ST, since my region's defacto document format is A4, a format with more height than width.
A high-res 4:3 screen which can be rotated 90 degrees would be perfect for reading documents and a "killer device" for our corporate laptops.
I really don't need a "cinema display" at work since like most people, I'm not in the movie or television industry.
Slashdot Mirror
Ok, the slash-filter botched the meta-code, but you get the point I hope ..
I don't see what would prevent you from screen scraping the archive, and bring our history with us to the new site.
url_pattern = "slashdot.org/archive.pl?op=bytime&keyword=&year=&page="
for year in (1998 to 2014):
thread_page = 1
while true:
url = url_pattern.replace(").with(year).replace("").with(thread_page)
threads = extract_threads( fetch(url) )
if no threads:
break
for thread in threads:
comment_page = 1
while true:
comments = extract_comments(thread + "?page=" + comment_page)
if no comments:
break
append_to_database(comments)
comment_page += 1
thread_page += 1
How long would it take?
The Slashdot archive spans 16 years.
Let's say there is an average of 35 pages worth of threads each year.
Each such page of thread links takes about 7 second to load.
In an hour you should be able to get all thread URLs.
Let's further assume it takes 4 seconds to load a comment thread and that there are 270 threads URLs per page.
That gives us a total of 16 x 35 x 270 ~ 150000 thread pages.
Getting all those threads so you can extract the comment related data and meta data would then take about a week (7 days).
Finally, lets assume each discussion thread has two pages, so we double that time. Still it's only two weeks of linear scraping to get the entire slashdot archive.
Now if you parallelize this, it'd naturally go much quicker. A few hours of scraping is all that's needed, or a bit longer if we don't want to put undue stress on the site (being good netizen).
Go for it I say, before the PHB DICE folk realize that they bought a platform for technical people by technical people and start removing useful features from the site (like the archive).
Please service enable the back-end.
If you do you'll be free to experiment with *your* user interface as much as you like, and the rest of us can create the UIs we want, including recreating and maintaining Slashdot Classic, even after you no longer feel like doing so.
I share in the common sentiment that the current design is more than good. To me it's close to perfect.
The few features I'm presently lacking I've easily layered on top myself via GreaseMonkey.
Any change you do will never agree with all readers, so IMHO the best thing you can do to cater for those who will hate your changes is to allow them to render the UI themselves. Seeing as large parts of the entire industry is moving to a REST based backend model, for channel independence (mobile & web using the same services) and UI evolution through competition (marketing agencies creating competing with skins / mobile apps using another company's backend services), why not follow suit and offer such services yourself?
If you provide me a REST interface to all the categories, threads and comments, then I can myself create the UI I want, on-the-fly (in my browser / mobile app) or on my laptop's local web server. With that option available I won't cry foul, since at least then I have an avenue of recourse.
Where did the PP state that apple gloss over new product models? I saw no such statement nor sentiment.
What I read between the lines was rather a compliment to Apple, namely that the PP speculated the reported R&D sum for Apple in the report might have been too low compared to the other company numbers. If improvement in OSX, IOS etc. are not counted towards the R&D budget for Apple, while the same kind of spending would be at another company (E.g. Windows 8 for Microsoft), then obviously the reported Apple figure would be too low.
However, speculations and specifics aside, the interesting message in the PP's comment, I think, was "Think about what the numbers mean and whether they are comparable or not to the other numbers in the set". Are apples being compares to other apples (no pun intended) or is the report showing a mixed fruit comparison?
No modpoints today, but I'd have awarded you a "+1 insightful".
Too seldom do people pause and reflect on what the numbers they see actually mean, myself included.
And yet developers are still treated like second class citizens in far too many organizations
Well IT is regarded as a supporting role, same as the girl servicing the coffee machine or the guy mopping the bathroom floor. They're all needed, just not viewed as part of the core business (which regardless of industry tend to be: selling more than last quarter, reducing expense compared to last year, coming up with new processes at least once a year and measuring KPIs for those processes).
I don't know why you would care as a linux user.
You do realize don't you, that when average Joe & Jane get fed up with Microsoft, they tend to move to the other platform with a usable desktop, the walled Apple garden and not Linux?
Rendering faster is cool and all, but for me Firefox 3 and now 4 .. I mean 5 is fast enough.
What I really yearn for is tab isolation. Are you guys doing anything to remedy the problems which surface once you open many tabs ?
The only reason I started using Google Chrome is because FF would simply stall once 30-50 pages were opened in tabs. I'd love to continue using FF due to it being far superior to Chrome in all regards but two; scalability and memory management. That chrome allegedly renders pages faster than FF isn't even in my radar of important factors when choosing between the two browsers. Scalability however is.
Doesn't Australia also have this decease ?
And this helps how?
If a site is overloaded, the service is denied to me. If *my* browser starts to "back off" it exacerbates the problem by increasing the outage I experience.
A site is placed in the net to serve users content and if a user can't access it, then that person is per definition subject to Denial Of Service. A browser constructed with the described mechanism has a defect built in by design.
Then you have nothing to complain about, do you?
Since you know the ad serving is a bottleneck, you must be valuing the advertisement important enough to warrant a slow internet experience...
Kickstarter seems like a very good idea and I'm glad I followed the links related to this particular post. ./ post, this "One man's quest" received some additional coin from me as well, since I consider his track-record impressive enough to warrant belief in his next enterprise.
There are some great minds out there and I hope this sort of venue can help those people pursue what they excel at for the betterment of us all.
Thanks to the
Are you seriously questioning the ethics of allowing patents describing *new*, ahead-of-their-time (non obvious) kind of stuff ?
That's precisely the kinds of patents I would like to see if we're to have a patent system at all, since ground breaking ideas pull civilization forward.
Not that I'm saying *this* particular pattern is ground breaking, I haven't read it, but your sentiment seems to be that since it was filed looong before most of us had a clue, then its somehow unethical !?.
What we do not want are patents which attempt to monopolize the obvious and the stuff of yesterday since those are the kinds that pull society in the other direction.
That statement leaves the definition up to a point of view.
From the US PoV this could well be seen as a white hat activity as the aim is to serve USGov interests, while from the targets PoV it would be deemed black hat. A Russian counterpart of this company would by your reasoning be a black-hat company from a US perspective but a white-hat (good) from Medvedev's, since it poses a threat to the USGov agenda and serves the RusGov's.
Well three actually: Data != Knowledge
Data processed may turn into information.
Information when consumed by an individual may turn into knowledge.
The sum of the world's knowledge is therefore not measurable since it resides in the minds of individuals, not in books or other recorded material.
Can you please elaborate on the problem you see?
I failed to see it (must be daft).
What is preventing you from sending in a patent application and then go talk to you colleagues / send papers to the academia press?
Wouldn't it be a good thing to be able to just "fire and forget" a patent application and then go on with your life and do all the academic stuff you enjoy, without having to worry about who might next emerge from under the bushes and make your life miserable?
Equates to my building and one of our neighboring ones. With 1 Gbps per apartment I fail to see the awe aspiring in the "accomplishment" from that perspective.
Assuming it wasn't my neighbors who got hacked and that the world's 500 million connected households have an average of 1Mbit/s uplink capacity, the feat might be interesting from another perspective than the consumed bandwidth; being able to orchestrate 100k drones without being traced. That's pretty cool since there must have been quite a couple of decoy routers and mechanisms in place to prevent tracing the origin of an orchestration like that. "Secret" command and control center of modern warfare, cloak and a 100k daggers.
Perhaps the USG should track these people down and recruit them in order to bolster the forces for their War On Internet (their goals seem aligned after all).
Fine Oracle, give Apache a JCK already !
Do that and they will have a reason to care about the future of Java.
Unless you also use a proxy such as Tor or Relakks, Google et'al will typically be able to piece together that you're you by looking at your IP address or network. A VM by itself won't do squat for your privacy.
If you have a dynamic IP which changes all the time, then it will take a bit longer (more clicks) through the web before "Google" can associate your current surfing session to the "file" they have on you.
So Tor/Relakks + short surfing sessions - logging into any site should hopefully keep your surfing somewhat private.
Just to be clear, Google's customers are the advertisers. We the users are their products.
But yes, if their products evaporate it will be a might challenge to sell anything to their customers and the nickle-and-dime folks at Google will feel that.
I put more faith in the article they referenced than the one in context.
From the first grade to 5th, I was a bit of the timid type that "played well with others" and tried to keep a somewhat low profile.
At 6th grade and through the rest of the school years I developed a "I don't give a rats *ss what anyone things"-attitude and I always said what was on my mind since I'd come to hate double speak and "political intrigue". The rationale was that it would be better if people knew that what I said was what I meant and thus know me based on that, rather than having to guess hidden agendas and conjecture their own ideas of who I was or what my intentions may had been.
Or as I would have then thought of it "Why would anyone say something they don't mean? That just makes stuff unnecessarily complex and it is therefore a stupid thing to do."
I also had the impression that most people were fools because they couldn't see the obviousness and patterns in situations that I could (rather arrogant I admit) and said so when "stupid things" were said and done. Not appreciated by some, why, I didn't understand then.
The first four years after graduation, I came to realize that this "open and frank" attitude didn't really work too well in the corporate world and I tried hard to suppress my outer voice and instead try to find different angles to what people were saying and how they were saying it. This provided a new realization that there probably were less fools around than I had previously thought. After yet a couple of years, it had turned my view of the world and people in general completely 180 and since then I believe that most people have a lot of valuable ideas and valuable which I could learn and benefit from. My personality had returned to the personality which "plays well with others".
Now this evolvement from one type to another and the return to the original shows that personalities can indeed change with or without effort, as postulated by the article I linked. However the outcome of my little anecdote, if used in the study under discussion, would have supported that theory as well, which would obviously have been a false positive.
I don't know if it is common that people change personalities during their youth and then regress to an initial personality or if my example was just a fluke, but since the article in question claims that personalities would remain the same into adult life, I simply felt obliged to provide a counter example to negate that theory.
You mean download advertisements collections?
That's the only content legal for downloading. I read it on the internet and also heard it on Fox News.
Hint: If the Ad collections you torrent aren't rar-ed and have nfo files you shouldn't trust the commercials.
Why bother with trying to link a MAC to a public IP address and how would you even do that? Provided the GP uses some NAT box, like take-your-pick model of a home router/wifi-AP combo at Best Buy, all the MAC will tell the storm troopers is that some machine with a MAC got an internal IP address from the router. It does not tell the troopers that this NATed IP actually downloaded a given torrent.
However, if the troopers find a file with the same message digest on one of the drives confiscated from the GP's home, then it'd be pretty darn hard to sell the court the tale that "John Doe with his MadWiFi skillz passed by the window and is the culprit you should be looking for". MAC addresses have nothing to do with it.
I'd appreciate a 3:4 display such as my first monochrome monitor for my Atari ST, since my region's defacto document format is A4, a format with more height than width.
A high-res 4:3 screen which can be rotated 90 degrees would be perfect for reading documents and a "killer device" for our corporate laptops.
I really don't need a "cinema display" at work since like most people, I'm not in the movie or television industry.
Good thing then that Ubisoft isn't focusing on strategy nor RPG games.
For their kind of titles, an in-game tutorial works rather well.
"Crouch, Jump, Point and shoot. Now play".