Slashdot Mirror
Deja, Google, Open Source, Oh My
blkros writes: "Over on Wired
there's an article about Deja News and the plans to try to get Google to open source the Usenet archives it got when it bought Deja News. Part of the plan is to have the Library of Congress oversee it and put it on university mainframes. Google has taken the archives off the web for now Aaagh!"
So how much would you pay for a years access to the archives?
I would happily hand over $10 a year. I really do miss dejanews in it's heyday.
Come to think of it, I would pay for access to Google web search as well, so long as the fluff is removed.
Bill, a liker of usenet.
I just found this article, which may or may not shed some more light http://www.localbusiness.com/Story/0,1118,AUS_6290 86,00.html
Actually, google.com makes real money off ads - its just that they're not obnoxious (and easily blocked) banners. Sometimes, when you do any of the somewhat generic searches, there is an URL returned at the top of the page, above the search results, which is 100% topical, but paid for. Advertising like that, I can appreciate.
Read more about it here: http://www.google.com/ads/index.html - they boast a clickthrough rate 4-5x the industry average, and you bet they make you pay for it!
"I will take the Ring," he said, "though I do not know the way."
I can't tell you how pleased I am to instead see whinging about how "Google plans to let people post". I could understand this being a tragedy and commiserate if, say, the September of 1993 had actually ended.
I'm also tickled to see the phrase "quality web publishing" used with a serious face. ;-)
(Actually, it does make a nice change from "The People demand that Google open source [sic] the archives!" vs. "I'm going to sue for copyright violation!")
"The Crystal Wind is the Storm, and the Storm is Data, and the Data is Life"
"MAKE.MONEY.FAST". Green Card Lottery. `P`H`E`R`O`M`O`N`E`S. Speed Seduction. C-A-B-L-E D-E-S-C-R-A-M-B-L-E-R-S. "Re: Songs about masturbation". Meow. Fuckhead cascades. "RE: re: Longest Thread Ever". Cocaine Pile #107.
Years from now, historians will be calling this the "SPAM Age", as in "Early SPAM Age Man lacked the augmented optics needed to filter advertisements from his field-of-vision".
k.
--
"In spite of everything, I still believe that people
are really good at heart." - Anne Frank
"In spite of everything, I still believe that people are really good at heart." - Anne Frank
They'll use the national security criteria to justify witholding the knowledge of whether or not they have any such archives.
I see even classic Slashdot is now pretty much unusable on dial up anymore.
NOBODY else kept archives of Usenet? Not even the
core heirarchies like comp.* and soc.* ??
That's very surprising to me. It's not like
dejanews was ever that good, that *nobody* else
needed to keep a usenet archive.
Talk about your single point of failure...
-fb Everything not expressly forbidden is now mandatory.
I have to admit I'm of two minds on the matter of the maintenance of the USENET archives. While there is merit to the statement that the Library of Congress (or another national equivalent) has little or no need for the USENET archives, and certainly cannot be particularly interested in bearing the cost of their maintenance, they are a valuable resource. The archives can be seen as a sort of "racial memory" focussed on the period of transition to an information oriented culture. As another poster stated, we gain a great deal from the letters of the Victorians that they likely thought we would at the time of their archiving. However, the suggestion that a government library be responsible for them is not particularly different from leaving them under the control of Google (which paid for them, adding another significant factor to this equation). I think that, if Google is willing to release the content of the archives, it should be maintained by an international organization, in the same spirit as the W3C. Advances in distributed storage and processing make this easier to accomplish than it would have been even a few years ago. After reading the Wired article, there seems to be some indication that this might actually take place, and I must say I would applaud it. Something as valuable as the collected discussions of a developing community should no more be left to a single nation than it should be left to a single company. I would like to state, though, that I minored in History, and am biased towards the maintenance of records, as I encounter far too many situations where our understanding of the past could aid us in the present, but we lack any means of gaining that understanding. I also know the value of letters and seemingly pointless writings (at least those that I have had to make use of :-)
"Be proud to be a fighter" - Martial Arts Adage
Are they obligated not only to delete their copy of it but correct your oversight in not saving a copy for yourself?
I see even classic Slashdot is now pretty much unusable on dial up anymore.
One word: junkbuster.
"we'll be rolling out a variety of new features such as message posting and enhanced newsgroup browsing."
Reading this statement makes it sound like Google is preparing to transform a historically public and open forum into a proprietary gateway.
Praise Google all you'd like for saving the archive and adding "features", but I'd be weary of how they manage this system and it's content in the future. The acquisition will probably do just as much for USENET as Geocities has done for quality web publishing.
if the library of congress had an archive of the alt.binaries.* hierarchy, including all the warez groups, that would be a pretty funny legal quandary. they'd be obligated to put it up (it's the public's archives!) and obligated to take it down (it's copyrighted material!) at the same time... hee hee.
eudas
Blessed is he who expects the worst, for he shall not be disappointed.
Exactly. And in the grandest western capitalist tradition, they're going to make some banner advertisement money out of it, too.
Pretty much everything this moronic reporter claims is no longer possible, is. The archives are a subset of what deja once had, but they are a superset of what existed for the last year.
Specifically, it is possible to:
search the alt. hierarchy
follow threads
search by keyword
It's baffling to think how this story could have been written with such pathetic fact-checking.
Have a look here:
http://groups.google.com/googlegroups/help.html
Maybe it's just the wired/hot-bot/lycos connection taking random pot-shots.
Who is going to pay for the storage and the bandwidth?
>Most were posted by the same crackpots who add X-No-Archive headers to their posts
:-)
Why does putting that on my posts make me a crackpot, exactly?
IMNSHO, due to the way Congress has extended copyright duration to ridiculous terms, they should require full deposit of source code (in its entirety) for all software that is registered, make that source code available for inspection (but not copying), and guarantee that it is preserved for the copyright term so that the purpose of copyright law (making sure that works eventually enter the public domain) is not completely thwarted.
Talk about eeeeeuseless. I'm willing to bet that there are $10 HAM radios with a better signal to noise ratio than Usenet in the last six months.
---
---
Slashdot: News For Zealots. Stuff That's Hypocritical.
I am sure that the value of the content far surpasses the cost of the copy.
Value: analysis of problems, recommendations in areas, poetry, stories, jokes, designs, solutions, ideas.
Cost: man hours spent maintaining the database, economic cost of spending money archiving usenet, hardware, etc.
For this reason, that archive should never be deleted and preserved by the U.N.
You know... There should be a non-profit archival system accountable to the U.N. for the internet. Similar to project Gutenburg in its aims.
Many "services" are fundumental to the future of the human-knowledge-network that allows researchers to quickly access information empowering them to do what they want.
(researcher = anyone from academic to child)
"Homo sum: humani nil a me alienum puto"
(I am a man: nothing human is alien to me)
My only political goal is to see to it that no political party achieves its goals.
I want to buy empty ads for Google and write it off as a tax donation. That way Google keeps running, they don't run ads, I've helped humanity help itself, and I get a tax writeoff.
Is that possible? Legal? It should be. It seems like more bang for the donated buck nowadays than donating to libraries.
Yes, seconded. Especially in the comp.sys groups, there's still enough folks participating to make Usenet my first resort for tech support. There are a few exceptions, but generally Usenet is faster and exhibits more expertise than the typical vendor tech support - some schmoe with a headset telephone and a three-hole punched FAQ. But you do have to have enough skill yourself to sort out the bullshitters from the real gurus. And wow - you should have seen the discussion on rec.sports.auto.nascar this week!
I can see the fnords!
Will I be able to download the entire deja archives as well as means to access it?
I wouldn't mind paying for a CD/DVD (how big is it, anyway?) of it.
One of the thing that really bugs me about search engines that they (undertandably) don't allow direct SQL queries against their data bases. Well, that and the ads.
To be able to refine my search as *I* wish it would be a great thing.
Anyone has any data about it?
--
Two witches watched two watches.
Which witch watched which watch?
eBay is currently in no danger of failing. They are one of the few dotcoms that consistently pull in a profit.
If Google fails -- that would be tougher. I love Google. But I used the net before it existed and I'll use the net after its gone, should it die.
They bought everything, so they own it. They have no plans to use it.
Comment removed based on user account deletion
ummm..they wouldnt need to return a damn thing. Learn what fucking copyright means. It means the right to COPY. IT doesnt have much to do with any original document.
Just what we need. I'm sure if the Library of Congress takes control of it, we will NEVER have the ability to Nuke posts from the archive . . . Not to mention, they'll probalby ignore the X-NoArchive pragma altogether.
---
seumas.com
A tiny minority do. I just grepped through several thousand sitting in the spool here, and 47 articles had expiration dates. Most were posted by the same crackpots who add X-No-Archive headers to their posts. Expires: headers are basically irrelevant to the discussion.
Storing a few weeks of Usenet isn't that complicated (but it's more than "a simple script"). Storing and being able to retrieve several years' worth is something else entirely. Come back to us once you've actually dealt with terabytes of data being randomly accessed by millions of people.
"Patriotism is your conviction that this country is superior to all other countries because you were born in it." -- GBS
It sounds interesting. I've never really read
anything about Google before to this regard.
- Alex
It was Google's decision to buy the company just for the data, and to take down the server farm, let go of the staff and sell the boxes. They could have kept the service running if they chose, but they chose not to.
The monthly charge for a 1000 ft cage at Exodus and the associated bandwidth is NOT cheap.
That is NOWHERE near true, alas.
Only because the LOC can only contain works which have been registered. Copyright law currently recognizes your IP right (whether you like it or not) to anything you create at the moment you create it. Registration, which will get your work into the LOC under appropriate circumstances, is only a tool to strengthen your copyright which you have anyway if you are the creator of a new work. Of course, if you create it it's copyrighted by default but the LOC doesn't have a copy, which is true of many of the works ever created.
Brackets contain world's first nanosig, highly magnified:[.]
So does this mean that the Library of Congress will now be home to the greatest collection of ASCII porn ever assembled by man?
47.5% Slashdot Pure(52.5% Corrupt)
The postings were freely available. Deja/Google never did anything to restrict that freedom. Just because someone builds an archive, that doesn't mean they have to share it. If you clip an article from the newspaper and save it in a scrap book for a few years, do you then have the obligation to show your scrap book to anyone who asks to see it?
Dejanews was useful (and I suspect that by the time Google's people get done writing their software, Google's front end will be even better). But it really bothers me that so many people think that the services and labor that went into building the archive, should somehow be nationalized or forced into public domain, just because so many people want them. Deja never had a monopoly; anyone can archive Usenet (especially now that storage is so cheap).
I hope Google opens things up too (that would be really nifty), but they're under no moral obligation to.
---
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
the archives the archives have no source they are not code. They are going to make them "free". I would guess under a license that does not make the public domain but it is not open source by definition open source is code. Sorry but this has been bugging me for a long time.
Cypherpunks: Civil Liberty Through Complex Mathematics. Those who live by the sword die by the arrow.
While many web forums offer a search function, this is useable at the site only and not indexable by net-wide spiders (such as Google). While in some cases this is a feature, it locks up the content in a way that prevents it from being found, used and archived by net users in general.
I know a few subject matters very well and am happy to be helpful to pass on knowledge, answer questions and participate in dialogue. When this becomes lost I have to answer the same questions again and again, wasting my time. Furthermore, my answers that may be of help to others are lost, depriving them of knowledge that may have helped them.
I have surface knowledge of a great many more topics. I research these, I try to further my knowledge in some, I have to learn about others for work or for other reasons. Being able to easily find information is invaluable and my publicly archived questions may be useful to others.
I know little or nothing about an even greater range of knowledge. Being able to read what others have asked and answered is a wonderful way to start bridging those gaps.
Unarchiveable web forums, mailing lists that don't archive messages on the web and even IRC let this human knowledge slip away.
Not that there isn't a place for all of the above, but I wish more people would consider things beyond their immediate needs.
Bleh!
First, Is Google Up To the Task?, and secondly, What Harm Has Google Already Caused?".
When is the last time you looked at a typical Usenet thread?
Repetitive messages of 99% duplicate text with "Me too!" appended. Identical spam copied to thousands of groups. Heck, September ("newbie month!") from successive years probably has nearly identical content.
Usenet is a compression expert's wet dream. :)
The plan to opensource the database has nothing to do with the Library of Congress. The idea for having the Library of Congress oversee it is just something "some suggest"
LEarn hjow to read people!!!
As someone pointed out, "The library of congress is a library of professional works, not the "my 2 cents" postings that tend to dominate USENET frequently."
Does this mean that some censorship will take place?
Most USENET servers do not archive newsgroups past a certain arbitrary point, since many of them do not have the resources available to do so.
Personally, I think Deja dropped the ball when they "improved" the site to include all the ads and such. I think the google interface, even though it's more limited, is light years ahead of where Deja was going in the last few years. Deja was soooo slow .. it went from like 2-3 seconds per page to like 20-25 seconds per page a few years back (when the ads came in with the new interface). Now with Google, it's lightning fast. Since google has taken over, I guess that I have benefitted from the improvement in speed that Google has to offer that in my mind makes the service much more valuable. I especially like the ability to view multiple responses simultaneously and the highlighting is much better. I can't say enough good stuff about the google interface, and as far as what's missing from Deja .. the _only_ thing I miss is posting .. Even the older articles I used less frequently and am willing to sacrifice in the name of the tremendous speed!
.. it's a major part of my very existence.
I use groups.google.com at least 10 times every day
--
- Aaron Hightower - Lead Programmer - Rush2049 Coin-op
Are you sure you're not confusing Google's explicitly sold advertisements with search results? There is usually a text ad in a shaded cell at the top of the results, in addition to 'adwords' adds on the right hand side.
I haven't seen any clearcut commercial bias in the actual search results google delivers. Can you cite an example?
My FOIA inquiry got a PDQ PFO from the CIA. They can FOD, FWIW.
---
---
Slashdot: News For Zealots. Stuff That's Hypocritical.
That's not very good. Searches used to only take a minute or so.
On topic: I have found the information on bugs/oddities in the usenet archives to be absolutely necessary in getting programs set up and working over the years, especially at a site where there is no usenet access through the firewall (moronic policy).
Because it's so much like pissing up a rope.
If your post was of any interest, it got quoted by all sorts of people, 80% of whom couldn't be bothered to add the header to their posts, and the other 20% of whom do want their posts archived and available.
Also, just because deja.com supported it doesn't mean that other archives did, so at best all it did was break threads when viewed through one particular site.
All of which is fine, except that so many people were so darn dogmatic about it, screaming at other folks who neglected to add the header to quoting posts... which is just plain silly. If you don't want your words saved, use the phone. If you don't want them attached to you, post anonymously. But don't yell at other people just because they're less weird than you are (not you personally).
"Patriotism is your conviction that this country is superior to all other countries because you were born in it." -- GBS
First Deja shifted from Usenet towards their lame shopping thing. Then they changed the Usenet searching interface to make it far less usable and more ad-heavy. Then they started inserting ad links into posts. Then they made all posts before ... June 2000, was it? ... "temporarily inaccessible" for many months, negating much of the value of the archive. And then they tanked; had nothing been done, they would have taken their entire archive with them.
In the middle of this, Google, the company with the absolute best search engine on the net and the most usable web site with the least ads and the best friendliness to geeks like us of any site like them, steps in and takes over the archive and promises development work to make ALL the data more available and more usefully searchable than ever.
This is GREAT! As a long-time Deja user, this makes me very happy and excited. Yes, it sucks that the archive is inaccessible for a few weeks, but that's not Google's fault. Deja shut down the servers, not Google - why on earth would Google ask them to? - and Google's providing some minimal functionality in the meantime. It beats the hell out of losing the information entirely.
Anyway, personally I'd choose some interruption of service with Google maintaining it afterwards, as compared to the status quo with Deja even if it were viable (which it wasn't). Having all the posts from 1995-1999 inaccessible was NOT an acceptable situation from my standpoint as a user.
I'm delighted that Google has done this. Mad props to the Googlistas responsible.
Google only has the archives from August of 2000 and after up on the Web at the moment. Currently the archives going all the way back to 1995 are offline.
Actually, these aren't Deja archives. The current Google offering is its own archive, which it's been pulling since August. As far as I can tell they put this up when Deja took its archives down so that there'd be at least a little continuity while they were working on writing their own archive access software (and/or porting Deja's NT-based stuff to whatever Google is running...a BSD, I think).
This whole episode reminds me of the old saying that no good deed goes unpunished. Google buys the Deja archive, thus keeping it from disappearing forever, and people bitch and moan that it's not available immediately, platform compatibility problems be damned. Google does the work to keep at least a recent archive up during the transition, and people bitch and moan that it's not enough, they should have done it differently, they shouldn't have done it at all, they should have turned it over to the Library of Congress, they should never have been born....
It is early yet, and very little coffee has entered my system, but do you not mean to say that the s/n ratios are low?
I can understand that concern (being not a U.S. citizen myself and knowing that LOC asks license fees for access to certain database tapes from any "customer" outside the U.S, meaning that the IP for certain subject classification data is not available at no cost in foreign countries), but is within the U.S, if I am not mistaken.
I nevertheless think it would be the best solution, as only the LOC has the archiving and classification power to handle those archives professionally. I would suggest that governments other than the U.S. come into a cooperating agreements and support (financially) worldwide mirroring and access at no cost for anybody.
I don't pay taxes to the US government; they have no jurisdiction over me, and hence no obligations to me either on either moral or legal grounds. So why they might choose to make their resources (say, a Library of Congress USENET archive) available to me as a courtesy, such a 'right' to their newsgroup archives would be even more tenuous than the relationship between me and a company providing archive access to customers. (Be the customers paying fees, or viewing ads, or whatever).
So if the archive ever did go to the Library of Congress, I would encourage them to make the archive available for high-profile mirroring; if the National Library of Australia had a copy I'd feel a lot better.
There was a suggestion that the Library of Congress maintain it. Nowhere did anybody from the government or the Library of Congress said they would maintain it or even if there is any interest in maintaing it. Personally, I'll give Google the benefit of the doubt and see what their plans are. They just bought the archives this month. Give them a little time to sort out what happened (remember that Deja was running with a skeleton staff before Google bought it) before anybody starts flaming Google. Now if it's a year from now and nothing's happened, that's a different matter...
What speaks against a cooperation between Google, the LOC and dmoz ? Anything to put it under a wider umbrella of supervision than just rely on "trust" the "good will" of google ?
Basically these three entities all want only the best for those archives to happen. No one so far has shown any signs why their good intentions couldn't be trusted, plus they are all very professional and they would represent a check and balance of each others intentions. dmoz would represent the best of the open source world, google the best of technical geekdom and LOC the best and most unbiased governmental input you could wish for. Of have you ever doubted the LOC's committment to the first amendment ?
I've been using Google since it's alpha test days. Back then, it was a very promising search engine with some serious limitations. (Example: you couldn't search for "AT&T" because all single letters were in the exclusion list of too-common words, so "AT&T" was treated as "AT".)
I've been pleasantly astonished at how Google has improved over the years. Even when they added advertisements, the ads didn't suck: they were on-topic, small, and loaded fast.
Yes, the current Google interface to the Deja archives isn't great. A lot of functionality is gone. Do I expect Google to make huge improvements in the next few months, even weeks? Based on their track record, yes, I do.
Stupid job ads, weird spam, occasional insight at
Just to reinforce how old and lame that joke has become, some dork columnist for the SF Chronicle picked up on it yesterday: http://www.sfgate.com/cgi-bin/article.cgi?file=/ch ronicle/archive/2001/02/20/DD166082.DTL
Congrats for having a totally suburban sense of humor.
When I hear the word 'innovation', I reach for my pistol.
Given that what I write on Usenet is *MY* copyright (by default under the Berne convention- I don't have to put "Copyright 2001 Andrew Oakley" on my articles, just put my name), I presume I can deny the US government the right to use my works.
--
Andrew Oakley - www.aoakley.com
correction: sorry for misusing the expression "open source world" with regards to dmoz, I guess I wanted to say broadly the best of transparency and openess vis a vis archiving and classification of web content by the public.
Really? I am actually a fan (trolltalk reader) and have developed a psychological profile of you in case < hypo> I ever get jaded enough to have my own troll account </hypo>. One thing that won't change: I think your open source metaphor is lame--sounds like it came from a Down's baby or a Protestant.
If you love God, burn a church!
Ewige Blumenkraft!
At the moment, 5 years worth of posts may seem like a lot, but the Internet is going to be around for a lot longer than that. Instead of worrying about the old deja archive, why not work towards getting a new "open" archive happening, starting from now. In 50 years time, whether we have 45 years worth of posts or 50 years worth of posts isn't going to make that much difference...
You're right in that, from what I can see, Google deserves fulsome praise for attempting to save this important historical resource. However, the fact remains that because of its very importance, the preservation of the USENET archive shouldn't just be left to the fortunes of commerce. Librarians are the experts in the long-term storage and archiving of information, and an instrument like the Library of Congress is an appropriate body to do so - even so, *several* bodies in several nations should be archiving this stuff, not one.
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
First of all... just buying all that data to keep it to themselves doesn't sound like a very smart move for Google. And honestly... It doesn't sound like Google (even the way it is these days) to start charging people to view search results from their USENET archive.
--
Ner lbh sebz gur HFN? Gura lbh'ir whfg ivbyngrq gur QZPN!
all you 1337 war3z k1dzi3z and mp3 pirates listen up- here's the plan...
Deja never bothered to archive binaries. and google won't either... but I think I've figured out a way to post binaries directly to google's caching system- break the file up into 98k pages, hosted on a free system like geocities. imbed a title, description and a link to an index of all the other segments of the file at the begining. Voila`! instant free binaries archive.
thank you.
::I will not moderate my opinions for your stinking karma
The tone of the article was such that it implied that Google should be providing this information to the public at large, simply *because* they bought it from what was Deja.com. There's the accusation that Google are doing something morally wrong by taking the archive offline - meanwhile ignoring the fact that Deja.com had already taken a large portion of the archive offline with little or no warning. Google, apparently, are online villains of the deepest dye for wanting to get some form of commercial return for the money that they paid to acquire the archive in the first place.
So let's start from first principles here: the fact that Deja had such a comprehensive archive is not remarkable. The remarkable bit is is that *nobody else has done anything similar*. Deja's value as a resource, both in the commercial sense, as well as in the historical sense, is in its rarity. Goggle, in acquiring the deja.com archives, *prevented* this resource from being lost forever. Yet they're apparently villains for not immediately doing whatever the Open-Source community wants them to. Talk about bloody-minded ingratitude.
There's an argument being made that this information is ours already, although from what I understand, this is legally problematic. However, if you don't agree with Google being able to commercially exploit *your* precious Usenet postings, the answer is straightforward: start posting with "X-No-Archive: Yes" in your headers, and write a *polite* email to Google asking them to remove all your posts from their archive.
For myself, I'm quite glad to see that Google have obtained the archive, and if they do as good a job of running it for easy access as they have with their search engine database, I'll be extremely pleased.
Meg Thornton.
Perkin's Postulate: Online tech support is designed to provide everything short of actual help.
Historians love to read old snail-mail -- reading letters written by Victorians tells far more about their culture than any books written in the time. What should it be any different for USENET?
Although that should probably include enough information to access the postings as separate items, there's a little bit less excuse to ask for the searching code (though there's no reason to not at least ask).
I remember that DEC had an archive of older postings in the late '80s/early 90's. I think that it was Gene Spafford that put it together. Does anybody know what happened to the older archive?
(as an aside, I remember a reference to the value of being able to access DEC drives at internal cost price)
--
Free Software: Like love, it grows best when given away.
Cheers, quokka
2) This strikes me as an important point. What deja.com is selling is not the rights to the posts, or the posts themselves, but the work that they put into archiving the posts, which is considerable. [...] So, while you may have put a lot of effort into writing that post for alt.silly.rantings, deja.com didn't sell that work, deja.com merely sold the work that went into collecting your work.
It's called a "compilation copyright", and lots of online services claim it. You're exactly right, the basis the collection, not its specific contents. Google/Deja has every right to make the claim.
> Does Yahoo 0wn Google now?
No. Google provides its search engine to Yahoo. Yahoo is a client of Google.
I do not understand all this sell-out talk. People who complain about inaccurate results can't seem to separate the (clearly indicated) sponsored links (i.e. ads) from the search results.
> a) Google offers a *free* search engine with no banners.
They do, however, provide ads in various other ways that are far more effective and more appreciated than banners.
> b) Due to their 'no banners' policy, Google does not have a way to generate revenue and struggles to stay financially afloat.
Incorrect.
> c) Google strikes a financially lucrative deal to provide Yahoo with targeted search results.
More power to them.
> d) As a result of Yahoo being Google's primary income generator,
Incorrect.
> they have a large degree of input in the day to day operation of Google.
So, just because Yahoo is a client of Google means that Yahoo influences Google's operation? I find that view a tad cynical.
> e) As a result, Google is now seemingly more interested in lining their pockets than they are in providing a quality search engine.
Learn to separate the sponsored links (ads) from the search results. The indication is clear enough.
I (and a _lot_ of other people) would much rather put up with tacky and inconvenient ads than having to pay a fee.
Alas, tacky and inconvienient ads are becoming less and less able to pay thier way.
Deja carried adverts, and despite being a popular service (look at the reaction to deja suddenly getting switched off) it made a loss.
Bill, just take the money.
What is at issue is the archiving of USENET messages, and archiving them isn't difficult. If you can't figure it out, you can even buy commercial off-the-self solutions to deal with it.
True, but I was thinking about the actual protocol rather than the Open Directory process.
Oh really? I bet a large number of the people to whom the Usenet archive is 'invaluable' would be absolutely outraged if they had to pay some small amount per month or per search. At some point, it's going to be charge or die for most of these companies (well, Amazon is a shop, that's different).
I believe posters are recognized by their sig. So I made one.
Nowadays, I think most peopled don't bother setting headers. But USENET is still a discussion medium. Just because a few companies decided at some point to archive the stuff doesn't mean that the user's presumption should change.
That could be part of the problem though-it's expensive to maintain those huge databases and serve all of the content. If Google doesn't see a benefit they might just keep the bulk of the archives offline.
For archiving, all you need is a leaf node connection with no users. You don't even ever have to store incoming messages in a news hierarchy, you just send them off to your archival storage system as they come in.
Another choice might be to run a traditional news server, turn off article expiration, and keep the news hierarchy on a file system with a hierarchical storage manager (too heavy-handed for me).
Google will honor X-no-Archive and nuke request.
Will the last company to abandon Linux please turn off the lights??!
The NNTP protocol is open source.\
Will the last company to abandon Linux please turn off the lights??!
If you put X-NO-Archive: Yes or did put a proper expiration date on all of your Articles then Deja.com did honor this. It also honored article removal for copyright issues and a Nuke form to remove any article by you.
Will the last company to abandon Linux please turn off the lights??!
Google should become more compliant with copyright law, not less. But, then, the company blatantly copies and retains other content as well ("cached pages").
When you publish your works on Usenet or an email list you are giving permission for people and companies that subscribe to that email list or usenet group permission to store, forward and propagate your copyrighted message. There are ways to protect your usenet posts, this can be a X-No-Archive: Yes: or Expiration: headers.
The same goes for that 20 Mb web page, you are giving people the right to copy your copyrighted web page for their own use.
Will the last company to abandon Linux please turn off the lights??!
Here is a really silly thought, but Usenet Archives would be a great tool for people looking for Prior Art to fight patents. If it is tech. related it has seen life on Usenet.
Not that this will ever get modde dup once there are 150 2 or higher replies though. Ah well. More wisdom thrown into tht ebottomless ether of /.
Does slashdot have to use this buzzword everyday? "open source" newsgroup archives? since when were newsgroup archives closed source---or any source.
First, as others have said, the archive is still there, at the same address (groups.google.com).
Also, the Wired article mentions that a single person called for the release of the archive, but no mention is made of a response from anyone at Google or the Library of Congress. So what? I call for Microsoft to release the windows source code and have the DOJ supervise it. Where is my Wired and Slashdot article?
There are multiple archives in existance, and Deja owns one of them. They collected it and maintained it privately, and they nor google owe you alimony because of the lifestyle you are used to.
Are there no other public or private archives out there? It seems strange that only there is only one service available. I recall a public web and usenet archive out there that stored a limited chronological range, but I don't remember the address...
I don't doubt that google will eventually do a good job with it, even though it does suck in it's current form.
LS
There is a fine line between being a cultivated citizen and being someone else's crop. - A. J. Patrick Liszkie
Google has taken the archives off the web for now Aaagh!
Google has taken the archive down only until they can integrate it with their own archive. Once this is done, it sounds like we will once again have a reliable source of old newsgroup postings.
I highly doubt that they will ever open source the information though. The terabytes of data that they purchased as a part of deja.com is probably the most valuable part of the deal. Why would they then want to turn it over to the government? What financial incentive is there for them? The only way they are going to recover their investment is to create a service like Deja's, only better and integrated with their own.
The following is from Google's press release on their aquiring the data;
Available now at http://groups.google.com, this powerful new Usenet search feature enables Google users to access the wealth of information contained in more than six months of Usenet newsgroup postings and message threads. Once the full Deja Usenet archive is added, users will be able to search and browse more than 500 million archived messages with the speed and efficiency of a Google search. In addition to expanding the amount of searchable data, Google will soon provide improved browsing capabilities and newsgroup posting.
A friend who works at Google said that they got the archives "barely" -- they were apparently copying data as technicians were tearing apart what was left of Deja's systems and hauling the equipment away.
I got the impression that there was a lot of work to be done to fix the data so it was in a coherent form, much less fit into Google's existing storage and databasing environment.
As long as they're still collecting news, plan on improving the existing search engine (my source says yes to this one thing) and it remains free-as-in-beer I'll be satisified.
What kills me overall is the decline in the overall quality of USENET. Too much good content has gone to crap, non-archived, non-searchable web forums (ahem) and what's left on USENET outside of a few newsgroups is spam, porn and isn't worth the time to search.
It seems that the spammers and trolls have scared a lot of people away from Usenet and thus have left themselves as there's no one left. The s/n ratios are the highest I've seen in years.
The only newsgroups I'm active on (alt.fan.james-bond and opera.linux (only on news.opera.no)) are both quite spam-free, with maybe one spam per 30-40 real posts.
When they took down the old deja, I was quite mad. Google's way of viewing the messages sucked. I can't post there anymore either. So I found news.interbulletin.com. It is a usenet service that allows you to view about 30,000 newsgroups and lets you post under any name. It also uses frames which I like becuase the whole page doesn't have to be reloaded. Their server is very slow right now becuase lots of deja users switched to their service. Its usually around 200/bytes per second on my 56k modem. Anyone who was mad too you can use news.interbulletin.com for a while and hope the old deja.com will sometime come back alive (probably not)
--
Mr. Uber-troll Epstein:
Are you running a "would you like me to 'open source' your car..." bot? It seems to be that way. If you can make a duplicate copy of my car for your use, more power to you. I won't mind. There, feeling better yet?
If you love God, burn a church!
Ewige Blumenkraft!
Well, they are talking about 1 TB for the 5 year archive. At 5 GB per DVD uncompressed that would be 200 DVD's. If you get 10:1 (a streach) compression that's still 20 DVD's.
MOVE 'ZIG'.
Hm. Does Google own the database of Usenet postings?
Google does indeed own a copy of the database of usenet postings. More on this later
You see, since every person ever write to the Usenet still retains copyright to their postings, isn't it in the slightest bit illegal to actually *sell* the database? Or at least immoral?
This is a funny bit of Usenet culture/law. While it is generally accepted that usenet users are giving others permission to copy there works they *do* retain copyright. So why can deja go around selling this work? IANAL, but here is how I see it, I think I'm (mostly) right.
1) When you post to usenet, you're sending your work to whatever every archives are in place, and you know it. By posting, you are giving any other user permission to view and archive the material. In fact, you yourself are commanding that the message be forwarded to all other connected computers, and therein lies the implied permission.
2) This strikes me as an important point. What deja.com is selling is not the rights to the posts, or the posts themselves, but the work that they put into archiving the posts, which is considerable. It is the same way that free software sell CDs with open source programs on them. They are selling the data itself and the work that went into collecting the data, not the rights to the data. So, while you may have put a lot of effort into writing that post for alt.silly.rantings, deja.com didn't sell that work, deja.com merely sold the work that went into collecting your work.
Do you see what I'm saying? Or am I just rambling?
Stupid like a fox!
Have you been living under a rock? NT means "no text." Although, it is meant to be put in the subject line - to warn readers that there is no need to waste time loading the body, which would be blank.
Man is born free; and everywhere he is in chains.
However, it could likely be argued that since a poster can exclude him/herself from the archive by using the "X-No-Archive: yes", and that header has been widely publicized, the copyright holder is implicitly giving a license to archive and make available copies of any article they post without such a header.
And since USENET itself rely on copy-store propagation, it could likewise be said that by the mere act of posting you are granting a license for free distribution of the material.
How do you 'open-source' something that isn't source code? Usenet archives are not 'source'.. they are documents.
If he was concerned about becoming famous, wouldn't giving his name accomplish more than insisting to not be identified?
And further, it isn't the owners of the archive (Google) that want the government to run it, it is a lot of people that got concerned when Google bought it, that want an independent copy of it, considering that the archive is becoming an increasingly important part of internet history.
USENET postings have expiration dates, which state to recipients how long the authors intends for them to be retained on public servers. Keeping postings any longer than that looks like a pretty clear violation of copyright. The argument that "you know when you post that..." is bogus. Music publishers also "know when they publish CDs" that their music will get copied, but that doesn't invalidate their copyright.
The fact is that DejaNews got away with this because they were big, and, hey, what could a random USENET poster do?
That's nonsense. There is no "work" involved in archiving USENET postings. DejaNews didn't manually classify or edit articles. What you need for USENET archiving is storage and a simple script. The whole USENET system is designed for easy, reliable replication and archiving. Yes, the storage costs money and it is "work" to buy new CDs and disk drives. But a "software pirate" doesn't acquire a copyright to the stuff he is copying just because it is takes time for him to copy the stuff.
Ok, here goes. Usenet posts are public, so anyone can copy them (although they do all exist on private servers-except universities etc, may want to talk to a copyright lawyer). So, Google bought a copy of these posts from Deja. If others wanted a copy of these, they could have made one.
So, now, let me get the straight, someone had the foresight to make an archive of these posts. Then they had the gall to give access for free. Then, when they couldn't make any money (running a server that big costs a fortune) they had to sell it. Now, people who have been benifitting from the debt of others are mad because the owners of the archive (which anyone with a lot of tape could have made) want the government to run it? I repeat, THE GOVERNMENT!!! Are you kidding. Face it, doing something that anyone could or should have done does not indebt you to society because you decide not to do it anymore.
And that whole bit about the underground Dela guy sounds like some guy trying to make himself famous. If he was so great maybe he would give his actual name. Famous open source people actually do something. Not try and get someone to give you all of their work (a search tool for Tux's sake) so you can do nothing.
Let's stick to stopping Microsoft, RIAA, the MPAA, and the government from taking all of our rights away. Google is one of the best net companies right now. They make a useful product and don't bury us under banner ads. Plus they have a kick ass search engine. Give 'em a break before you try to treat them like Bill Gates.
I seriously hope they wouldn't delete a thing. In a way I'm really ambivalent about honoring the X-No-Archive too...
Sure, it would be great if they could present a "washed" version as well without much of the junk, but only if a full version is availble for those who are interested. After all, we can't expect the LOC to prioritize the same as the entire public.
What about the Canter and Seagal Green Card spams and the resulting "wars", for instance? Would they be removed as spam and flamewars, or kept because they are an important part of internet history? (Namely the first really major spam incident and the start of a long series of high profile spammers and the fight against them).
Once upon a time, not so very long ago, any half-assed computer geek who could formulate an efficient query on Dejanews could find all the tech support info needed to be able to perform just as well as the most seasoned and grizzled network and systems gurus. The dot coms and other "jump-on-the-internet-bandwagon" companies who needed I.T. staff hired such geeks in droves. Soon thereafter, there were still not enough geeks to go around. Salaries rose to lofty heights, even for minimally-experienced geeks who could just get by adequately in their highly technical jobs, so long as they were armed with a T1 Internet line and good old Deja to make them look like heroes in their bosses' eyes.
Fast forward to December 2000, when the dot coms and just about every other computer outfit are now beginning to wake up to the reality of the "new economy" and realize that the sky really is falling. Thousands are losing their jobs. With the loss of that exquisite Dejanews mother of all tech support databases, only the very best of the "True Geeks (tm)" who can, on their own, continue to support large complicated networks, Unix and (gack!) NT/2000 servers, without having to rely on Deja as a crutch, will survive in the coming recession/depression. Deja was the best tool for finding someone else who'd already discovered the solutions or workarounds for those wierd, obscure error messages and crashes you suddenly started getting after you installed the latest Borg service patch.
Now that Deja is gone, all these marginally-experienced sysadmin geeks will now start falling on their faces at work. This whole thing with the economy collapsing, and loss of our online resources, and mega-mergers, and attacks on Open Source, and secret backdoors/clandestine surveillance mechanisms built into our commercial software, and patent and copyright offensives, and whatever else doomsday prophecies you want to throw in here.... all this is leading up to a Darwinistic "survival of the fittest" in the information technology world. How many professional computer geeks out there think they have acquired enough knowledge and experience since 1993-1994 (roughly the birth of the internet "surge") until now to be able to survive in this biz without your online crutches to help you get by? I'm pretty sure I have.
This reminds me so much of the ending of the original Terminator movie, when Sarah was on her way to the mountains in Mexico, and the old man tells her, "There's a storm coming"..... and she says, "I know."
Well, fellow geeks and nerds... there is a storm coming. Brace yourselves for it.
> Being able to search through years of discussions is a powerful tool--hopefully it will remain a free one.
I've used deja in the past to track information down, also to see if a potencial employee was a risk. ( do you realy want to employ a person that write about how to kill your boss in 10 easy ways ).
The fact that Google has bought the old archives gives them the leverage to use old information, in what ever manner they like. They can go right ahead and never offer that data to anybody again.
Or as i see it, pay for data. You want to know something. I have the information. You can get it from me in 3 minutes for 19.95 or you can spend the next 10 hours looking around for it.
ONEPOINT
spambait e-mail
my web site artistcorner.tv hip-hop news
please help me make it better
if you see me, smile and say hello.
If you want to do something useful about Deja, make a method of distributing newsgroup archives that various people have collected through the years. I know I've got tens of thousands of posts sitting around on my machine...if one site kept track of where those posts were, I'd be happy to share them to the world. Which isn't to say that I've come up with much...
I don't really understand what's so interesting about Deja's code. Should be no major problem to create a search engine / interface with all the code that is out there for indexing etc. and all the capable people willing to write / enhance free software.
;-)
The archived postings are the interesting part. At groups.google.com it says that there is a terabyte of data. Maybe it could be made available for download per FTP, one tar.bz2 file per month per newsgroup. Different projects could then try to use the data... Tools like MG (Managing Gigabytes) can create an inverted index that reduces textual data to about 40 percent and is searchable. Well, that's still 400 GB, but HDDs are getting cheaper all the time
Um, that's probably because Yahoo is now using Google as its search engine. See this press release.
Please, God, don't let the net go to subscription hell. We feared this when the big subscription push came a couple years back- and for good reason. I (and a _lot_ of other people) would much rather put up with tacky and inconvenient ads than having to pay a fee.
I must admit though that if some people were willing to pay, it'd make a killer business model. On the other hand, the way that Google already really good. Besides that, the owners of the company rock. I doubt they'd do it.
----
Do you even know anything about perl? -- AC Replying to Tom Christiansen post.
The freaking article is entitle Deja 'Revolt' Against Google, how anyone could have completely misread it and gave the horrible write up we just got is quite amazing.
This leads me to the main question: Major sites such as Google, eBay and Amazon, have become a valuable part of the 'Net and have become an intrinsic part of the World Wide Web experience for many people. Yet, these companies are yet to prove their viability and could collapse at any time if their investors grow tired of shouldering their debts and underperformance. What will happen to the 'Net when the next big dotcomm to fall is eBay or Amazon, or Google? Especially since Google's USENET archive and WWW cache have become invaluable to a number of people.
Does this justify asking the government to step in and take over these resources so they are preserved for posterity as Frank Davies and many others have suggested or is would this be undue interference by the government?
Finagle's First Law
I read that Google only bought the archives from Deja, but not their code or hardware. So where is that code now?
I beg to differ. Television changed war. People saw with their own eyes what war in Vietnam really was and this contributed to the end of the war. When the US was bombing Yugoslavia, I remember some flame wars on alt.culture.yugoslavia that should be preserved forever. For the first time there were citizens of the warring countries arguing about it. Some good points were made by all sides but unfortunately there were the typical slashdot type flamers there too making funny but, considering what was happening, sick jokes like 'Oh now this is good - we are dropping smart bombs on stupid people.' I'm sure there are many other issues should be forever archived for future historians such as abortion and gun debates, conspiricy theories about the US navy shooting down a TWA plane, etc. I would love to be able to go back and read flamewars from the sixties dealing with the Vietnam war, or pre-Perl Harbor debates on weather the US should get involved in WWII, because all that was way before my time. Unfortunately they didn't have the Internet. There is a lot of stupid crap on USENET but I hope they keep it forever. Maybe even this little discussion will be read by a historian that isn't even born yet! So you better get fp! (The meaning of fp is intentionally left as an exercise for said unborn historian, but it was I that first said it.)
The reason that Yahoo's results for the same search mirror Google's is that Google rents their search to Yahoo, not that Google tunes their search to Yahoo's results.
Is it selling out to sell another service read access to your database? I don't think it is.
Google even uses the ODP, the biggest competitor to (historically) Yahoo's core: a hierarchical breakdown of websites.
Does Google own the database of Usenet postings?
isn't it in the slightest bit illegal to
actually *sell* the database?
I would have to think that by the act of posting a message on the a newsgroup you have given permission for it to be distributed and copied via NNTP to the various and sundry news-servers on the net.
Very few of these servers are available on an open basis. ISPs almost always require some sort of compensation for access.
Whether the Deja archive is a news-server or something more woud be a point for lawyers to argue.
I would say that it is quite clear that the transfer of a news-server and it's contents from one commercial entity to another is a common occurance - any time an ISP is bought out this will obviously occur. So the idea of your posts getting bought and sold - get over it, it's already happened, and will continue to happen.
For my own case, I feel that the usefulness of the Deja archive as a source of knowledge far outweighs the loss of whatever small value my postings may have, and as such I happily provide such under the BSD license.
I hope that other Open Source users will take the same view.
MOVE 'ZIG'.
I don't think there are, TBH. There are individual archives of various groups at universities in the US (and presumably in Europe) but even now, when many people claim that Usenet is dead, the daily throughput on a fully loaded news server is huge (Colt's peering news server suggests 23Gb since 1am today which translates to between 3 and 7Gb to its downstream feeds in the last 11 hours). Dejanews made the commitment not only to keep a running archive of Usenet but also to collate those archives stored by other agencies. I think the only other near competitor was Remarq and they fell over last year. Deja's store, as Google said the other day, runs to many terabytes of data and must have a pretty solid dedicated feed (the last time I was involved with news server management was about five years ago and the company I was working for recieved its primary news feed from UUNet's satellite delivery service, a dedicated 47Mb downlink), so it was a major investment to keep it going, hence the attempts to leverage the data by turning it into discussion channels for shopiing and whatever, something that was doomed to failure as, as useful as Usenet is, the signal to noise ratio in the most dedicated of groups is sufficient to make searching a frustating process if only for the number of 'me too' noise that most discussions generate. It's good that Google have managed to secure the archive in that respect, but maintaining it and making it usable isn't going to be easy.
They most certainly have not taken the archives down. They can be accessed Here.
Sigs are awesome huh?
To my simple mind the Open Directory Project seems to be the best bet for a model for searching Usenet. It's arranged hierarchically, it's searchable, it's open. And instead of having all the archives stored in a single location, what about having them distributed? Just a couple of random ideas while I've got other better things to do.
As storage becomes cheaper it will only become more important to have accessible usenet archives. Text messages aren't getting any bigger, but our capacity to store them is greatly increasing.
Being able to search through years of discussions is a powerful tool--hopefully it will remain a free one.
So here's my suggestion to Google: Do whatever you want with your own interface to the data, which will probably be excellent. But if you want to be heros, find a way to get this data into the public domain.
Oh, wait. Obviously it's the Source part.
Send your friends messages of love at fuck-you.org
I agree with you on the "washed" version. What is there now, should be kept "as is". That is a historical documentation of what really happened.
But I do think that a thorough "brainstorming" session is needed to decide how usenet is handled in the future. As for sociological research, I believe, the whole research is already on its way to be useless, because thousands of people just post to incite responds in order to see "how the society is reacting". It's already an experiment in the hands of sociologist and nothing much natural anymore about it. You just might find enough willing posters who are aware of it and NEVERTHELESS play the game with you together, because it's just fun. The guinea pigs are aware what's being tested in them and their reaction is already "tailored" and therefore the results "skewed".
In general I am not that ambivalent about the need for x-no-archives requests to be respected, especially not, if I know that any little company can open up a forum and deny posters x-no-archives and nuking requests, but are happily making the archives available for any person to search and misuse.
But if all the newsservers would really be owned by the public and not by companies, I could possibly agree to wave my x-no-archive rights, hoping that a regulation about it would be decision upon a democratic vote of the public.
At the moment I feel that the traces a poster has left in the archives can too easily haunt someone. It encourages "profiling" of a poster and can be abused. Privacy of posters was not at all protected backwards for years and years. I do feel that many people were not aware when they posted for what purposes these archives could be used in the future. I think that it is even more important to realize that, considering that the archives are owned by commercial companies. I don't know if newsservers owned and maintained by LOC would respect privacy and allow AC posts.
I could imagine that the government would hesitate to allow AC posts, but I don't know. The whole thing seems to be a war between two mutual exclusive desires, the protection of your privacy in longterm archives (achieved through AC posts) and the fact that you are using a public forum for which you could argue that privacy rights are given away as soon as you post.
I don't know which rights are more important and which ones should supercede the other. If usenet would be under the protection of something like LOC, I believe more people would realize (and actually for that reason be more conscious about what the say and how they same something) that "you are really posting for the public and for the archives". Can't help that thinking this post, would it be archived for the next ten years or so at LOCs servers, makes me at least a bit more respectful for the work of the archivers and think twice, if I should continue with mumbling my opinions straight out of the blue.
Sadly, those days of readily available, easily navigable information are long since gone. With Google's recent sell-out to Yahoo, I've been getting results that seem to be less focused on accuracy, and more focused on whoever is paying Google the most for the top ranking. Results pages are also strangely similar to Yahoo's results for the same search. I'm not saying this is necessarily a bad thing, but it's surely a shame to see one of the major backers of open-source information sell out to corporate politics without so much as a fight.
Deja is sadly, heading in the same direction. Back in it's hayday, I could always count on Deja to find those obscure little tidbits of Linux information when I needed it, but now, it seems to be nothing more than a shell of it's former self, providing a haven for spammers, crapflooders, and again, the corporate machine.
It's a shame that what were possibly the last two refuges that us Linux user's have from the corporate machine that is MS, Yahoo, and AOL are apparently being stripped for parts and left to rot at the hands of the almighty dollar.
-atrowe: Card-carrying Mensa member. I have no toleranse for stupidity.
This means it should be feasible, not only to get at least a decade of Usenet archives, but, if other universities have similar backup policies, to get multiple redundant, geographically distributed Usenet archives that can be cross-checked agaist each other for historical accuracy.
This may seem like a minor point, but I do know of at least one instance where Deja's archives appear to have been tampered with -- and it would be unrealistically naive to expect that there is no incentive for certain parties rewrite history.
Finding the spots where people may have tampered with history could turn out to be as interesting as anything that comes out of the Usenet archives.
Seastead this.
I have to admit, the immediate goals of such a project illude me. USENET postings are already public. These are open forums, and the groups can be read from most libraries or other public sources.
I don't see the value in the long term achival of USENET posting. The library of congress contains just about every copywrited work ever written. This serves not only as a national archive of our author's produced works, but gives our legislature access to the documentation and research they need to do their job. Would the archiving of USENET posting serve the long term mission of the nation's library?
It also bothers me slightly to think that people's comments and flame wars will langish forever in the federal library. I don't think access to USENET postings is something the nation craves or needs. What the nation needs is access to works that have been researched and published, works from professionals. The library of congress is a lbrary of professional works, not the "my 2 cents" postings that tend to dominate USENET frequently.
----------------------
Kurt A. Mueller
kurtm3@bigfoot.com
PGP key id:0x4FB5FB1D
Lawrence Lessig is my personal hero.
No it's not a stretch. Compressed a text file recently? Average ratio is at least 50%. Usenet posts as a whole contain mountains of redundant information. I'm assuming the 5TB includes headers here, because headers will be the most compressible parts. Still, the number of DVDs is irrelevant. Any institution that requires the full Usenet archives will have the resources to purchase a DVD jukebox with the needed capacity.
"Hot lesbian witches! It's fucking genius!"
Hm. Does Google own the database of Usenet postings? You see, since every person ever write to the Usenet still retains copyright to their postings, isn't it in the slightest bit illegal to actually *sell* the database? Or at least immoral?
:-)
At least I am giving no permission whatsoever for someone to sell my posts...
Someone could argue, though, that by posting to the Usenet you have implicitly maken your work public domain, but I doubt that you can get rid of your copyright that easily. Books still have copyright, and you even paid money for them, so shouldn't you be getting more?
Maybe, but even when privacy buffs post to netnews they know they're posting to a known, public forum -- I know some privacy nuts who won't post in public forums for this reason. Since it's a public forum anyone can read, it also means that anyone can reasonably harvest the data. When you post to a web forum, you don't know what data they're collecting, plus USENET has no banner ads or hidden submit buttons..
I think the controversy you refer to was the initial hullabaloo when Deja first went online. People felt funny with the idea that "anyone" could basically grep through a huge news spool and pull out potentially embarassing rants that were hitherto considered "gone", like forgotten party gossip.
This was a non-issue, AFAIC. If you feel embarassed about your opinions, keep them to yourself. If you feel like holding someone's opinions from yesterday to them today, remember that your opinions, as well as the facts, have probably changed since then too. And the reality is that the fact the "old" news kind of scrolled off into the bitbucket was really more of a defacto situation than a dejure one -- there was nothing stopping individual sites or users from archiving news except money and time.
What I want to know is if they are still collecting new news articles.
DVD can hold up to 16 GB (today). Check this link.
So we're at 6 (or so) discs. And 20 discs isn't a big deal. We just bought a family tree maker program that came on (gasp!) 20 CDs...
I think most libraries, and people needing quick USENET archive access could live with that.
If you could be told what you can see or read, then it follows that you could be told what to say or think - BoC
As other have mentioned posts to USENET are public, therefore, a poster may own the copyright to it, but there is no protection under copyright law. It is public domain. What google does own is an extremly complete database of public USENET communications. It also happens to be a database that ANY other person/entity could have compiled, but did not. Here lies the problem. Google paid deja for the time and expense that went in to the compilation and storage of that database. They also purchased the software that deja had written to search/access this large database. So in all reality, I don't know how google could be forced to turn over data, that is already in the public domain,just because they were the only one's smart enough to buy it from someone that compiled it all. It is clear that you can copyright stock analysis or similar data that is based on information that is wholly in the bublic domain, but due to it's value when compiled becomes copyrightable on it's own. From the sounds of things Google does not intend to do this, but it certainly would be within their rights.
Clearly, the resource we've come to depend on so heavily has fallen into the control of folks who don't appear competent in administering it; but, what a huge resource. I was pissed a week ago, when I hit deja looking for the answer to a question I assumed would appear in mere moments. What a shock! And, of all organizations; Google!
But, really, folks, if you look at the realities of what the techs at Deja had to deploy to make that work, it shouldn't be suprizing it came to an end. What amazed me most was the religious availability those guys made sure their fleet of Linux boxes maintained.
I can't imagine how this project could be replicated in lesser projected time frames, even as an Open Source endeavor. Perhaps a Gnutella model? Who knows?!
Sad, days, indeed...
Linux rocks!!! www.dedserius.com
www.dedserius.com
VB != VisualBasic
Perhaps highly compressed indexed by year on dvd's? think this would be possible? Large dictionary compression maybe? I'd buy a set for my own use. I fear these archives are going to be commercialized to the point where end users may not be able to have free access to them any longer. government run the site? shit i'm sure they'll figure a way to hook DCS up to it... bleh
===sam=== free nessus vulnerability scan = www.vulnerabilities.org
This is directly equivalent to maps. Just because you put a road down on a map does not mean you own the road. It also means you are not entitled to scan and sell commercially prepared street directories just because they depict public streets. This AFAIK also holds true for all those gene maps, it's far more reasonable to copyright your map of the human genome than to claim you own it by virtue of mapping it.
Tangent:
Google could seed the archive with markers (same as how map companies put minor errors in their maps) in order to demostrate a copied archive came from their investment.
Xix.
"Everything is adjustable, provided you have the right tools"
The archives are not offline..they are at http://groups.google.com/
If anyone questions the value of the net as a source of historical data, check this out: Webhistory Mailing list archive ...where you will find the real-life (ie not a lame historical re-enactment) transactions which produced the modern web, one of the most significant events of the century. The 1993 archives contain mail from Marc Andreeson and others about creating forms, images, etc in the Mosaic (Netscape precursor) browser.
Kibo is that you?
"I challenge Google" ?!?
Google's purchase is the best thing that's happened to those archives since they were started. No, it's not open source. No, Google won't give away something they probably just spent a huge chunk of change on.
And yes, there is no doubt in my mind that the archives WILL be back.
Were any of the whiners quoted in the article actually paying attention to Deja's service? The searchable archives kept shrinking! Eventually it would have reached this point weather or not they had been bought out.
We're going to have to accept the realization that for having public archives of Internet material, there are really only two options: 1: Collectively do it ourselves through taxes and the government(s) (what I'm in favor of) or 2: Actually have a pay service that you pay a monthly fee for. It simply requires to much manpower and storage otherwise. I mean, can you imagine a library operating through selling advertising? Aside from the practical business realities, would you want to live in a world so corpratized that you have to endure ads in the library? Or strategic "upsells" and "click throughs" that mean when you look in a how-to book a speaker comes to life and says you can buy that saw for $49.95 at Home Depot? (this is pretty similar to what Deja was doing in its late incarnation).
---
DO NOT DISTURB THE SE
Google today announced plans for an open-source public celebration of the event. Attendence will be open-source as guests will not have to pay to attend and there will be open-source food for all. Though bands have yet to be announced, there will be an open-source concert to entertain partygoers at this historic event.
/., Wired, et. al.
Public domain is not the same thing as open-source you fucking morons at
~GoRK
Oh, yeah, and since this comment is open-source for you to read, i'm selecting to license it under the LGPL so Jon Katz can go publish it in his damn book if he wants and I dont have to get my colon in a knot over it!!!
this is useable at the site only and not indexable by net-wide spiders (such as Google). While in some cases this is a feature, it locks up the content in a way that prevents it from being found, used and archived by net users in general.
Actually many static pages of Slashdot are included in Google.
I remember that before the "real" publishing of Slashcode (1.0?), lots of people were saying that they were waiting the source to implement NNTP output from Slash. Did any of them succeed?
I suppose that anyway Slashdot wouldn't enable it as it goes against its business model.
__
__
Men with no respect for life must never be allowed to control the ultimate instruments of death.
GW Bu
Well if Google plays into the Al Gore tune, and doesn't concede within an ample amount of time here's a nice list for someone to wget/snarf up old Usenet archives.
I wonder what if any, are the underlying factors for this move.
Chicks worthy of Usenix pornspam status
"When I was a Buddhist, it drove my parents and friends crazy, but when I am buddha, nobody is upset at all"
What will happen to the 'Net when the next big dotcomm to fall is eBay or Amazon, or Google? Especially since Google's USENET archive and WWW cache have become invaluable to a number of people.
Does this justify asking the government to step in and take over these resources so they are preserved for posterity as Frank Davies and many others have suggested or is would this be undue interference by the government?
I think that Google is the main contributor to the Internet Archive, a non-profit that holds teras and teras of Geocities pages.
__
__
Men with no respect for life must never be allowed to control the ultimate instruments of death.
GW Bu
--------
Nice article, but you need to use your own resources a little better. Wired has had several articles about Brewster Kahle and The Internet Archive (archive.org).
TIA is *already* archiving Usenet, although its coverage has been less thorough than Deja. They're a .org with the clear mission and mandate to
archive Usenet, and make it available to those who
want it. They're not in the search-engine
business, but would supply the content to those who
want to build one.
Hopefully this info will be good for a follow-up article.
Being able to properly and efficiently use the combined experience of thousands of geeks who have gone before makes me less of a "True Geek (tm)"?
How does using Usenet differ from troubleshooting a network client with a manual in hand? And really! Who can be expected to know every single error message the Collective can attack with?
I like to think my net admin and hardware skills are pretty l337 but I'll be the first to admit I'm not fully MCSE-compliant. That said, I've worked for the past 3 years at sub-200 employee companies with tiny networks and tinier I/T staffs. The upshot of which is I have to be net admin, security manager, help desk, warehouse admin, and reporting specialist, all in a single domain model with only 3 other servers and maybe 50 clients.
It's a little hard to be a perfect MCSE when the resources to learn and experience from just aren't there. Wait, let me clarify - I get a LOT of experience in different areas (see previous list), but it doesn't get very deep for the most part. At the same time, I don't think using Deja/Google doesn't make me a paper MCSE either.
There has to be a middle ground in the real v. paper MCSE elitist crap that goes on. Some of us are just luckier in our work environments, others, not so much...
And another thing, I install free/shareware and test it like it's going out of style. I currently have 97 folders under Program Files and countless shell and browser plugins. I see a lot of strange crap and without Usenet I'd never be able to figure some of it out.
P.S. While I'm at it, if anyone can tell me why Gimp for Windows dated 10/23 and 11/26 won't save files in WinNT, I'll be very grateful! And why Forte for Java 2.0 on jdk 1.30 crashes during install?
P.P.S. I know you didn't specifically mention NT and MCSE's, that's just my perspective. GTRacer
-- Not willing to pay upwards of $60.00 for the GT3 demo disc...I'll pay $30!
Defending IP by destroying access to it? That makes sense, RIAA/MPAA. Go to the corner until you can play nice!
Usenet is wide open. It's public. Deja doesn't own anything that's particularly unique. There are probably other, more complete, Usenet archives out there. Google is just using a copy of already available public information. There's nothing stopping anybody else from also archiving Usenet. If there's a need for it, somebody else will continue to do what Deja has done for years (if somebody else hasn't been doing it already).
Uh...Adam? Does Yahoo 0wn Google now?
That would be horrible if it's true. Yahoo has one of the nastiest AUP/TOS on the Internet. I hate it big time. I have most of my personal hosting on Yahoo Website Services, which is so lame...I gotta get out of there because according to their TOS they 0wn everything that's on my site.
Trouble is, my main email address is with them and my business site is there. Moving is going to be hell. I gotta do it though.
And no, I didn't CHOOSE YWS...they bought Simplenet a year ago and made the choice for me.
Yahoo SUCKS.
----
http://www.msgeek.org/
Knowledge is power. Knowledge shared is power multiplied.
Just the contrary, that's exactly what I hoped would be accomplished, if LOC would get involved.
... chinese government:-). To me it would seem to be the best thing which could happen, if LOC could be asked to become owner of those archives. This is in order to protect those archives and the posting rights of the posters from any biased group. This is also to make them available at no cost to the public, and protect them from being distroyed or misused.
Everybody says usenet is a public forum, but the medium on which the forum were uphold and their public speech recorded, was never public. Not all the newsservers and connections were government owned and paid by people's tax dollars.
The usenet archives collected by deja news, the data, are owned by a commercial company. They can decide whatever they want to do with them, the IP value of the posts is neglible, but the monetary value of a complete set of data of all archived posts is high. The fact that deja-nes.com originally treated the data from usenet groups the way oldtime usenet users were expecting them to be handled, doesn't mean that they couldn't have done differently.
I used to use deja-news.com since early 1997. At that time deja-news.com was still ok, but usenet itself was a pretty mess. Many technical worthy groups were complaining about serious flamewars and thousands of useless posts. Old usenet users were whining and crying out for the good ol' times, when men were real hackers and did their business "among themselves". Deja-news.com was respecting poster's right for nuking their posts out of the archives, something which I consider a right, which needs to be protected, as well as posters x-no-archive requests.
Later, when deja-news.com turned into deja.com, the whole thing became more commercial. Now, as they can't make a profit, it's sold. The new owner can do whatever they want with the archives. Who guarantees you that they don't go down the drain in a year from now and sell the archives to the
If usenet is supposedly to be a public forum, then the public should own the archives of this forum and LOC should get involved and protect those archives for the public. At least then for the first time the archives of a "quasi public forum" would also be owned by the public and the regulations about how to handle the archives is under democratic rules determined by the tax payer and voters.
Why you conclude that a commercial company is more prone to respect those rights to nuke, x-no-archive etc than LOC, who is funded by tax payer's money, I don't understand. The public is posting, so the public can demand that LOC respects those rights.
Contrary, any commercial company would want to archive EVERYTHING against the will of the posters to collect larger archives.
LOC's involvement could be very well the best which could happen, because they would be the most unbiased and most professional archivers.
I would understand if you would say LOC might be the toughest cataloger, because contrary to any open/free source zealots, they are possibly courageous enough to classify and discriminate content while cataloging. That's what librarians usually use their brain for, but of course, I hear already thousands of kids screaming "no, I want my 0.02 cents worth of flamewar/porn archived til the next millenium". If you could fear something from the LOC than it might be their professional scrutiny towards what they consider content worth archiving. I doubt that they would trash anything which is worth to keep, certainly not all the historically important archives of the beginning eighties and the like.
Since that archive is nothing more the collection of the copy righted writings of 100s of thousands if not millions of people, they have to "Open Source" it. If they don't I will write a cease and desist letter and send it to Google that they return my copy righted work to me and then delete them from the archive.
Burn Hollywood Burn
People are misreading Google's press release, which states: "This acquisition provides Google with Deja's entire Usenet archive (dating back to 1995), software, domain names including deja.com and dejanews.com, company trademarks, and other intellectual property." Folks, this does not mean that Google had the ability to just keep the status quo with Deja's existing server farm. Note that the press release says nothing about hardware, staff, or the full intellectual property behind Deja's operation -- it's obvious that Google didn't get a whole lot from Deja other than the archives and the domains. The fact that they're in the process of transitioning the service to their own server farm and search interface hardly means that they made a bad business decision -- it was likely the only business decision.
That's right. Otherwise, Deja would have just disappeared, like so many other .com tragedies have lately. Someone else has already posted on the fact that Deja had techs in the server farm ripping out hardware as soon as the sale happened.
So cut Google some slack -- they're the savior here, not the destructor. Think proverbial silk purse. Nobody likes not having access to the full archives, but I feel confident that Google wants to get them up and running, and they're almost invariably the best people to get the job done.
Just get the date sort working, Google guys, OK? ;)
Eschatfische.
sorry for the many spelling and grammar mistakes of my previous post.
Even though I really believe they should have waited until their new solution was complete before pulling the deja site down to avoid interruption of service, I'm going to give them the benefit of the doubt in believing they are going to do the "right thing".
--
Twivel
Has anyone tried submitting FOIA requests to the CIA, FBI, NSA, NRO, etc, to try to get copies of any Usenet archives they may have? If they have such archives, it is unlikely that they will meet any of the criteria that would allow them to deny a FOIA request, e.g., privacy, national security, etc.
The beta search interface also prevents people from following newsgroup threads or searching by newsgroup hierarchies -- such as "alt." -- or by dates.
Try going to the Advanced Search. You can also view a thread by clicking on the View Complete Thread link when viewing an article! I haven't seen a search including dates yet, but if you're going to complain, at least do some research...
There's no place I can be, since I found Serenity.
If you love God, burn a church!
Ewige Blumenkraft!