How Google Saved USENET

Masem writes: "Salon has a well-written article article on the recent revival of much of the USENET archives from '81 to '90 by Google. It mentions that much of the recovery was thanks to years of work in transferring data off 140-some 10" magnetic tapes (~120megs of data) to a more conventional format in order to recover much of the early posts. Even a reference to the previous Slashdot story is made." Update: 01/07 23:52 GMT by T : btempleton adds: "O'Reilly Network asked me to do an article on similar themes and rememberances of USENET history." Thanks, Brad.

Comments

Title give an impression.

[ImaLamer]

I think the title of the story should be How Google Saved USENET.

Yes, google saved the historical record of the USENET, but it needed not to save the USENET from anything else. USENET is alive and well.

Re:Title give an impression.

[baronben]

Haveing 30 odd thousand mac v. windows flame wars might not seem like a great thing to save now (espicly if you were invovled with them), hell, no one might care in 100 years, but a lot of histroy is based on reading the common writings of everyday people. How great is it to be able to read the dirary entries of a Frenchmen durring the middle of the revolution, or to look at the account book of a middle ages merchent. Most of it may seem mundane, but histroy is made up of many mundane momments. Thats why we have grad students, to sift thrugh all the "Me to" posts and "M$ Sux" posts to find the really meaningfull stuff. You must admit, its great to look at the google archives of the birth of Linux or the first mention of AIDS.

Didn't search USENET as much before Google.

[reaper20]

Google Groups is awesome, especially when searching for some obscure piece of hardware advice or settings.

I don't have to worry about getting and setting up a news client, and it's just one tab over from my default search engine.

Google did save USENET for me - though I never post, searching through all the linux and comp newsgroups is usually faster than looking up a HOWTO.

groups.google.com always has the answers...

[ThomasMis]

As a software developer, no matter what problem I run into, somebody else has already run into that problem and has asked my question and recieved an answer on groups.google.com. Whenever I get stuck on anything at all, it's the first place I run to. groups.google.com is the single most useful site you can point your browser (konqueror!!!) towards. I'm not sure how they make money over there at google, but what a great service they are providing!

Re:groups.google.com always has the answers...

[aussersterne]

This is absolutely true. I am often asked "What book(s) can I buy, to learn what you've just told me? How do I gain the knowledge in [subject X] that you have? I don't care if it takes me a decade, I just want to learn it, but I can't seem to find out where. Is it written down?"

I tell them: it is a decade's worth of learning, and then some, but not from books. It is all from USENET. I became a competent C programmer who writes more efficient code and makes fewer fundamental mistakes thanks to usenet. I learned to use BSD and then to use Linux as fast and furious as I can type and to get myself out of any system problem, save my data from nearly any corruption thanks to usenet. I am able to network these odd things, build these robots, and have this "cool stuff" that you like so much that works so well thanks to usenet. I can make nearly any computer go, now matter how old or wierd or what media or operating system it uses (a feat which makes you a legend in your own department) thanks to usenet.

It's not my knowledge... I humbly picked it up in the mid and late '80s and early '90s and still constantly refer to it, first through Deja and now through Google. It is our knowledge, collective and stretching backward in time. To ever lose the news archive would be a tragedy -- the amount of searchable data on everything from chemistry and biology to computing and electronics to literature and politics is truly stunning. With the news archive, you can learn to hotwire together any two things so long as they have *wires* to do something useful; you can learn to brew just about anything including some of the best beer ever; you can learn just what the HELL James Joyce is talking about at times in Ulysses. Every question has been answered before you even asked.

The only sad thing has been the degree to which the groups have been turned into a boulevard of endlessly flashing neon porn signs in the last few years, almost to the degree that anything else is drowned out by the brightness.

Study USENET. Use USENET. Live and learn. Amen.

What kind of backup is used now?

[msolnik]

I was wondering what kind of backup googles uses now for all its info? What happens if one day a script kiddie breaks in and rm -rf / all the boxes? Do they have tape backups? How many etc. I also wonder how much they spend on it.

Google getting involved

[prof187]

Google seems to be getting involved with a lot of things. It's nice to see that a group is not only trying to push the Internet forward, but also trying to preserve the past.

Save the posts

[Kefaa]

I am sorry they will allow requestors to delete their own postings. While we might wish it otherwise, 10, 20, 50 years later, this may be the real historical value. To purge, seems the equivalent of having a letter to the editor removed from newspaper archives.

To those who feel like "they are walking around with their baby picture stapled to their forehead", we all mature. What I thought at 20, 30, and 40 show how I grew. What other archive in human history can provide the transitional opinions, discussions, and outright imbecilic flames wars?

While we would hate to have someone pull out our post in support of the flat earth theory, to act as though we all believed the earth was round is rewriting history. Convenient for us, but misleading to the future.

The question now becomes, what happens after Google and Slashdot, when the archive is tera-bytes large? Will it take 100 years for the next conversion?

Re:I've really got to wonder...

[frank_adrian314159]

I can't imagine there is much money to be made off the technology, because it's all text - the same search tech applies. So, as far as I can tell, there is no business reason to be doing this.

If you build it, they will come...

The old USENET posts are an information archaeologist's garbage heap. If information has any intrinsic value at all, this is the place to find treasures. Just because some folks see dirt doesn't mean there isn't gold to be mined.

Google saved usenet?

[Anonymous Coward]

I guess you read a different article than the one I saw. The article discusses the archiving that saved a bunch of old usenet posts for posterity. Google didn't do it. They didn't even exist back then. They just happened to be willing to take on the responsibility of making that data (which other people had already migrated from mag-tape to a more modern media) available to the public.

Google/Deja has killed USENET, not saved it

[markj02]

USENET used to be an informal discussion forum, like something where you might talk with others like you would around the water cooler. Google, AOL, and similar services have greatly expanded the user base for USENET, which means that it isn't much of a community anymore. And by archiving and republishing in perpetuity, thinking people have to watch carefully what they say, or they just post anonymously (or don't participate at all anymore).

This was probably an unavoidable turn of events. Nevertheless, whether it is Google or some other company, I consider it wrong for them to republish this stuff, in particular as part of a commercial venture. It's the equivalent of digging out old security surveillance tapes and broadcasting them for the amusement of the masses. It's wrong, and the fact that people find some sort of voyeuristic delight in it doesn't change that. The backup tapes that Google used should have been destroyed.

Re:Google/Deja has killed USENET, not saved it

[BlacKat]

Erm... Usenet is a PUBLIC system, any and all posts you make there are in the public domain.

The information is preserved for posterity, not for making money or other commercial exploits.

I can't really believe you think we'd be better off destroying information instead of preserving it!

Re:Google/Deja has killed USENET, not saved it

[DaCool42]

Preserving newsgroup posts is hardly the same as posting private information. When you post onto a newsgroup, you post to whoever wants to read it. If you don't want it to be read, don't post it. It's that simple. It's like walking in front of a camera in a TV studio and hoping nobody sees you on TV.

Re:Embarrassing posts archived

[Zymurgy]

Nah, sorry. I sure don't mean to be inconsiderate to your concern, but the whole essence of USENET is that it is not private, and that the USENET reading public will read what you have posted. Would you have posted something in the first place unless, at the time, you had wanted someone to read it?

Consider your embarrassment an experience to recognize your growth! How about it?

Also, though my opinion holds no more weight than the next man's, I think it's somehow "wrong" to remove your posts from Google's archive as the poster right above me mentioned you could do. If it's true that doing this would actually permanently remove one's postings from the archive, is it one person's right to delete a piece of history?

Anyway, I think what Google has done is extraordinary. Dejanews before them provided a wonderful service, but it's great to see Google bring back the entire archive!

Thank you for your time, folks!

Message forums (Slash) are killing off Google.

[BrookHarty]

Allot of the good gurus are moving over to slash ran message forums. Talking to a guy who is a perl guru, he has moved most of his perl help requests from usenet to Perl Mongers. I've been seeing this trend in the last few years, as independent subjects are moving over to a website based web forums. I even spend more time reading 5 mailing lists and a dozen message forums, and dont touch usenet anymore.

With these message forums and mailing lists not linked to a usenet group, there is a lot of wasted knowledge that is not shared. I would love to see a slash-mod or some type of mailing list enhancement that posts a overview or some kind of daily message post to usenet.

The whole idea of usenet was knowledge sharing, not binaries and spam ads. Glad google has saved usenet, but some effort needs start using it again.

Humm, Maybe Slashdot should enhance a usenet forum? Thou 5-20,000 posting a day on a usenet might be a little much. Maybe only 2+ posts make a moderated usenet group.

Re:Wow, similar story

[jazman_777]

released today in the San Francisco Chronicle. Read it over at sfgate.com [sfgate.com]. I/m surprised two independent media organizations would review the same company about the same thing and release it in the same general time frame! Amazing~

Gee, the print media has a hierarchy: All editors read the NY Times, the LA Times, and the Wash Post to see what the consensus important stuff is. The editors of the LA Times and the Wash Post read the NY Times to see what the important stuff is. The editors of the NY Times decide what's important stuff to print. This is why all the newspapers look the same.

great for digging dirt?

[Quixote]

OK: how long before a presidential candidate's Usenet postings will be dragged out for the whole world (US) to see ? :-)

yes, you sum it up

[markj02]

So, you are saying that USENET has changed from an informal discussion group to a searchable perpetual repository of technical support Q&As, plus a repository of background information on people who were foolish enough in the 1980s to post under their own names. I agree. The part I don't understand how you think that constitutes "saving" USENET. USENET didn't use to be much of an on-line community compared to some of the others, but it was a community. Once it became archival, anonymous, and searchable, that went away. Who, after all, wants their every word recorded and replayed into perpetuity?

The Usenet archive is not saved yet

[osswid]

Google is a private startup. They might still go out of business, or be bought by someone. Even if they have a successful IPO, these could still happen later.

What happens to the archive when they're bought by someone else, or end up in bankruptcy court? Will it go the away of the online digital photo storing sites, vanishing one day without a trace, taking irreplaceable data -- data of immense academic historical interest -- with it?

Google should promise to donate the archive to the Library of Congress, do the transfer now, and make a social contract with the net community to turn over the reigns on this project if they're acquired or go out of business.

On Behalf of the USENET Preservation Society...

[cowbutt]

Google did save USENET for me - though I never post, searching through all the linux and comp newsgroups is usually faster than looking up a HOWTO.

As a regular USENET poster, I'm gratified that you've found our posts useful, but please, please do consider participating yourself!

"But I don't know anything worth posting!" , I hear you cry. Well, for a start, since when has that stopped anyone on USENET, myself included! Besides, I'm sure everyone knows something about something, even if it's "only" mexican cooking (alt.food.mexican-cooking) and Italian manga (alt.italian.anime-manga).

Take the trouble to subscribe to a few groups and get involved. Keep them as lively discussion fora, not dusty historical archives and a spam collection!

I discovered USENET in 1992, and I've rarely gone away. It's definitely the most consistently interesting and useful part of the Internet, IMHO.

--