Slashdot Mirror
Digital Dark Ages?
angkor writes "The digital dark age--Will all the information from this computer age slowly vanish as our delicate hardrives expire? That's what it looks like. Better start printing everything out."
← Back to Stories (view on slashdot.org)
Does this include getting your server slashdotted in record time??
They really should warn the people that they are going to be posting a link to their server, and that extremely heavy traffic will arrise.
Maybe it's just me, but whenever it looks like a harddrive is about to die (funny noises, etc. or just getting old) we replace it before it does. Also, we back up critical information, often in more than once place. This sort of practice should, in thoery, prevent this from happening. These things are replacable.
The snow doesn't give a soft white damn whom it touches. -- ee cummings
ignorance, suppression, warfare, famine, strife
;)
Sounds like a fit description of the Msft dominated computer industry alright.
Fact is, that's just what the comp industry WANTS - the old name is 'planned obsolence', nothing, very little anyway, is built to last. At best it's made to last 3 years then you thro it away and buy another. Gotta keep them customers spending $$$!
A co-worker was talking about archiving his ancient family photos with a scanner and CD writer - I told him if he's lucky within a generation a descendant or relative will take up the job of transfering them from CD to holographic crystals or whatever is the format du jour at the time. Just like the DNA code is recreated every generation.
I print out ALL online transactions involving $$$, just in case there's a dispute
try { do() || do_not(); } catch (JediException err) { yoda(err); }
Along with data copying, technology is delivering home users progressively better storage mediums. From 5.25" to 2.5" floppies, to hard drives, to CD-Rs, each media lasts longer than the previous. We'll eventually get it to archaeological standards.
In the year 2675, when some archeologist try to puzzle together what the world looked like at the beginning of the century, any info at all will be very valuable.
Even your collection of porn.
The solution to both saving ancient works on paper can work just as well for digital media. Keep copying the work to the latest storage media! None of the original texts that we do have have survied. They are all copies made from generation to generation. Thus with digital media. The best of the web (lets say, research articles) will be preserved and transferred to new storage media as it develops. Your blog about your day at the beach prolly won't.
"Overhead, without any fuss, the stars were going out."
You obviously haven't been on an archaelogical expidition ever. Most of what archaeologists and the anthropologists who tag along with them are concerned with, is the trash of past societies and cultures. Most often, the shards of pottery that they laboriously extract from the ground are in so many shards because they were discarded by their original owners/makers.
Your trash says an awful lot about you, as does the random splay of stuff strewn around your room. Future archaeologists may not be interested in the porn on your hard drive (unless they have to dig it out), but future anthropologists would find it very interesting (and not in the normal manner people find porn interesting, though that may be there too, never know). It says alot about you, an inhabitant of wherever you are, living in the year 2002, as does all the collected sundry data on your drive. It may certainly seem boring as hell to anyone else, but historians and anthropologists can get a whole lot of useful information out of it. It's no less boring than reading through book after book, or letter after letter in the dead tree sense, and in some ways it's alot easier, as you can't write a regular expression to pull whatever interesting tidbits you are looking for out of a book.
Amen, to that! And more often it's practically a full-time job, just shuffling all of it around, from one over-flowing server to the next.
--Logan
Stuff that matters: circuitbreakers, vacuum-cleaners coffee makers, calculators generators, matching salt+pepper shakers
Which one's more vulnerable, a set of negatives and a single set of prints bent into a camera shop envelope high in my closet, or a digital photo on my hard drive? Sure, hard drives have a designed window before obsolescence, especially in the consumer market. Basically that's because the cost of enhancing their reliability is less than the cost of a whomping new drive that dwarfs the old one every three years. Even so, though -- hey, how many photos do you have from your great great grandparents' trip to Tahoe in the year aught-six?
If we're talking about preserving the works of Aristotle, I'm betting on hard drives to do a better job than monks with feather quills. (Not that the monks didn't draw better pictures in the margins, doodling along the way.)
"Fundamentalism" isn't about divine morality. It's about human authority.
There are several ways this could go. Obviously, we have to be circumspect, since the U.S. gov't is literally considering copy-control legislation that would make Linux illegal.
You can say it'll never succeed - won't all Linux's rich patrons prevent it? But I would have said the same about quite a few other things that have already happened... and it's in our interests to act as thought it might.
However, assuming something slightly less than the worst, DRM will of necessity be something which you can enable or not. IOW, as long as they'll let you, buy all the fast, new DRM drives you want, and use Linux to run them. Linux will simply ignore the DRM features and use the drive normally.
The problems come when you're forced to use a DRM operating system with your DRM hardware (quite a reversal from the old antitrust days, eh?); you will find it very difficult to take some/all of your data back to Linux/other non-DRM OS.
You can probably see why MS loves this now; DRM technologies, even optional ones, will have the nice effect of preventing interoperability with open source operating systems, thereby locking everyone in even further. Let alone the myriad other possibilities for abuse, censorship, and bottlenecking...
If we allow our government to do this, both in the context of MS's current status as a monopolist, and in the ongoing (anti-) regulation of the media industries, we are doing the gravest disservice to future generations.
We're on the road to Tycho.
Given the propensity of M$ & others to use proprietary file formats in an effort to lock in the client base and to lock out competition. (And don't tell me about standards like because XML [tagged data storage & transport streams] without DTD [document tag definitions aka data context] is pretty damn useless [the difference betweeen data & information.])
I have quite a few files that I can no longer access except as raw byte streams because the applications that created them no longer exist or because the meta data information that controlled that creation is no longer available.
Even printing sh.., uh, stuff, out is pretty useless because most paper is acid based and turns to ash over a very short time. The inks are not much better.
I have books printed in the 17th century that are still quite readable (high rag content acid free paper,) and a 1901 Sears catalog (acid washed wood pulp paper,) that I accidentally put my thumb through in the late '80s.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
first, we need to think logically.. Every bit of information we have discovered that is aincent was discovered by sheer luck and accident. NOONE back in 985 BC set aside the stone tablets thinking that "someone will want to read this in 3000 years. EVERYTHING we find out about the past has been accidental. Nothing has ever been intentional archives preserved for the distant future.... If there were we might have a whole bunch more knowledge than we do today. (we re-invent things every 50 years.. because we lose how it was done 100 years ago.. My great grandfather's workshop was filled with things that were over 100 years old yet I have seen marketed today as "A TOOL BREAKTHROUGH! The Self Ajdusting wrench!")
I take EVERY digital photograph I shoot and burn it to CDROM. nothing ever get's deleted in my photography.... Even the blurry shots of the floor (Hey it might make a good background) Granted, CDROM's will be non-existant in 20 years.. but it's replacement will be here BEFORE it goes away.... so I transfer it... or my kids will or my grandchildren... Just like how I transferred my parent's and grandparents legacy media to current (Film, photos, Encode a Edison phonograph tube to mp3.... etc...)
It takes PEOPLE to make information survive... no magical device or media will.
Do not look at laser with remaining good eye.
I think that one of the biggest problems when it comes to archival is legal. Often, companies don't want their information archived. After they publish a product, they want it to sell, then just go away. This is the issue with abandonware. If a company releases a game, or program, then stop supporting it, they shouldn't stop people from archiving it. If people don't archive it, it will just dissapear. This is what many companies wan't, but is it really the best thing to have happen?
The biggest problem with maintaining archives may be that some people actually want thier information to just dissapear.
Floppy Disks.
Yes, They will still be 1.44 MB. They will still be included in all computers. They will still work slowly. But they're reliable! And they will still use FAT12..
*gag* isn't it time this particular media format died
It's one thing to lose technical data, but what about all that stuff that's much more personal and is (will be in 10+ years) sentimental? Things like (digital) baby photos, personal e-mails, etc.
How many people have grandparents who still have a box full of all of the letters they wrote each other when they were younger? OK, a few people might still write the occasional letter to each other, but this is really a thing of the past. And you can't compare the personal effort that goes into actually writing a letter with an e-mail. Just the fact that someone has actually gone to all the trouble to write the letter out makes it infinitely more satisfying when you read it.
How many people in (say) 20 years will have an actual photo album with real photos in it? How many people do you know now that have a photo album you can't view without turning on a computer?
It think it will be in 20+ years when the current digital-data generation are older that these things will really start to tell.
"Because it's there." - George Mallory, when asked why he wanted to climb Mt Everest, March 18, 1923 (New York Times)
Hardware isn't really a problem. Anything important can be put on a CD-ROM and preserved for eternity with some confidence; except that today the files may largely be in proprietary unpublished formats (e.g., just about any common format you use) that will take significant effort to read fully at an arbitrary point in the future.
The solution is straightforward and well underway, courtesy of the internet and WWW: published open data formats. The only reason for using a proprietary format these days is the effort that software makers put us through to do otherwise. Have you gotten tired of dismissing MS Word's objections to the use of RTF yet?
When we just say no to software that uses anything but open published formats, we'll get the software we need.
ThosEM
Certainly we will still have the data just as we now have the equivalent of all those floppies we had. Wether we can read it all or not is unimportant. What is important is wether we can read what we need to. Most such documents are proted across since they are regulary used. The problem will come when some historical documents are needed. As long as they exist somewhere, someone (Historical, Researcher, database programmer) will be able to look them up.
There was an article in Scientific America about bombing records from the Vietnam war stored in a mainframe being recovered to remove the bombs. Where there is a need there will be a way to seek and find the past. We know what the Egyptians ate for breakfast how much more will our anscestors know about us?
-- RTFM:Slackware::Beer:Saturday
You've got photojournalism people shooting digital because it's faster and offers some image structure advantages at high speed- no negatives to keep around for a 50 year retrospective.
And finally, you'll have the home consumer trying to back up all his photos to CD, organize them, and get thru the thousands upon thousands (note- most neg drawers aren't well organized either, but... ) of images that are labeled DCP_00389 or some otherwise useless name.
And then the hard drive crashes
And then it's gone.
I know a guy, a keen photographer who got his wife and kid out the house before the fire really took hold. He didn't get his twenty-odd year collection of thousands of slides and negs out though. This was in the Eighties before he could have afforded to get them all digitised to a high standard.
All he has now are a few prints and some contact sheets of all that work. His pics of his mother -- gone. Snaps of his beloved boyhood dog taken with his birthday-present first camera -- gone. Forever.
He still shoots film and slide, medium format too. He digitises *everything*.
Galileo is a lot more important than Marvin Minsky.
Huge chunks of what Minsky said are irrelevant. Books have been written mocking his 'pie-in-the-sky' dreaming. (See Herbert Dreyfus)
His important work has been published. His notes, just like the chips of rock that fell away when hierglyphs were carved in stone, need to just go away.
I know Minsky was just picked as an example, but the point is, the wheat has to be separated from the chaff, or all of everything gets lost in a sea of 'information-enthropy'.
And then the hard drive crashes
And then it's gone.
You know, I think in many ways it's good to loose stuff like this. Sure, it's upsetting for a while, but you get over it.
Memories are just that - in your memory, and whilst photos are good for jogging memories, that's all they do. For anyone who's not actually in the picture, they mean nothing. And really, it's far healthier to look to the future than reminiscing about past events. This might seem heartless, but how often do you actually look at 10-20 year old photos? Maybe with dead family members it's another matter, but if they were really close, you should be able to remember them without a photo.
And it's amazing how much crap you can assimilate over time. After I went travelling for a year with just a rucksack (two pairs of jeans, some T-shirts, a couple of pairs of shorts, etc...) I was horrified when I returned to realise how much junk I had in my parent's house that I'd previously considered important. Most of it went straight in the bin, as I sure as hell wasn't carting it to my next house.
Bringing it slightly back on topic. Yes, I've had hard disk failures. In one case, I even lost about a years worth of mail. But after being initially cross about my mail, I realised that I didn't actually need it anyway. The rest of the stuff I never even missed, as I'd backup up about the 5% that was useful.
For actual important stuff, like source code or documents, you just need to be disciplined enough to copy them somewhere reasonably regularly. I use local CVS for all my own source and just back up the whole tree every couple of days. I download stuff into a folder like '2002-07' for this month, and every month I backup anything to CD that is likely to be useful. Everything else can just be downloaded again, re-MP3'd, etc...
I'm just worried about how long my CD-R's will last...
Now, sure things are stored on HD's, but they are easly copied to new media... such as DVD-roms, etc. Any technology today has to be able to take data currently written to a HD.
But here comes "Digital Rights Management" or DRM. a hardware and software based double punch to our fair use rights. This is what could prevent us from making back-ups, keep us from moving to new forms of media.
It is the beginning of the digital dark age.
--T
http://www.theMediaBunker.com
Go to your public library - to a section of books of interest to you. Note the publication date of a dozen or so and whether the publishing company appears to be still in existance. Now imagine that these books had never been printed in book form but published only on digital media at the time, which was perhaps encrypted and perhaps, like Windows XP, even node-locked to a specific computer.
How many of the "books" would you still be able to read?
How many would you be able to read only by paying a company specializing in copying obsolete media to current media?
How many would you never be able to read without hiring a good "cracker" (whose efforts would probably be illegal under the DMCA)?
This is our future. Spooky, huh?
I think these folks misunderestimate the sheer volume of information we have collected about ourselves. Modern historians have been able to piece together a more or less complete history of the Greek and Roman worlds 2500 years ago using a few thousand written documents and archeological digs. We have more information than we can possibly process for every era of American history for at least 200 years back.
.01% will still probably dwarf the information we currently posess about the world 1000 years from now.
So yes, 99.99% of all information in existence today will probaly be lost 1000 years from now. The remaining
For starters, we still publish about as many books as any other society in history. There are books available on literally every topic available, and most of them have thousands of copies in circulation. So imagine that 99.9% of all books are nuked, chances are the majority of those books will still survive, and historians only need 1 copy to make use of it.
Finally, this article massively underestimates how easy it is to preserve digital information. 10 years from now, terrabyte hard drives will be commonplace, and no doubt second-generation DVD-R's will hold tens of gigabytes of data. All you have to do is copy those files en masse to the latest format every 10 or 20 years, and you've preserved the information. One person can do that in his spare time quite easily. Furthermore, file formats aren't *that* hard to reverse-engineer. Even if the world forgot what a Microsoft Word document looked like (which is extremely unlikely) they should be able to look at the raw data and figure it out well enough to at least read the plaintext. And I doubt we'll ever forget what ASCII means.
As for people losing their personal correspondance-- perhaps 99.99% of people will lose their email correspondance at some point in their lives. So in a nation of 300 million people, that leaves only 30,000 complete email correspondances for future historians to peruse. Imagine how much we'd know about Greek or Roman times if we had the complete correspondance of 30,000 average Greek or Roman citizens...
In conclusion, I think quite the opposite is true. Historians 1000 years from now will have more material than they can possibly process about the early 21st century. The trick will be in assimilating all that information into something useful, not finding enough to work with.
I wonder why Y2K didn't serve as a wake-up call? Maybe it's because basically nothing bad happened? Yes, it cost a ton of money to correct the problem, but there were no huge catastrophes like segments of the media had predicted.
In the same way, yes, hard drives will crash, and people will lose stuff. But this is nothing new! The idea of a "digital dark age" where hard drives start crashing left and right, and history starts going down the drain, is absurd. It ranks up there with the pre-Y2K hype about society crashing and people roaming the streets in search of food. But hey, your story is a success if people will read it and take the hype to heart, right?
"I am a cipher, a cipher, wrapped in an enigma, smothered in secret sauce" -Jimmy James
If a historian found a bean mix receipe from 300 years ago that a regular person from New England ate he would shit his pants in excitement. The list of ingredients would tell the historian a huge amount about what the average person of the time would have access to. Most of us could give a crap what kings ate, kings had things shipped to themselves from everywhere in the world, they were rich. Most historians care about what the normal people ate, and how they lived and want to know everything about those people.
Imagine if vanilla was on the list of ingredients? This would tell us that New England had regular trade with south america. Imagine that an item was listed with several substitutions... This would tell the historian that some items were not available all the time.
This is just stupid. This is the kind of story I would expect on AP or Reuters.
Only the most expensive books were produced on acid-free paper. They had a shelf life of maybe a couple hundred years, at best. Witness a book a friend of mine loaned me (mcgrew.info/gem). Yellowed and crumbling with the first nine pages missing, this is afaik the only copy of this book in existance.
Except the one I am posting on my hard drive and the internet. This book was in grave danger, but as soon as it is posted it is more or less permanent.
Warantee of one year? WTF, don't any of you back up your data?
Well, data backups are easy now. Fifty years ago data were stored in filing cabinets. All it took was a fire in a single facility to destroy decades of records, like the fire in St. Louis in the early 1970s that destroyed the military records of thousands of WWII vetrans.
No, computers aren't endangering our heritage, they can protect it. What is endangering our heritage is digital rights management. Lock up your art and throw away the key... real smart move, guys.
Steve
theFragfest.com
The biggest problem is finding an 8086 machine on which I can still play digr. Go ahead. Practice juggling with those 80GB western digitals. They are nearly indestructable. The data? We'll find it.
I've heard this complaint so many times and it just doesn't ring true.
If digital storage was like paper storage this would be an issue but the truth is digital storage is unique in 2 ways:
1. You can make infinite perfect copies
2. The storage capacity grows exponentially over time.
I still have papers I wrote 15 years ago. The 20 Meg 5.25" harddrive that they were originally stored was trash 10 years ago along with 3 or 4 other drives that they lived on over the years and yet my papers remain. They remain because I wanted to keep them (and I'm good about protecting my data.) They are on a completely different filesystem (EXT3) on a completely different operating system and yet I can still get to them, read them and print them out. They are now on a RAID 5 array that is backed up to a separate drive with all my other important data.
In the article he states about physical things "Mostly, stuff lasts". That is just not true. How many of those documents that we printed out back in the early 90's before everything was email based are still around? I know several people who have all their email going back 5-10 years. It's simply much easier to keep digital stuff around.
Most people upgrade to a new machine and bring their data over with them. The drives fail but the files that people care about stay. Crashes can be devastating and people certainly do lose data but the same thing can be said about fire in the physical world. Keeping 2 digital copies of important stuff makes it hard to lose it. If you lose one copy, make another one. The odds of losing both before you can make a new copy are very slim.
It's also much easier to keep digital things organized and search through them.
I think digital things in general will always have better lasting power than paper things. Internet based backup services will make this much more so in the coming years. For a few dollars a year you can have all your important files stored somewhere off site on redundant media. Try doing that with paper?
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
My concern remains strong, however. For every tape that was saved and rescued by TUHS and by your own stellar recovery abilities (which I am grateful for, by the way), how many have been lost? And what if, god forbid, trailing-edge.com goes down in five years, or ten? There may be mirrors if we're lucky, maybe, and some people will have copies of the tapes they've downloaded, but how will we find them? Poof, it can vanish all too quickly. And those original tapes are already in hard shape, some portion of them will be completely unreadable in ten years, and we can't say which portion that will be.
For the most part, I think that TUHS and the PDP10 archives have done so well because of the efforts of a few hardcore packrats. Most of the Slashdot readers who have so casually poo-pooed this article are the same sort of person, myself included. We save everything. We do backups. We feel like we could restore our computers after a crash.
But that viewpoint is so short term. What happens when we die? (no, i'm not discussing theology here!) What becomes of our computers and our scattered tapes labeled "/usr (dump) 1994-02-12"? Will we have digital executors who look after it for us? I somehow doubt it.
The argument can be made that most of the lost information is unimportant, but I'm not sure I buy that either. A lot of it may well be. A lot of it will be accumulated junk a future society can live just fine without. But it is impossible to know what will be and won't be important in the future. You really never know. And while I don't think we want to save every single bit of information ever created, we should at least do ourselves the service of trying to come up with a better solution than just trusting everything to work itself out in the end. There's no harm in thinking about it a little, people.
Not entirely. 20 years ago, perhaps 30 by now, we wrote a bunch of specialized census information onto 556 BPI 7-track odd parity tapes, and some onto 556 BPI 7-track even parity tapes. And some tapes that were mixed mode, with specialized software to read them. The IBM 7094 goes away, and we switch to an emulator running on a 360. Slowly, and without much plan, we start switching over to programs that run native on the 360. Finally there's OS change, and the emulator goes away (i.e., we aren't willing to pay the service bureau enough to keep it's license current). Some of the tapes haven't been converted yet, but that's no problem. 7-Track tapes are a long established standard, and everyone has a bunch of drives, even though the new 9-Track drives can't read them. Put the tapes into storage. Fast forward a decade. Lots of the documentation has been lost, but surely we could read them if we needed to. Another decade .. it turns out that tapes become unreadable if left to themselves even in a temperature controlled vault, we'd better pull them out an check, probably copy them all over. But where do we find a 7-track tape drive? There are a few places, but nobody even half-way close. And they're expensive. And we don't really know for sure that we can read the tapes. And ... we dither. But we aren't really paying much attention to the problem either, we just aren't deciding what to do, so we keep the tapes in storage while the number of 7-track tape drives dwindles, and the magnetic domains become weaker, and the documentation becomes sparser....
So when it comes time to do a time series study, 1960 doesn't get included. Nobody knows how to get at the information. Or whether or not it even still exists.
There may be legal problems, but there are also both organizational and technical problems. And they are all significant. In this case all of the factors would have needed to cooperate to get the problem solved. And to maintain their cooperation over time.
And we still don't know how important the loss of that data was. We may never know. It could have been worth multiple millions, or nothing. We can't even tell. So everyone is just ignoring the event, because it's too uncomfortable to think about. And while we ignore it, there are the tape cartridges from an IBM 3330 that are sitting around in storage, because somebody wanted them cleared off his desk. And that kind of tape cartridge was only in use for a few years, and was never widely popular. Nobody knows what's on those cartridges, but it probably isn't as important as the census data might have been. And it's probably unreadable too. And I have a box of 5 1/4 single density floppies that have the original source code for one of our major projects. If there is a version that got converted, I don't know where it is. And I don't have a 5 1/4 inch drive. When I got them, I has a Mac (made great sense to give them to me, huh?), and by the time I was coerced into a PC, the PCs only had 3 1/2 inch drives. So it never made sense for me to have them, and I don't even use the project. But I have the only copy that I know about. Maybe it won't be important.
Data is already evaporating right and left. I see it happening every day. Most times it doesn't matter much, but you can't always tell at the time. And often the reasons that it evaporates are technical. And organizational. Legal problems are rarely the issue, though they can be in unusual circumstances, like proprietary software that the company stops maintaining for some reason (like going out of business).
I think we've pushed this "anyone can grow up to be president" thing too far.
And I find it arrogant that you assume my information won't be valuable. The inane babblings of the dominant cultural leaders of a time are not nearly as useful to archaeologists as the information left behind by common individuals. The people who write the record don't accurately represent the lives and spirits of average people.
I think we have an opportunity with technology to preserve more than the party line, the "fiction agreed upon" by history's victors.
You say something to the effect that if your loved ones are all that important, you should be able to remember them without a picture.
But even if this were so, how do you show your child what his granddad looked like, who died before your child was born??
The point of archiving data is not just so YOU can remember it. It's so people who had no chance to see it firsthand can also get a look at how things were (regardless of the sort of data it is).
~REZ~ #43301. Who'd fake being me anyway?
Digital Data is the most fluid data storage system ever created. If information is truly important it will transverse from storage system to storage system as the systems change. When I got a computer I typed my documents on word processors and stored them on floppies. When hard drives came out I copied the floppies to the hard drive. When cd's came out i burned my harddrive files to cds. When DVD's come out I burned the CD's to DVD. The rate of growth of the storage medium is great enough that no data need be lost. If its extremely important....have backups...duh.... And as far as people dying. Since when does being dead make your password unhackable???.... With the future of storage medium heading towards holograms and other futuristic storage mediums I don't forsee a loss of any truly important data. And there's a lot of data that doesn't truly need to be kept....just like my garage acumulates junk I no longer need.....
We've secretely replaced the Enterprise's dilithium crystals with Folgers crystals. Lets see if they notice.
I mean, not to flame this guy, but his mom loses some email and suddenly there's going to be a time where all digital information stored on hard drives is lost?
Jesus, it's not like every hard drive on the planet is going to die simultaneously at an unknown future date....and in the meantime, new hard drives are manufactured and new storage media ara invented, did it ever occur to him that people might migrate their data along the way?
Horrible, horrible article.
This message brought to you by the Council of People Who Are Sick of Seeing More People.
Is it?
Is this nation really under God?
Does the nation follow the laws of God?
We have the death penalty in America. God said: Thou shalt not kill.
We punish the guilty. God says: forgive on another.
and also, God says: Vengence is mine.
Are we misleading people about the true nature of God and His commands to us?
God, if he were here, now, in the flesh, as they say, would probably go to the temple and whip the moneychangers (accountants).
"Piter, too, is dead."