Slashdot Mirror
Digital Dark Ages?
angkor writes "The digital dark age--Will all the information from this computer age slowly vanish as our delicate hardrives expire? That's what it looks like. Better start printing everything out."
← Back to Stories (view on slashdot.org)
Anything that's worth backing up has already been backed up on tape.
You honestly don't think that the contents of your hard drive have any sort of historical importance, do you?
Just because you've saved every free pr0n pic you've ever downloaded and categorized them neatly doesn't mean that some future archeologist is going to find them interesting. I can find them useful immediately. Please send any such collection to me at my hotmail address. Thank you.
I have been pwned because my
Install a web server, publish everything you have, then let Google cache it...
We probably will enter some sort of digital dark age eventually. I mean, there aren't an infinite number of hard drives in existance. And one day they may start manufacturing only hard drives with hardware DRM in them. Then, one day when the last of the non-DRM hard drives are crashing, we'll either have to not use hard disks (maybe there'll be something new), or get new DRM hard drives. This is actually my one doubt about serial ATA, which otherwise sounds awesome. Can anyone confirm whether or not serial ATA has DRM or not?
The GeekNights podcast is going strong. Listen!
Does this include getting your server slashdotted in record time??
They really should warn the people that they are going to be posting a link to their server, and that extremely heavy traffic will arrise.
This has been bantered about by practically everyone in any sort of media outlet. You've got librarians trying to figure out how to store all of the supposed 'research' that exists out here. Journals are going out of print because they can publish faster and easier on the web.
;P
You've got photojournalism people shooting digital because it's faster and offers some image structure advantages at high speed- no negatives to keep around for a 50 year retrospective.
And finally, you'll have the home consumer trying to back up all his photos to CD, organize them, and get thru the thousands upon thousands (note- most neg drawers aren't well organized either, but... ) of images that are labeled DCP_00389 or some otherwise useless name.
And then the hard drive crashes
And then it's gone.
Nothing will change until this starts happening. Give it 3 to 5 years, or however long it takes joe and Jane to upgrade their computers and start losing stuff. Then some sense will get back into the world
...is Souls in the Great Machine by Sean McMuller which looks at a world where all computerized records are wiped out in a great war. They are awash in information but can not read any of it, and thus are reduced to a 1600s to 1800s-style society. Good reading and a good point worth considering.
But the only reason these archives can be built and maintained is that it is legal to do so, thanks to the hard work of preservationists like Bob Supnik (see his SIMH "old iron" simulation packages) and Warren Toomey who have secured such licenses. Without such permission, many other archives of historical software that I've assembled myself cannot be distributed to the rest of the world.
Maybe it's just me, but whenever it looks like a harddrive is about to die (funny noises, etc. or just getting old) we replace it before it does. Also, we back up critical information, often in more than once place. This sort of practice should, in thoery, prevent this from happening. These things are replacable.
The snow doesn't give a soft white damn whom it touches. -- ee cummings
A number of posters have noted that most people have little of importance on their hard drives. I'm not so sure. One of the trends in historical research has been to refocus analysis on the lives of ordinary people. As it turns out, this is a problem since ordinary people didn't tend to write in the public record. Often, things that were incredibly popular are virtually undocumented because no one thought them important enough to preserve.
Let me offer one example. When historians want to document the impact that computers and the "information revolution" had on people's lives, there's only so much value in the Wired archives, for example. How did everyday people (not e-publishers or the digital literati) interact with machines and each other? This kind of research depends on many small bits of information, and if there is sytematic bias in which (or whose) information gets preserved then research will inevitably be limited by that bias. In short, don't underestimate the value of large numbers of seemingly unimportant documents.
This raises the question: what can be done to preserve the electronic record created by everyday users? Is any preservation medium cheap and easy enough to become ubiquitous in off-the-shelf systems?
Make cheese not war 8:)
http://www.penny-arcade.com/view2002-07-01rl.html
We do not live in the 21st century. We live in the 20 second century.
No, if your data really has value, carve it in clay and burn it. Or carve it in stone. While those methods are still not completely safe, they are at least reasonably safe.
Given the amount of data to store, we should probably build pyramids again, and carve our data into the stones of the pyramids. Given how long the Egypt pyramids lasted, this seems like a really secure way of storing the data.
Of course, I don't want to be an archaeologist in a few thousand years trying to decipher those strange texts e.g. inside the Linux Kernel Pyramid...
The Tao of math: The numbers you can count are not the real numbers.
ignorance, suppression, warfare, famine, strife
;)
Sounds like a fit description of the Msft dominated computer industry alright.
Fact is, that's just what the comp industry WANTS - the old name is 'planned obsolence', nothing, very little anyway, is built to last. At best it's made to last 3 years then you thro it away and buy another. Gotta keep them customers spending $$$!
A co-worker was talking about archiving his ancient family photos with a scanner and CD writer - I told him if he's lucky within a generation a descendant or relative will take up the job of transfering them from CD to holographic crystals or whatever is the format du jour at the time. Just like the DNA code is recreated every generation.
I print out ALL online transactions involving $$$, just in case there's a dispute
try { do() || do_not(); } catch (JediException err) { yoda(err); }
This is exactly the kind of problem that Danny Hillis and the The Long Now Foundation have been pointing out for years. Digital data doesn't last.
"Science historians can read Galileo's technical correspondence from the 1590s but not Marvin Minsky's from the 1960s."
That's why they started the 10k year library project. A part of this project that interests me especially is the Rosetta Project. It's a "near permanent archive of 1,000 languages". It's still a work in progress, so I hope they succeed. In my eyes it's definitely a worthwhile endeavour.
siener's youtube channel
I was under the impression that the defining characteristics of the dark ages was ignorance,
Witness George W. Bush, the Senate, the House and 50% of the US population.
suppression,
Witness DMCA, PATRIOT, RIAA etc.
warfare,
Witness the War on Drugs, War against Terrorism, War against Poverty not to mention all the real wars and civil uprisings around the world.
famine,
Witness Africa.
strife
Witness MS vs GPL, RIAA and MPAA vs Consumers etc
you know, BAD STUFF
Witnes Hilary Rosen and Jack Valenti. Now - picture them in an XXX-rated movie.
We do not live in the 21st century. We live in the 20 second century.
Try one of these for your data archiving. No software dependencies, long media life, etc.
For shift.com the dark age has already begun... ./ effect
The solution to both saving ancient works on paper can work just as well for digital media. Keep copying the work to the latest storage media! None of the original texts that we do have have survied. They are all copies made from generation to generation. Thus with digital media. The best of the web (lets say, research articles) will be preserved and transferred to new storage media as it develops. Your blog about your day at the beach prolly won't.
"Overhead, without any fuss, the stars were going out."
There are several ways this could go. Obviously, we have to be circumspect, since the U.S. gov't is literally considering copy-control legislation that would make Linux illegal.
You can say it'll never succeed - won't all Linux's rich patrons prevent it? But I would have said the same about quite a few other things that have already happened... and it's in our interests to act as thought it might.
However, assuming something slightly less than the worst, DRM will of necessity be something which you can enable or not. IOW, as long as they'll let you, buy all the fast, new DRM drives you want, and use Linux to run them. Linux will simply ignore the DRM features and use the drive normally.
The problems come when you're forced to use a DRM operating system with your DRM hardware (quite a reversal from the old antitrust days, eh?); you will find it very difficult to take some/all of your data back to Linux/other non-DRM OS.
You can probably see why MS loves this now; DRM technologies, even optional ones, will have the nice effect of preventing interoperability with open source operating systems, thereby locking everyone in even further. Let alone the myriad other possibilities for abuse, censorship, and bottlenecking...
If we allow our government to do this, both in the context of MS's current status as a monopolist, and in the ongoing (anti-) regulation of the media industries, we are doing the gravest disservice to future generations.
We're on the road to Tycho.
Given the propensity of M$ & others to use proprietary file formats in an effort to lock in the client base and to lock out competition. (And don't tell me about standards like because XML [tagged data storage & transport streams] without DTD [document tag definitions aka data context] is pretty damn useless [the difference betweeen data & information.])
I have quite a few files that I can no longer access except as raw byte streams because the applications that created them no longer exist or because the meta data information that controlled that creation is no longer available.
Even printing sh.., uh, stuff, out is pretty useless because most paper is acid based and turns to ash over a very short time. The inks are not much better.
I have books printed in the 17th century that are still quite readable (high rag content acid free paper,) and a 1901 Sears catalog (acid washed wood pulp paper,) that I accidentally put my thumb through in the late '80s.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
> and you don't have to make annoying backups everytime because of this fact.
;-)
This assumes that only one drive in the array will fail at a time, and between complete verified drive rebuilds. The Raid 5 drive arrays I've seen put together are usually built from a group of new drives, all the same drive model all purchased at the same time. I've seen enough bad production runs for various hard drives to know that it is _too_ easy to get stuck with a group of lemons.
Now imagine a lemon fails. You slap in the replacement, and think all is well, you order another hot-swappable replacement. While it's on the way, two more drives fail. To use a quote in backdraft, that little blinking light in the corner of your vision is your career dissipation light, and it just went into overdrive.
The following additional situations make me think offsite, up-to-date backups are still a VERY good thing:
- Lightning strike or massive power surge
- Water damage (pipe breaking?)
- Drop-damage (well, actually it's the sudden stop)
- Fire (I'm sure SOME companies have a Milton working for them)
- Earthquake
- Tornado
- Hurricane
- People unexpectedly parking their vehicles in your building, violently.
- Pissed off employees with physical or electronic access to the data
- Theft/burglary
And let's not forget good old human nature. "Oops, I didn't mean to delete that..."
"He who laughs last usually had a VERIFIED backup."
first, we need to think logically.. Every bit of information we have discovered that is aincent was discovered by sheer luck and accident. NOONE back in 985 BC set aside the stone tablets thinking that "someone will want to read this in 3000 years. EVERYTHING we find out about the past has been accidental. Nothing has ever been intentional archives preserved for the distant future.... If there were we might have a whole bunch more knowledge than we do today. (we re-invent things every 50 years.. because we lose how it was done 100 years ago.. My great grandfather's workshop was filled with things that were over 100 years old yet I have seen marketed today as "A TOOL BREAKTHROUGH! The Self Ajdusting wrench!")
I take EVERY digital photograph I shoot and burn it to CDROM. nothing ever get's deleted in my photography.... Even the blurry shots of the floor (Hey it might make a good background) Granted, CDROM's will be non-existant in 20 years.. but it's replacement will be here BEFORE it goes away.... so I transfer it... or my kids will or my grandchildren... Just like how I transferred my parent's and grandparents legacy media to current (Film, photos, Encode a Edison phonograph tube to mp3.... etc...)
It takes PEOPLE to make information survive... no magical device or media will.
Do not look at laser with remaining good eye.
Floppy Disks.
Yes, They will still be 1.44 MB. They will still be included in all computers. They will still work slowly. But they're reliable! And they will still use FAT12..
*gag* isn't it time this particular media format died
Hardware isn't really a problem. Anything important can be put on a CD-ROM and preserved for eternity with some confidence; except that today the files may largely be in proprietary unpublished formats (e.g., just about any common format you use) that will take significant effort to read fully at an arbitrary point in the future.
The solution is straightforward and well underway, courtesy of the internet and WWW: published open data formats. The only reason for using a proprietary format these days is the effort that software makers put us through to do otherwise. Have you gotten tired of dismissing MS Word's objections to the use of RTF yet?
When we just say no to software that uses anything but open published formats, we'll get the software we need.
ThosEM
Actually, historically, a "Dark" age (there have been several... the so-called "Dark Ages" is merely the longest series of them in Medieval times) is a period of time *during* recorded history when the historical record is in pieces or non-existant. While other problems can be applied to a Dark Age, these are usually causes, but what defines a Dark age is the result: reduced historical record.
There were 2 or 3 in the Roman empire, one that I believe lasted about 30 years. Several more cropped up before and after Charlemagne. A much smaller one is happening with books produced in a specific timeframe in the early 20th century (I disremember which). Because of the acid in the paper, they'll deteriorate and fall apart rapidly. Luckily, project gutenberg is making an effort in getting the info out of books this old.
So, it's OK to be wrong.
Quite contrary to this story, the advent of digital data storage and the Internet have led to something never before possible in the history of mankind: near instantaneous massive duplication. It is now possible for digital data to be copied effortlessly and transferred all over the globe. The trick, is doing it.
.zip format.
Our data storage needs have kept pace with data storage ability for some time now. I don't see this ending anytime soon. But it might, eventually. It stands to reason that there will come a time when we will have a want of things to store for all the space we have. I don't count on it in my lifetime, but it could happen.
The trick, then, is getting the data from here to there. How do we do it?
1. The written word is still the most important medium of human communication. Project Gutenberg is doing a bang-up job of digitizing AND distributing written works, and this is a project we should all support. I would also like to see a similar project with scientific journals being digitized (if not already) and widely distributed to universities, who can host them publicly or privately.
2. Someone suggested CDs, but these are impractical. CD-r's have a shelf life of 100 years, and CD-RW has even less. These could work as storage medium, but you would have to be diligent in keeping them up-to-date. What we really need is a physical storage method (like CDs) that have the capacity of magnetic storage media, like HDs.
3. Open file formats. It stands to reason that computers will always understand ASCII (or possibly UNICODE) text. It would not be difficult to append text-only information to the end of even very complex documents, that could be retreived even if the file format itself was no longer known. xml-based file formats do this to a degree, but it depends on the universitality of the
4. All of this is useless if we ourselves are not diligent in keeping up with our digital information. In the Middle Ages, copying an old, worn-out parchment or scroll could take weeks, even months. Now it's possible to do it in a fraction of a second, so there's no reason we shouldn't.
I currently keep my important data (emails, writings, website) in the following locations: My hard drive, a backup file on another hardrive, a CD-RW, a CD-R (which I change/update every six months or so) The server at my school, and the my webserver which is offsite. I personally would like to see off-planet massive storage, but until storage space exceeds storage demand, we will always be faced with the question of "What is important enough to backup?"
My most important data on my computer is the pictures from my digital camera. Right now I'm keeping one copy of all the pictures on my hard drive, and as I take more pictures everytime I get ~650 megs worth I burn them onto a CD backup as well. I'd really like to be able to take them off of my hard drive to free up space, but then I hear that CDRs have been known to fail, which would be incredibly upsetting for me. Worse yet would be going back after a couple years have passed and finding that the CDRs have died with age. Of course the worst case scenario would be having my hard drive die in a couple of years, and go back to the CDRs only to find that they died at some unknown point in the past.
As such, does anyone have any recommendations for average people like me out there who have data that is very important to them, but for whom corporate measures like commercial data backup services just aren't practical? Is there a better practice I can do than what I'm doing already? How about specially designed long life CDRs? Does such a thing exist?
Think about it. 98% of what's out on the web is crap. The stuff that's really valuable get's copied, in general. People do mirrors, or download pages. I doubt much of real value will be lost in the long run. I mean, geez, I'm going to be really bummed when my porn collection goes bad, but I downloaded it from others, so it's still out there somewhere.
With our rapidly increasing HD sizes, backup methods and media aren't keeping up. I've already lost 2 large HD's in the last 2 years, and with my shiny new 80 Gig drives, I've got a Raid-1 setup, but still if they both fail within a short amount of time from each other, I'm outta luck.
Moreover, the advancement of HD tech makes it almost certain that when one fails in a year, I won't be able to get an exact replacement to reload it from the RAID.
Does anyone know of a PRACTICAL way to back up 80 Gig's of info? AHSay.com offers online backups, but the initial backup would take weeks through my ADSL modem, and then incrementals would be pretty much useless. I suppose I could use DVD-RW, but at 4.7 Gig a disk, we're talking 20ish disks, at several hours a piece. And doing incremental backups that way is a nightmare. It seems that my only real option is to use something like a MonsterTape backup storage device, but systems with 80Gig capacities and up START at $4000 a piece, and the tapes are 80 bucks a piece. With 80 gig drives available for $129 bucks (Pricewatch), it doesn't seem like a good option.
The Dopester
"Yes, I'm a Karma Whore, but I'm doing it to pay my way through school."
1,000 years - is that long enough? We have parchments that are 5,000 years old, we need to match or even exceed that. If civilisation is to come to a thundering catastrophic end, it might not get back up to our level of technology (sufficient to read the disks) for 10,000 years. this is a little better, but I'd like a bit more still.
And then the hard drive crashes
And then it's gone.
You know, I think in many ways it's good to loose stuff like this. Sure, it's upsetting for a while, but you get over it.
Memories are just that - in your memory, and whilst photos are good for jogging memories, that's all they do. For anyone who's not actually in the picture, they mean nothing. And really, it's far healthier to look to the future than reminiscing about past events. This might seem heartless, but how often do you actually look at 10-20 year old photos? Maybe with dead family members it's another matter, but if they were really close, you should be able to remember them without a photo.
And it's amazing how much crap you can assimilate over time. After I went travelling for a year with just a rucksack (two pairs of jeans, some T-shirts, a couple of pairs of shorts, etc...) I was horrified when I returned to realise how much junk I had in my parent's house that I'd previously considered important. Most of it went straight in the bin, as I sure as hell wasn't carting it to my next house.
Bringing it slightly back on topic. Yes, I've had hard disk failures. In one case, I even lost about a years worth of mail. But after being initially cross about my mail, I realised that I didn't actually need it anyway. The rest of the stuff I never even missed, as I'd backup up about the 5% that was useful.
For actual important stuff, like source code or documents, you just need to be disciplined enough to copy them somewhere reasonably regularly. I use local CVS for all my own source and just back up the whole tree every couple of days. I download stuff into a folder like '2002-07' for this month, and every month I backup anything to CD that is likely to be useful. Everything else can just be downloaded again, re-MP3'd, etc...
I'm just worried about how long my CD-R's will last...
Now, sure things are stored on HD's, but they are easly copied to new media... such as DVD-roms, etc. Any technology today has to be able to take data currently written to a HD.
But here comes "Digital Rights Management" or DRM. a hardware and software based double punch to our fair use rights. This is what could prevent us from making back-ups, keep us from moving to new forms of media.
It is the beginning of the digital dark age.
--T
http://www.theMediaBunker.com
I think these folks misunderestimate the sheer volume of information we have collected about ourselves. Modern historians have been able to piece together a more or less complete history of the Greek and Roman worlds 2500 years ago using a few thousand written documents and archeological digs. We have more information than we can possibly process for every era of American history for at least 200 years back.
.01% will still probably dwarf the information we currently posess about the world 1000 years from now.
So yes, 99.99% of all information in existence today will probaly be lost 1000 years from now. The remaining
For starters, we still publish about as many books as any other society in history. There are books available on literally every topic available, and most of them have thousands of copies in circulation. So imagine that 99.9% of all books are nuked, chances are the majority of those books will still survive, and historians only need 1 copy to make use of it.
Finally, this article massively underestimates how easy it is to preserve digital information. 10 years from now, terrabyte hard drives will be commonplace, and no doubt second-generation DVD-R's will hold tens of gigabytes of data. All you have to do is copy those files en masse to the latest format every 10 or 20 years, and you've preserved the information. One person can do that in his spare time quite easily. Furthermore, file formats aren't *that* hard to reverse-engineer. Even if the world forgot what a Microsoft Word document looked like (which is extremely unlikely) they should be able to look at the raw data and figure it out well enough to at least read the plaintext. And I doubt we'll ever forget what ASCII means.
As for people losing their personal correspondance-- perhaps 99.99% of people will lose their email correspondance at some point in their lives. So in a nation of 300 million people, that leaves only 30,000 complete email correspondances for future historians to peruse. Imagine how much we'd know about Greek or Roman times if we had the complete correspondance of 30,000 average Greek or Roman citizens...
In conclusion, I think quite the opposite is true. Historians 1000 years from now will have more material than they can possibly process about the early 21st century. The trick will be in assimilating all that information into something useful, not finding enough to work with.
I wonder why Y2K didn't serve as a wake-up call? Maybe it's because basically nothing bad happened? Yes, it cost a ton of money to correct the problem, but there were no huge catastrophes like segments of the media had predicted.
In the same way, yes, hard drives will crash, and people will lose stuff. But this is nothing new! The idea of a "digital dark age" where hard drives start crashing left and right, and history starts going down the drain, is absurd. It ranks up there with the pre-Y2K hype about society crashing and people roaming the streets in search of food. But hey, your story is a success if people will read it and take the hype to heart, right?
"I am a cipher, a cipher, wrapped in an enigma, smothered in secret sauce" -Jimmy James
I've heard this complaint so many times and it just doesn't ring true.
If digital storage was like paper storage this would be an issue but the truth is digital storage is unique in 2 ways:
1. You can make infinite perfect copies
2. The storage capacity grows exponentially over time.
I still have papers I wrote 15 years ago. The 20 Meg 5.25" harddrive that they were originally stored was trash 10 years ago along with 3 or 4 other drives that they lived on over the years and yet my papers remain. They remain because I wanted to keep them (and I'm good about protecting my data.) They are on a completely different filesystem (EXT3) on a completely different operating system and yet I can still get to them, read them and print them out. They are now on a RAID 5 array that is backed up to a separate drive with all my other important data.
In the article he states about physical things "Mostly, stuff lasts". That is just not true. How many of those documents that we printed out back in the early 90's before everything was email based are still around? I know several people who have all their email going back 5-10 years. It's simply much easier to keep digital stuff around.
Most people upgrade to a new machine and bring their data over with them. The drives fail but the files that people care about stay. Crashes can be devastating and people certainly do lose data but the same thing can be said about fire in the physical world. Keeping 2 digital copies of important stuff makes it hard to lose it. If you lose one copy, make another one. The odds of losing both before you can make a new copy are very slim.
It's also much easier to keep digital things organized and search through them.
I think digital things in general will always have better lasting power than paper things. Internet based backup services will make this much more so in the coming years. For a few dollars a year you can have all your important files stored somewhere off site on redundant media. Try doing that with paper?
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
(the only thing worse than a spelling mistake in a post is a mistake in the subj:)
When I was in highschool, a friend of mine gave me a picture of her in the park. She was off center and some guy was in the background. Several times I considered taking scissors and cropping that guy out. After all, I didn't know him and he wasn't nearly as cute as she was. Fast forward a few years, and I'm scanning my pics and posting them to my site, and I see the picture of her. Only this time, I recognize the guy in the background. He's a friend of mine now. So you never know what'll be important or interesting later, and you don't always need to wait a few hundred years for your perception to change.
jred
I'm not a mechanic but I play one in my garage...
You say something to the effect that if your loved ones are all that important, you should be able to remember them without a picture.
But even if this were so, how do you show your child what his granddad looked like, who died before your child was born??
The point of archiving data is not just so YOU can remember it. It's so people who had no chance to see it firsthand can also get a look at how things were (regardless of the sort of data it is).
~REZ~ #43301. Who'd fake being me anyway?
Well, it's a new definition for "dark ages", that's for sure.
I was under the impression that the defining characteristics of the dark ages was ignorance, suppression, warfare, famine, strife -- you know, BAD STUFF.
Actually, the period we call the "Dark Ages" is a period for which we have few written records. It's only 'dark' because we can't 'see' what was happening back then.
Mike van Lammeren
It will challenge your head, your brain, and your mind.
I mean, not to flame this guy, but his mom loses some email and suddenly there's going to be a time where all digital information stored on hard drives is lost?
Jesus, it's not like every hard drive on the planet is going to die simultaneously at an unknown future date....and in the meantime, new hard drives are manufactured and new storage media ara invented, did it ever occur to him that people might migrate their data along the way?
Horrible, horrible article.
This message brought to you by the Council of People Who Are Sick of Seeing More People.
But would they have be so vulnerable and without leadership if half their uper class kids were not retarded from cumulative lead poisoning ? We spent nearly a week debating this point in history class....This and maybe the popes' failure to allow Edward8 to have a divorce may possibly be 2 of the biggest turning points in history.
errr....umm...*whooosh* *whoosh* Is this thing on ?