Slashdot Mirror
Digital Dark Ages?
angkor writes "The digital dark age--Will all the information from this computer age slowly vanish as our delicate hardrives expire? That's what it looks like. Better start printing everything out."
← Back to Stories (view on slashdot.org)
We probably will enter some sort of digital dark age eventually. I mean, there aren't an infinite number of hard drives in existance. And one day they may start manufacturing only hard drives with hardware DRM in them. Then, one day when the last of the non-DRM hard drives are crashing, we'll either have to not use hard disks (maybe there'll be something new), or get new DRM hard drives. This is actually my one doubt about serial ATA, which otherwise sounds awesome. Can anyone confirm whether or not serial ATA has DRM or not?
The GeekNights podcast is going strong. Listen!
This has been bantered about by practically everyone in any sort of media outlet. You've got librarians trying to figure out how to store all of the supposed 'research' that exists out here. Journals are going out of print because they can publish faster and easier on the web.
;P
You've got photojournalism people shooting digital because it's faster and offers some image structure advantages at high speed- no negatives to keep around for a 50 year retrospective.
And finally, you'll have the home consumer trying to back up all his photos to CD, organize them, and get thru the thousands upon thousands (note- most neg drawers aren't well organized either, but... ) of images that are labeled DCP_00389 or some otherwise useless name.
And then the hard drive crashes
And then it's gone.
Nothing will change until this starts happening. Give it 3 to 5 years, or however long it takes joe and Jane to upgrade their computers and start losing stuff. Then some sense will get back into the world
...is Souls in the Great Machine by Sean McMuller which looks at a world where all computerized records are wiped out in a great war. They are awash in information but can not read any of it, and thus are reduced to a 1600s to 1800s-style society. Good reading and a good point worth considering.
But the only reason these archives can be built and maintained is that it is legal to do so, thanks to the hard work of preservationists like Bob Supnik (see his SIMH "old iron" simulation packages) and Warren Toomey who have secured such licenses. Without such permission, many other archives of historical software that I've assembled myself cannot be distributed to the rest of the world.
A number of posters have noted that most people have little of importance on their hard drives. I'm not so sure. One of the trends in historical research has been to refocus analysis on the lives of ordinary people. As it turns out, this is a problem since ordinary people didn't tend to write in the public record. Often, things that were incredibly popular are virtually undocumented because no one thought them important enough to preserve.
Let me offer one example. When historians want to document the impact that computers and the "information revolution" had on people's lives, there's only so much value in the Wired archives, for example. How did everyday people (not e-publishers or the digital literati) interact with machines and each other? This kind of research depends on many small bits of information, and if there is sytematic bias in which (or whose) information gets preserved then research will inevitably be limited by that bias. In short, don't underestimate the value of large numbers of seemingly unimportant documents.
This raises the question: what can be done to preserve the electronic record created by everyday users? Is any preservation medium cheap and easy enough to become ubiquitous in off-the-shelf systems?
Make cheese not war 8:)
This is exactly the kind of problem that Danny Hillis and the The Long Now Foundation have been pointing out for years. Digital data doesn't last.
"Science historians can read Galileo's technical correspondence from the 1590s but not Marvin Minsky's from the 1960s."
That's why they started the 10k year library project. A part of this project that interests me especially is the Rosetta Project. It's a "near permanent archive of 1,000 languages". It's still a work in progress, so I hope they succeed. In my eyes it's definitely a worthwhile endeavour.
siener's youtube channel
Quantum encryption seems like a good solution...they're trying to mathematically prove that it's unbreakable.
Try one of these for your data archiving. No software dependencies, long media life, etc.
Perhaps IBM's new storage technology, Millipede, could help stave off the impending "Digital Dark Ages".
0 61 1_millipede.shtml
Millipede is such an incredible technology not only because of its ultra-high density, but because the data actually exists in a physical form, albiet on an incredibly tiny scale, unlike current hard drives, which just toss around magnetic charges. Magnets don't last forever, but you seal up a polymer film in a metal case, and it'll last pretty much forever.
IBM dropped their HDD division, but I don't think they'd even think about dropping millipede. This technology could very well be the future of long-term data storage.
I just hope it comes through in a pure format, and soon (without DRM).
http://www.research.ibm.com/resources/news/2002
Karma: \Kar"ma\, n. [Skr.] (Buddhism) One's acts considered as fixing one's lot in the future existence.
Quite contrary to this story, the advent of digital data storage and the Internet have led to something never before possible in the history of mankind: near instantaneous massive duplication. It is now possible for digital data to be copied effortlessly and transferred all over the globe. The trick, is doing it.
.zip format.
Our data storage needs have kept pace with data storage ability for some time now. I don't see this ending anytime soon. But it might, eventually. It stands to reason that there will come a time when we will have a want of things to store for all the space we have. I don't count on it in my lifetime, but it could happen.
The trick, then, is getting the data from here to there. How do we do it?
1. The written word is still the most important medium of human communication. Project Gutenberg is doing a bang-up job of digitizing AND distributing written works, and this is a project we should all support. I would also like to see a similar project with scientific journals being digitized (if not already) and widely distributed to universities, who can host them publicly or privately.
2. Someone suggested CDs, but these are impractical. CD-r's have a shelf life of 100 years, and CD-RW has even less. These could work as storage medium, but you would have to be diligent in keeping them up-to-date. What we really need is a physical storage method (like CDs) that have the capacity of magnetic storage media, like HDs.
3. Open file formats. It stands to reason that computers will always understand ASCII (or possibly UNICODE) text. It would not be difficult to append text-only information to the end of even very complex documents, that could be retreived even if the file format itself was no longer known. xml-based file formats do this to a degree, but it depends on the universitality of the
4. All of this is useless if we ourselves are not diligent in keeping up with our digital information. In the Middle Ages, copying an old, worn-out parchment or scroll could take weeks, even months. Now it's possible to do it in a fraction of a second, so there's no reason we shouldn't.
I currently keep my important data (emails, writings, website) in the following locations: My hard drive, a backup file on another hardrive, a CD-RW, a CD-R (which I change/update every six months or so) The server at my school, and the my webserver which is offsite. I personally would like to see off-planet massive storage, but until storage space exceeds storage demand, we will always be faced with the question of "What is important enough to backup?"
My most important data on my computer is the pictures from my digital camera. Right now I'm keeping one copy of all the pictures on my hard drive, and as I take more pictures everytime I get ~650 megs worth I burn them onto a CD backup as well. I'd really like to be able to take them off of my hard drive to free up space, but then I hear that CDRs have been known to fail, which would be incredibly upsetting for me. Worse yet would be going back after a couple years have passed and finding that the CDRs have died with age. Of course the worst case scenario would be having my hard drive die in a couple of years, and go back to the CDRs only to find that they died at some unknown point in the past.
As such, does anyone have any recommendations for average people like me out there who have data that is very important to them, but for whom corporate measures like commercial data backup services just aren't practical? Is there a better practice I can do than what I'm doing already? How about specially designed long life CDRs? Does such a thing exist?
Think about it. 98% of what's out on the web is crap. The stuff that's really valuable get's copied, in general. People do mirrors, or download pages. I doubt much of real value will be lost in the long run. I mean, geez, I'm going to be really bummed when my porn collection goes bad, but I downloaded it from others, so it's still out there somewhere.
With our rapidly increasing HD sizes, backup methods and media aren't keeping up. I've already lost 2 large HD's in the last 2 years, and with my shiny new 80 Gig drives, I've got a Raid-1 setup, but still if they both fail within a short amount of time from each other, I'm outta luck.
Moreover, the advancement of HD tech makes it almost certain that when one fails in a year, I won't be able to get an exact replacement to reload it from the RAID.
Does anyone know of a PRACTICAL way to back up 80 Gig's of info? AHSay.com offers online backups, but the initial backup would take weeks through my ADSL modem, and then incrementals would be pretty much useless. I suppose I could use DVD-RW, but at 4.7 Gig a disk, we're talking 20ish disks, at several hours a piece. And doing incremental backups that way is a nightmare. It seems that my only real option is to use something like a MonsterTape backup storage device, but systems with 80Gig capacities and up START at $4000 a piece, and the tapes are 80 bucks a piece. With 80 gig drives available for $129 bucks (Pricewatch), it doesn't seem like a good option.
The Dopester
"Yes, I'm a Karma Whore, but I'm doing it to pay my way through school."
1,000 years - is that long enough? We have parchments that are 5,000 years old, we need to match or even exceed that. If civilisation is to come to a thundering catastrophic end, it might not get back up to our level of technology (sufficient to read the disks) for 10,000 years. this is a little better, but I'd like a bit more still.
Has anyone ever wondered if; assuming further developements in computing; none of this will matter? Its entirely possible that an artificial intelligence running on molecular scale hardware could have a thinking speed millions or billions of times faster than a human being. A network of these could in one year do more thinking than all human beings in all history. They would create as much or more new knowledge and information in that time period. Essentially, all achievements humans have ever made would be as important as the mating habits of dinosaurs. While its difficult to say whether these AIs would have any interest in our history, the information would hardly be relevant. I do not find this possibility frightening. I have no doubt that the AIs we create will require teaching much like a small child. They will learn everything we pass on to them. Essentially, while humans might eventually become extinct, most of our knowledge and culture would be passed on to some extent. It would be no different than generation changes with biological children. YOU won't live on forever, but your descendents might. Some of what you pass on to your children might last a very long time. That is one of the few legacies we leave behind in this life. In a similar manner, we would leave similar knowledge to artificialy intelligent descendents. They would quickly grow beyond us, much as biological children might become taller and more educated than their parents, but such is the nature of change.
More than 90% of the works of ancient the more famous Roman or Greek authors have vanished, to say nothing of the more lesser known writers.
Something which is a greater loss to historians and archeologists is the lack of documents from regular people. Private letters, business records, etc. These can tell a lot more about society than pieces of fiction.
(the only thing worse than a spelling mistake in a post is a mistake in the subj:)
When I was in highschool, a friend of mine gave me a picture of her in the park. She was off center and some guy was in the background. Several times I considered taking scissors and cropping that guy out. After all, I didn't know him and he wasn't nearly as cute as she was. Fast forward a few years, and I'm scanning my pics and posting them to my site, and I see the picture of her. Only this time, I recognize the guy in the background. He's a friend of mine now. So you never know what'll be important or interesting later, and you don't always need to wait a few hundred years for your perception to change.
jred
I'm not a mechanic but I play one in my garage...
I was asked my my employer to find a way to archive digital photograhs for 100 years. My solution? Print them to black and white film as "color separations" (R, G, B).
When I was in Brazil, a friend of mine told me about his efforts to get the Brazilian Gov't to adopt Open Source technologies. One of the reasons he brought up was historical documents. He was concerned (and rightfully so IMHO) that MS's proprietary .DOC format would only be readable as long as Office products were around. Even then, proprietary file formats have a way of going extinct.
With Open Source, theoretically a file format would never 'go extinct' since the original code for it will always be around.
That's probably the best Open Source argument I've ever heard.