The Digital Dark Age

zygan wrote to mention a Fairfax Digital article about the possibility of a digital dark age, as a result of the increasingly short-term lifespan of digital storage. From the article: "It is 2045, he suggests, and his grandchildren are exploring the attic of his old house when they come across a CD-ROM and a letter, which explains that the disk contains a document that provides directions to obtaining the family fortune. The children are excited. 'But they've never seen a CD before - except in old movies - and, even if they found a suitable disk drive, how will they run the software necessary to interpret the information on the disk? How can they read my obsolete digital document?'"

Comments

Nothing to see here.

[dgatwood]

I love how that was the first thing I saw for this story....

Re:Nothing to see here.

[a.d.trick]

Not only that, but this isn't really new either, I have to deal with this thing every time Microsoft dumps out another version of Office.

Re:Nothing to see here.

[dgatwood]

For the folks who don't get the joke... the first thing I was in a story about information being inaccessible is an indication that the story was inaccessible. It's ironic in a rather apt sort of way. That, in turn, is funny.

I metamod moderation abuse.

this should be soluble.

[yagu]

Scary article. But probably too true.

In my opinion data archival screams to be handled in as simple an lowest-common-denominator a way as possible. For me, that means text for documents, and picture formats that would seem guaranteed to be around for a long time, if not forever. I'm guessing a good candidate for pictures would be something like jpg. I can't imagine jpg going away or ever being a non-decipherable picture format. Video might be a tougher nut to crack but I would guess some flavor of mpg.

Note that none of these flavors: text; jpg; nor mpg, include or imply any reliance on vendor proprietary formats (yes, I know there's a certain proprietary tinge to the picture and video forms, but they're pretty universal). So, storing and archiving for historical purposes rules out Microsoft and all of their formats. This would especially make sense considering there are already huge compatibility issues with Microsoft documents among their various versions of their products.

Also, for retrieval assurance it no longer makes sense to me to use "dead" or "inert" methods for storage, e.g., tapes, cds, dvds, etc. Instead, at least for my purposes I maintain multiple physical and current storage devices for all of my important data. This has been a recent (last three years) development for me when I started reading about early failures of the supposedly rugged storage.

So, that being the case that introduces (introduced) the need to devise a strategy for forward migration of all of may data so nothing got left behind. Fortunately, this has been mostly easy since right now the "active" storage du jour seems to be hard disk drives, and the capacity has grown sufficiently with each new generation of drives I have been able to simply roll my data forward onto the new drives with the new data with plenty of room to spare.

This shouldn't be an approach foreign to comapanies with reasonably competent data shops either. But maybe a philosophical change. All is not lost, and hopefully all will not be.

Just my $.02. ~

Re:this should be soluble.

[Anonymous Coward]

this should be soluble

That could be a problem. At least a CD won't get damaged by water.

Re:this should be soluble.

[damnfuct]

Well, exactly. If you put directions to your family fortune in some sort of obscure and soon-to-be-obsolete format, you deserve to have it lost. If your kids can't figure out how to open a raw text file, your money is safer missing.

Re:this should be soluble.

[Cruciform]

Even resorting to paper these days you want to make sure you've got archival quality equipment.

Some inkjet pages fade considerably in just two years. After a decade they may just be yellowing pages with no discernible content.

Re:this should be soluble.

[merreborn]

I'd think bmp would be preferable to jpg. bmp is to images what .txt is to text (and while ASCII is arbitrary, it's a single substitution cypher, and therefore easily crackable) -- the simplest, uncompressed format. I've written 1-bit (black and white) bitmaps by hand. I couldn't ever hope to do the same in jpeg.

Re:this should be soluble.

[jd]

I would personally opt for PNG for images, to avoid loss of data. Video almost has to be MPEG, as neither MNG nor APNG have really gone anywhere at this time and the BBC's high definition format isn't getting much adoption yet either. For audio, MP4 would seem the best choice - less loss of data, but more likely to be readable in the far future than Ogg Vorbis (which is a shame) or AIFF (yay! AIFF's gonna die!)

No matter what form you store the data in, if you want it readable in the far future, you've got to remember two things - there's no guarantee ANY specific technology will exist, and there's no guarantee ANY specific timeframe for the reading to take place.

What you want, then, is to do the reverse of the language decoding that has taken place over the years. Imagine yourself faced with a puzzle every bit as baffling as Egyptian Hyroglyphics, only stored at a vastly greater information density and probably in an electronic format. What would you want/need, to be able to recover the data?

Well, there would seem to be a few things that are essential. First, the explorer in the future will need to know the data is there and in what form. So, if you're using optical storage, make that clear (along with frequency). If you're using N-state logic, make it clear what N is. If there are M layers, tell them the value of M. You don't need to know all of the technical information, because all they need is where to start looking.

Secondly, the information needs to be correctly indexed. Languages are broken because types of information can be grouped and identified. The same will be true here. So, produce a contents list with corresponding data formats and/or MIME types, along with the offsets within the medium.

Thirdly, a key is a REALLY good idea - something analogous to the Rosetta Stone. Let's say you're using binary logic and a fairly rudimentary FS on the storage medium with text-based directories. The key would be a printout of the root directory in binary, again in ASCII and a third time as a set of records describing the logical layout. The printout would also need the offset of the directory. From this, it would be trivial for someone in the year 3000 to determine how offsets were calculated, how the data was laid on the disk and how the data is connected.

If physical storage is going to be used, ensure the various media used will last about the same length of time. So, if you're aiming for a hundred years, CDs may just about work. But you must NOT have the CD in contact with sulphides or anything else which will destroy the surface. The CD must be kept cold (but not so cold it is damaged) to slow decomposition. It should also be kept somewhere where accidental exposure to UV is impossible.

If you're keeping paper notes with the data, as I've suggested, the paper must be acid-free and the inks must be long-lasting. Most modern paper is of very low grade, as are most modern inks.

If you're looking more at a time capsule that is for the FAR future (we're talking something that happens AFTER Star Trek), then you've got to be extra careful but it should still be possible. I see no reason why you couldn't have physical storage under ideal conditions which could be retrievable after a thousand years or so. You just have to be very careful on what you choose to use. Same with paper. If you're looking to produce the next Beowulf (no, not the clustering technology), then you're probably going to want to look at vellum or some other extremely high-quality medium. I'd also look up early inks on the Internet and modify a recipe that could be used as a refill for a printer ink cartridge. Many early inks are highly stable (iron oxide is one example) and fade more by damage to the medium than decay of the ink.

Re:this should be soluble.

[QuietLagoon]

Only 2045? That's just 40 years from now.

I have a record from around 1915. Caruso on shellac, 78rpm (give or take a couple rpm). It still plays as well as it ever did. You can hear every note.

What's the lifespan of punch cards?

Re:this should be soluble.

[Xzzy]

I would personally opt for PNG for images, to avoid loss of data.

I'd try XPM. ;) Sure it's a bloated format, but it's a human readable text file. Bitmap images can easily be seen in a text editor.

As long as the ASCII charset is not lost and forgotten, the file would remain decipherable to any computer geek. Can JPG, PNG, or GIF claim the same thing?

Re:this should be soluble.

[Coryoth]

Thirdly, a key is a REALLY good idea - something analogous to the Rosetta Stone.

Not exactly replying to your post as simply having my memory spurred with regard to something relevant: if you're really interested in storing information for future generations then The Rosetta Project is an interesting on. They seek to have as many distinct languages as possible printed on a small disk, beginning in large print but decreasing in size as it spirals inwards to the point where it is micro-etched. It's easy enough to figure out how to read it, and as long as you cna build tools to magnify it you can read everything on it.

Jedidiah.

Re:this should be soluble.

[Pharmboy]

I have heard the same for photographs. Today's photographic paper isn't the same as older stuff, with less silver, and it tends to fade quicker. While we can rely on 100 year old photographs, our decendents may not. Most paper nowdays is relatively acidic as well, so it breaks down faster with any exposure. This would mean books as well. While there is good paper that is better than the old stuff, most is made to be cheap, not high quality.

Re:this should be soluble.

[wonford]

Algorithms may well get lost, so I'd advocate zero compression - straight text or some lossless image format (bmp, tiff etc); but I would be more worried about the longevity of the actual storage medium itself... I seem to remember reading an article about a US-built 'time capsule' wherein straight text had been etched into some sort of corrosion-free metal along with a codex (maybe someone else may have heard of this). Not too practical for us little people, I know :-)

Does anyone know how long a recordable CD/DVD will actually last if well kept? At least it should be EMP-proof

Re:this should be soluble.

[jcr]

What's the lifespan of punch cards?

It depends on how they're stored. Stick them in a no-humidity constant-temperature environment like an abandoned salt mine, and they should be fine for a couple of centuries.

-jcr

Re:this should be soluble.

[carl0ski]

You can't really call CD dead.
it should be supported for another 5-10 years

until music CD's disappear from the picture give or take 5 years before DVD bluray etc drives dont bother adding cd read capability.

as far as i'm concerned DVD will die long before CD will.

Re:this should be soluble.

[noyren]

heh, if they look in my attic they will find a bunch of cd's, but they'll also find a bunch of computers that can read them ;)

Re:this should be soluble.

[QuietLagoon]

couple of centuries

That's what I thought. Thanks.

Re:this should be soluble.

[ZorinLynx]

I dunno, I doubt it'll be impossible to read a CD even in 90 years.

Even today, you can find places to convert old 8mm home movies into a more modern format.

CD-ROM drives are resilient devices; I'm sure millions of them will survive in working condition for many decades. Some will eventually be owned by data conversion services that will do this for you.

You can still readily find equipment to play 78RPM records, reel-to-reel tape, 9track computer tapes, TK50, and other dead formats. It may be difficult, but not impossible to recover your old data in 50-100 years.

Whether the CD media will hold up that long is another story. Polycarbonate is pretty stable, but we're talking a long time during which an accident could happen. An accident that merely tears or crumples paper can destroy a CD!

-Z

Re:this should be soluble.

[Kesh]

Yep. I worked selling printers for Epson for a while. Even their good (consumer) archival paper is only rated 80-100 years, when using their good archival level printer (about $700 USD). Supposedly their cheaper printers would get 50-80 years when using archival photo paper and the normal inks.

Of course, that's their own ratings, so I dunno how accurate it is.

Re:this should be soluble.

[Anonymous Coward]

Nickel. It was nickel. Pages of text were optically reduced to microscopic size and photographically etched onto discs of pure nickel. The goal was to come up with a way of storing data for 10,000 years. As long as the basic science of optics survives, and the discs themselves are extant, the information on those discs will continue to be readable forever.

Re:this should be soluble.

[jacksonj04]

Also, the plans to build the devices are now digitised in a modern format. Even assuming the format becomes obsolete, details on how to read it will be transcribed into the latest format eg. even though vinyl records are more or less obsolete, details on how they are read are available on the internet.

All it needs is archive.org to keep the records, and it will be possible to step back through the algorithms and plans in order to extract the data. I cannot see the CD-ROM specification being lost forever when it's so easy to migrate an entire knowledgebase across mediums and formats.

Re:this should be soluble.

[digidave]

That doesn't mean it's a bad format. This is all about formats that will stick around, which JPG won't. Like the GP said, BMPs are practically human-readable if you know what you're doing.

Re:this should be soluble.

[Anonymous Coward]

No, XPM is better than BMP. Why? It is a plain text file. Once you can read ASCII, you can read XPM. It is basically the same as BMP -- uncompressed image data -- but is written in a format that can be parsed easily. Plus, since it is stored as a character array, it makes it easier to load into C/C++. Which I imagine will be around in some form in 2045. Hell, Fortan is still here and useful!

Re:this should be soluble.

[AdamWeeden]

Another vital point to add to that is BMP is lossless.

Re:this should be soluble.

[Anonymous Coward]

Have you noticed a common thing between text and JPEG images? You're proposing that they be the data-level formats (i.e. not the media) that will be used to store things for the Unforeseen Future.

I noticed one thing in common, see. Both newline-delimited plain text (ASCII, basically, and the ISO LATIN 8-bit variants + a big handful of regional adaptations) and JPEG images will be "the state of the art of the past" in the future. As such, they will almost certainly be among the things that are taught in e.g. introductory software engineering classes. Plaintext certainly is today, yet consider that free-form text was considered reckless by sixties punched-card standards. Likewise education has progressed; linked lists and old-school static storage management (as in "this list grows this way and that one goes that way"; not malloc and free) today are mostly just discussed briefly or mentioned in a lecture handout; instead students are taught red-black trees and stuff like that which wasn't so long ago considered advanced topics.

In some advanced vision of the future I think that JPEG will certainly be a part of some course assignment. Discrete cosine transforms aren't on their way out after all, regardless of whether Mr. Barnsley continues to seat his fat arse on those wavelet patents or not. This quality of the Really Good Algorithms and data structures will save our asses once the local hardware guy gets his anachronistic "Cee-Dee Raahm" drives working again...

Re:this should be soluble.

[kabz]

Yeah, I can just imagine ...

You find the CD buried in a box in the garden.
You see the Microsoft logo. An old, long-dead company.
You scrape some dust off the CD.
You read through the logos and fine print on the CD.
You see the logo 'PlaysForSure' (tm)
You groan and throw the CD in the trash.

Re:this should be soluble.

[Anonymous Coward]

So what? 1000 years befor not everyone could write, so today you get the "best" of what was been told then in 1000 years you'll see only the "best" of today, not everythin little kids(like me) write in their blog it's all this darwin stuff, only the best prepared will survive

Re:this should be soluble.

[anothermortal]

Not to be rude, but for the love of God, don't you have something to do besides write 1-bit bitmaps by hand?

Re:this should be soluble.

[damnfuct]

I see what you mean, but I had meant the file format itself, (probably should have made it clearer)

Re:this should be soluble.

[arminw]

...the paper must be acid-free and the inks must be long-lasting.....

Something printed on good acid free paper by a laser printer ought to be pretty durable maybe? How well does the toner fused to the paper stay legible for a long time provided the paper stays intact?

Re:this should be soluble.

[Lumpy]

probably true? how about IS true.

I have a side business as a data mage. I have a nice collection of very VERY old dat astorage reading/writing devices that sit un-used for 99.997% of their life now. but every once in a while I get a call from a friend of a friand's colleague. and I make a very tidy sum reading the files off of that Bernulli disk or 9 track tape in ebcdic format and but it to a modern format like CD.

There are several of us that exist, and there always be some that will have the ability to read long forgotten data formats. Hell I bet I have a working paper tape reader here somewhere.

My ex wife called me a fool for buying and keeping all this "crap" as she calls it until the day that I dragged a 9 track tape reader into the living room, set up an old XT that was able to use the isa card that connected to the drive and spent the afternoon reading 5 tapes to the drive and then transferring via rs232 to my laptop to burn to CD.

I made $8000.00 in one afternoon because I was the only one able to help them.

She finally shut the hell up when I returned with that check and threw it on the table next to the bills.

There are businesses that make their good living off this exact need.

My biggest customers are small businesses wanting 5.25 disks read or the popular bernulli's or the odd TRS-80 business compter disk (Model III and Model VI types typically) that needs to be read. I do turn away business, some balk right away at the price I charge, but I knw they will not find someone else locally and certianly not near my low price nationally. This help seperate the curious from the real customers.

Re:this should be soluble.

[russellh]

My advice is pick the really important things to preserve and save them well. eg- print it out. Everything else: just do the minimum, and throw away the crap. Most of it will probably be fine anyway, for our lifetime, but the important stuff you've already taken care of.

(btw, the specific problem with burnt CDs is the decay of the organic dye, iirc. the blue ones last the longest.)

Re:this should be soluble.

[BrokenHalo]

Your mention of acidic paper brings up a favourite hobby-horse of mine. I've been saying for some time that we are due at some point not just for a digital dark-age, but one more akin to the last real dark age.

Up until about 1830, paper was typically produced from rag pulp, and this stuff was good for a long time.

Since then, the use of acidic wood-pulp in paper production has led to the majority of books and documents produced over the later 19th and 20th centuries being likely to crumble to dust in a relatively short time. I have many books printed as recently as 1990 that are very brittle and are consequently disintegrating, despite having been comparatively well looked after.

Re:this should be soluble.

[Kadin2048]

I take a little issue about AIFF. It's basically just a container format, generally for an uncompressed 44.1kHz/16-bit audio stream of audio data.

Seems like that would be the very best format to store it in. At least someone could cut away the file headers and have a shot at decoding the audio data pretty easily -- I can't say the same for MP3 or MP4, if you've lost the decoding algorithms.

Uncompressed digital audio, stored in some kind of easily dissectable container (AIFF or WAV) seems by far to be the best way to it. After all, once you figure out the sample rate and depth, you really just need to run it through a DAC (or a software equivalent). It's essentially the same as a bitmap for graphics. And if you wanted to make the sample rate known, you could begin the recording with a blast of test tone, and note its frequency and relative volume somewhere where the finder would see it.

Realistically, in the near-term I think the best choice for audio is the plain old Red Book compliant Audio CD, just because of the market penetration and amount of time the format's been around. I'm pretty confident that as long as that form factor sticks around (12cm dia by 1.2mm thick) drives will probably have CD-DA backwards compatibility, unless there is some radical change in how the data is stored away from using lasers. I know Red Book has some error correction data over and above what uncompressed DA would, but the format is so widely known that I think the advantages of the format being well known outweigh the disadvantages that someone would have trying to 'unpack' the data by hand, bit by bit.

But as a data format only -- independent of the physical media you choose to store it on -- uncompressed formats are the answer. For text, ASCII. For graphics, Bitmaps. For sound, as close to the uncompressed digital audio as you can get, in as simple a container as possible.

Re:this should be soluble.

[iamhassi]

"I doubt it'll be impossible to read a CD even in 90 years."

Agreed. Here's some reasons:
--even today, the most modern PC you purchase reads CDs with no problem
--CDs have been the defacto standard for audio for 20+ years and there's nothing to indicate they'll be replaced in the near future, so as long as we want to listen to audio CDs there will be CD drives
--CDs drives and media are incredible cheap to produce, insuring no other cheaper standard will come along and knock CDs off their pedestal as being the cheapest around
--No major audio music manufacture has endorsed any other format as a standard to replace CDs, further indicating they will be around for awhile

I think CDs are a safe bet. You could have recorded your voice to record almost 100 yrs ago and still have no problem finding a record player to play it on so I'd be willing to bet you have 100+ years left in CDs.

Besides, there is no format available today that we can be sure that we'll be able to read in 100, 500, or 1000 years. Even the old clay tablets require special "players" (interpreters) to read them. As society progresses we stop using the old and move on with the new, how can you insure we'll still be using engrish in 1000 years? We'll all probably be speaking some form of chinese ;)

Re:this should be soluble.

[belmolis]

For audio, MP4 would seem the best choice - less loss of data, but more likely to be readable in the far future than Ogg Vorbis (which is a shame) or AIFF (yay! AIFF's gonna die!)

It would be better not to use lossy compression. Either use uncompressed linear PCM data with a nice, simple file format (e.g. snd, or if you must use WAVE, the minimal version) or if compression is necessary, use a lossless compression method like FLAC.

Re:this should be soluble.

[Kadin2048]

You're absolutely right.

Paperback popular books are especially bad: I've even heard some people say that they're made intentionally like this, so that they don't get passed around too many times before they fall apart. Thus people buy their own copies. While I'm not sure that's true (I think it's more likely just cost-cutting on the part of publishers), and I suppose future generations probably won't suffer too badly by losing all the $6.99 copies of "The Davinci Code," it is distressing when I see the supposedly 'archival' copies of important documents, or textbooks, printed on crummy paper.

It's not as if you can't buy 100% cotton rag paper anymore, it's just a lot more expensive than the Staples $1.99/ream crap that people use most of the time. Granted, there's a lot of stuff that's being printed out every day that doesn't merit saving, but when you want something to last, you need to have the right material to put it on.

Personally, I do all my personal (dead tree) correspondence with an IBM Selectric II on Strathmore Legacy 24 lb., 100% cotton, bright white, Monarch size, laid executive stationary. I bought about a thousand sheets of it from a stationary store that was going out of business (another peeve of mine there) and suspect that it'll last me the rest of my life if I use it judiciously.

Whether the people I write to decide to keep my letters or throw them away is their business. But at least I know that if they decide to keep them, they'll always be readable. If you want it to last, you can't beat carbon black pounded into cotton.

Re:this should be soluble.

[weinerdog]

My job involves digitizing old (pre-20th Century) documents with a view to preserving and providing easy access to them. We have come to the conclusion that preservation of digital resources is entirely possible, but it is an ongoing process: you cannot just put stuff on a CD and bury it somewhere, confident it will be readable in 500, 50, or even 5 years. You have to take active steps to ensure your data remains available.

Media rot and media and file format obsolescence can cause problems if you just digitize and forget. In the library and archival community, microfilm is generally considered the safest way of preservation because it has a theoretical shelf life of 500 years (though no piece of mircofilm is close to that old) and can be read by anyone with a magnifying device. Even so, microfilm gets lost, damaged, or otherwise rendered unusable. Paper can last even longer, with proper care, but it too is susceptible to damage or loss. Some of the items we have preserved no longer exist in their original form because the last copy was destroyed as a result of use or damage due to less-than-perfect (i.e. real-world) storage.

Regardless of the media, the most important aspects of preserving something are to have multiple copies and to actively maintain your collection. If you actively monitor your collection and migrate file formats and media as needed, you should be alright. If you just file and forget, all bets are off.

Nothing can guarantee 100% retention: every year, the last copy of something disappears forever. But the stuff that still exists today is, by and large, the stuff that someone thought was important enough to actively maintain, or else the stuff where so many copies were made that some were bound to survive regardless of the conditions under which they were stored.

Re:this should be soluble.

[NanoGator]

"Not to be rude, but for the love of God, don't you have something to do besides write 1-bit bitmaps by hand?"... said the guy being needlessly critical on Slashdot.

Re:this should be soluble.

[DeafByBeheading]

ASCII? That's too complicated. Let's go with a unary encoding: 'a' is 1, 'b' is 11, 'c' is 111, and so on...

Re:this should be soluble.

[jonfr]

But we are already in digital dark age. Every tryed to read a document from a computer that was made in the year 1990 - 1995. It is hard, it is even harder to find a 5" Floppy drive that works and software that can use it and it is going to be a nightmare to find a program that reads an long old dead property formats from those times.

We are in digital dark age, it has already started and the end is many years from now. The price of this digital dark age is going to be hig.

Re:this should be soluble.

[Kadin2048]

I agree. But there should be something available to store stuff that's not quite important enough by itself to warrant printing out, but might be worth storing anyway.

Also, there are those things which gain importance by being a complete record. For example (and this is a weak example, I admit) take all my email. It's far too much to print out, and it wouldn't be worth the paper anyway. However, that's not to say it's unimportant: if I could keep a complete record of every email I ever wrote, for my entire life, that repository in toto could be important or worthwhile. It's important though only because it's a complete record, and useful really only if it's searchable and indexed (or sorted). Very few messages are by themselves important, but it would be neat to have the ability 30 years from now to go through and see what was on my mind, at any given point in time.

For things like this, you really need a storage medium that's both random access and supports a high data density. To go with that, it's probably acceptable for it to have a shorter lifespan than the "time capsule" applications.

Re:this should be soluble.

[bergeron76]

Isn't PNG or XPM similar?

Re:this should be soluble.

[bergeron76]

Why not take this a step farther and create an engraved document? Or perhaps a marble etching or stamped steel plate?

If the majority of us were all erased (not mentally, but physically) from the Earth, at least something would remain for future civilizations...

Re:this should be soluble.

[Green Salad]

...they will find a bunch of cd's, but they'll also find a bunch of computers that can read them...

I'm pretty sure the computer's broke or something. I tried plugging the computer into a regular hydrogen outlet and the hydrogen receptacle claimed in a very insistent voice that device didn't meet code for an approved hydrogen appliance. I figured I was Foobar.

Then, I remembered a story about how energy used to be distributed. After extensive research, I sucessfully built a 120 volt sine wave generator, plugged the computer in, and spoke into the mouse. Guess what? The computer didn't say a thing back.

Still more research made it clear that sound would come from the silver platter things. I held one tight against my ear when I spoke into the mouse...still nothing.

After still more research, I connected the big vacuum tube thingy (which also runs on 120 volt sine waves) to the computer. After a few microblicks, I saw some strange white symbols glowing on the surface. It that looked like this "C: Drive Error: No Bootable Device."
BR> Now where getting somewhere. I'm sure the pattern of glowing dots is tightly compressed data being shown on the screen. I've got a pattern recognition expert looking at the pattern of glowing dots on the screen. It's probably some sort of ancient code.

Yesterday, someone said that feelings of surprise, anxiety and disbelief was represented by a glowing pattern of dots that looked like this =:O

Re:this should be soluble.

[HunterZ]

FLAC is good for lossless audio compression. If you're going to use MPEG for video and PNG for pictures then you may as well use FLAC for audio.

There are also lossless video compressors such as Huffyuv.

Also, as the parent mentioned, there are some special considerations required when using CDs as a long-term storage medium. They're better than magnetic media, but in my opinion CDs are still too fragile and susceptible to degradation. I've had discs flake off after sitting undisturbed for a couple years in a soft case; I've also had discs become unusable due to the tiniest scratch on the top, where there is for some reason little to no protection against damage to the data layer. Hopefully the holographic storage we've been hearing about for years now will be a reality in the next decade.

Re:this should be soluble.

[WinterSolstice]

I couldn't disagree more.

We are no more able to predict 90 years from now than the engineers on the Eniac could predict iPod Nanos. They wouldn't even see the point, let alone the how.

I lost everything... *everything* from my childhood to digital obsolesence. I lost the midi songs I wrote (I stored them on 8 inch, that should be readable, right?) the poetry and daily journal I kept (It was in Wordstar and MS Works... there's a converter right?), the art that I did (Splash... everyone can use that!). I've lost just recently thousands of lines of code I kept backed up on Zip disks. The disks are currently "blank"/"corrupt".

The future is not so bright, but this year I switched to all open-source everything. I keep the source and a compiler with my stuff, so theoretically it could be reverse engineered/ported as long as the data stays "live" in my multiple network shares. I started using PNG for all graphics, and HTML/XHTML for all text. There is code for a simple reader with my text, and I have a version of each doc in OO.o format, HTML/XHTML, and TXT. My kids photos are now being converted to PNG to save them as well as possible.

Obviously, I should just count on the dye on CDs lasting forever though, right?

-WS

Re:this should be soluble.

[Mancat]

Hmm... And you say this just as I threw away two SCSI Bernoulli drives yesterday. Damnit!

Re:this should be soluble.

[BrokenHalo]

If you want it to last, you can't beat carbon black pounded into cotton.

... Which reminds me of some old Japanese advice I once read, to the effect that in the event of a fire, you should throw your important documents into your well; the idea supposedly being that sumi (lampblack) ink on mulberry paper should be able to stand that kind of treatment...

Re:this should be soluble.

[kerohazel]

Two things.
1) BMPs can actually make use of RLE compression (which is still fairly human-readable but only if you know how it works).
2) They're rendered upside-down. So when the aliens of the future try to figure out what humans looked like, they're wonder why pictures of us show us all standing on our heads.

Re:this should be soluble.

[jonom]

We can't rely on the 100 year old photographs any more either, colour or b&w. In fact, we can't rely on photos well into the 1970s because of the same problem.

Companies like Corbis are spending millions trying to digitize old photos before they rot. The pictures are being kept in cold storage to slow the aging process.

Managed to find the link to an interesting article from the Washington Post on the problem from a couple of years ago: http://tinyurl.com/d8cvm It's long but a good read.

Re:this should be soluble.

[euxneks]

I think anything open source is a good idea to store data in. Anyone in the future can then look at the spec sheet or whatever and then develop new programs if they need to look at the data.

Re:this should be soluble.

[thejynxed]

I develop photographs for a living. I can tell you right now that matte finish paper is much better than this glossy crap that everyone seems to like nowadays. Glossy paper is not only more acidic, but it can "smudge" if handled too soon out of the developmental process. There is also the fact that glossy paper retains skin oils more than even regular writing paper... In essence you get tons of fingerprints all over your pictures, and the oils from those prints eat into the paper over a short period of time. This is not a problem with matte paper, as it contains a higher silver content, and has a more textured surface that doesn't retain skin oils to the degree of glossy paper. Matte paper photos also don't fade as fast as glossies when exposed to direct sunlight. Matte photographs also produce better results when scanned, and require alot less color, hue, brightness and contrast manipulation in Photoshop when creating copies from a print. I've noticed that with glossies, you get colors that are overly bright, and the reflective shiny surface of the paper (the same shiny surface that makes the photos hard to see at certain angles in bright light) can throw off digital scanners a bit and make the colors off-balanced. For instance: If you scan a black & white glossy and a black and white matte photo of the same subject matter, the results from the glossy make the black appear as a greenish color while the results from the matte photo are identical to the original. The supply cost for the each paper is the same (At least up until you get into the 11x14 - 12x18 range, then matte cost slightly more since it isn't as common as glossy), so I encourage people to use matte. BTW - Almost all of the professional photographers in my area (Jamestown/Erie/Buffalo/Niagra) choose matte photographs for their personal collections. They get glossy prints only for customers who want them.

Re:this should be soluble.

[Anonymous Coward]

I've written 1-bit (black and white) bitmaps by hand.

Man, I've been desperate to see porn, too, but that's beyond the pale.

Re:this should be soluble.

[exKingZog]

Also, BMP files are lossless. If your only copy of a faded old photo is digital, you don't want jpeg artefacts appearing, losing yet more detail. The same applies to art created on computers - it needs to be stored in a lossless format, otherwise the museum curators of the future will be spending their time restoring digital masterpieces from heavily-compressed jpegs.

Re:this should be soluble.

[baadger]

JPEG is a stupid format for producing photo archives today. Nevermind 2045.

Here we are entering the realm of HDTV, MPEG-4, wavelet compression techniques for video and we look back at 320/252x240/288 MPEG-1 and cringe (atleast I do). In the future our JPEG's may be seen on the lower end of useable quality. ... Hey these old JPEG's aren't detailed enough for our holodeck compute... better run it through Photoshop XII Virtual Reality Edition.

We should be using lossless JPEG2000 / Some kind of wavelet / PNG / some kind of compressed but LOSSLESS format. Atleast that way out descendants can't blame us for not doing our very best to preserve our documents (given our current limitations in scanning and restoration techniques).

Re:this should be soluble.

[deimtee]

Back in the late nineties Xerox rated the toner in their Docutech / 6135 range at four hundred years.
Interestingly, in the same document that they guaranteed this, they only rated their archival paper at three hundred years.
This was for printing archival Govt. documents back then. I don't know what their current standards are.

Re:this should be soluble.

[Hosiah]

I'm guessing a good candidate for pictures would be something like jpg.

You *are* aware what a lossy format .jpg's are, right? .png's are nearly as universal, and they'd be able to reproduce the original image data better, so you could shrink 'em smaller. Just my 1.999-cents. Used to work with microfilm, so i'm used to thinking it terms of "highest-definition resolution for smallest amount of space". Unfortunately, .png's do take up more "space" (memory), but it might be negligible if you could make every pixel count instead of "close enough". Interresting points, all round...

Re:this should be soluble.

[Shano]

Who says they're upside down? Left-right, top-bottom is a completely arbitrary order for processing raster data.

Some systems (such as the old BBC Micro) had a graphics origin at the bottom left - as do most graphs in mathematics. I'm not aware of any language that is written bottom-up, but there's no fundamental problem with it.

Re:this should be soluble.

[fireboy1919]

It is a plain text file

Only if you want it to be, and most people don't. There are both ascii and binary versions of these, and readers will generally be able to read both (I've never found one that can't).

Unlike BMP it doesn't have a pallette, which is a more important distinction: you don't have to create code for that if you have to write something that reads it.

PNM is perhaps the most obvious graphics format you can get, though - either in binary or text mode. There's no real reason NOT to represent the images in binary mode. Anybody with half a brain can figure out how to decode them.

Heck, the library I wrote to read, write, and manipulate them is about 100 lines of code, and that's in C++.

Re:this should be soluble.

[anthony_dipierro]

You could have recorded your voice to record almost 100 yrs ago and still have no problem finding a record player to play it on so I'd be willing to bet you have 100+ years left in CDs.

The thing is, I could probably build a crude record player in my garage. Do you think the average person is going to have the tools to build a CD player in their garage in 100+ years? Sure, this won't be a problem for really valuable data, but for less valuable data it might not be worth it to go through the trouble.

This of course assumes that the CD itself lasts 100+ years. I'm sure this isn't true for a non-pressed CD.

Important information should be copied periodically, certainly at the point where you upgrade from one system to another.

Re:this should be soluble.

[i7dude]

this is rampant oversimplification, but my gut feeling is that as long as we have books...we have a way of passing things on. honesly, we may find a solid way of arching digital data and find a medium capable of surviving, and moreover a way of allowing future generations to pass it on...but, it seems to me that books, may very well be the first and last way of us passing knowledge between era's.

i think we should keep digital archives around for future generations to access and reverse engineer, but it also seems that we should create a technological bible of sorts, that outlines, not in extreme detail, some of the most important mathmatical and scientific principles that we base our current technology on. that way, if our keys do not always pass on, at least a snapshot of our thinking and design paradigms will.

these are pseudo-intellectual ramlings at best. i could have just said "books are the way to go."

dude.

Re:this should be soluble.

[spectre_240sx]

I'm not sure about the non-archival prints lasting that long, but that sounds fairly on target for archival prints (Btw, a 2200 can be purchased for much less now). There are also sprays that you can use that will extend the life of your prints. I think www.inkjetart.com has some for sale, but I can't remember. In the end, though, archival prints right now have a good chance at outlasting a regular C-Print. That's pretty good in my eyes.

Re:this should be soluble.

[spectre_240sx]

Vinyl is obsolete? I think Technics, Denon, Stanton, Numark and many of the high end audiophile quality turntable manufacturers would disagree with you pretty quickly on that point. Just because something has moved into a different market doesn't make it obsolete. Vinyl isn't going anywhere for a while, and that's why it's still possible to purchase turntables. How long do you think CDs will stick around, even in a small market? I don't see it being more than 20 years personally... For that reason, I think vinyl will outlive CDs.

Re:this should be soluble.

[petermgreen]

also remember that restoration (at least in the digital sense, things done to an actual physical picture may be a different matter) takes away real information and replaces it with the restorers ideas. So really you should always store a losslessly compressed unrestored version.

Re:this should be soluble.

[Anonymous Coward]

I think anything open source is a bad idea to store data in. Anyone in the future will have trouble decoding the data due to multiple file formats, multiple versions of the software, forks, branches, etc.

Re:this should be soluble.

[superflippy]

I know some people who are into the whole scrapbooking scene. Those people are obsessed about everything being acid-free and archival quality. It makes me wonder sometimes whether, a couple hundred years from now, the primary documents left to remember this time in human history will be cutesy scrapbooks made by bored housewives.

Re:this should be soluble.

[pipingguy]

Video might be a tougher nut to crack but I would guess some flavor of mpg.

No kidding. I can take .avi video with my Canon SD300 but people I give it to can't view it without the proper CODEC. This is a major problem that nobody ever talks about for some reason.

Re:this should be soluble.

[pipingguy]

Boo-ya! My go-to guy said I was good to go with this, party on, dude!

Main Entry: soluble
Pronunciation: 'säl-y&-b&l
Function: adjective
Etymology: Middle English, from Middle French, capable of being loosened or dissolved, from Late Latin solubilis, from Latin solvere to loosen, dissolve -- more at SOLVE
1 : susceptible of being dissolved in or as if in a liquid and especially water
2 : subject to being solved or explained

Re:this should be soluble.

[Anonymous Coward]

It should be noted here that you can print on only one side of the paper, becasuse laser printer ink will stick to the opposite page in only about three years when the paper is held at room temperature.

Re:this should be soluble.

[Anonymous Coward]

When half of the bits have rotten away, an unpacked format will be playable, although with terrible quality. Recovering a (even losslessly) packed audio stream in such conditions seems much more difficult.

Re:this should be soluble.

[Anonymous Coward]

A Rossetta Key is really a good idea.
The scenario would go something like this...

Great-Grandkids find old CD in attic...
Place it in CD-ROM drive
They find not one but two files
The first is a lump of data they don't have a clue what it is
the second is a very basic text file explaining
a.) what the data is
b.) how the data is encoded
and c.) how they could go about reading it
The kids get to watch family movies!
This is of course assuming that the averagew kid of 2110 can reverse engineer mp4 technology.

f[pppp

[Anonymous Coward]

I cannot fail it!!!!

Re:f[pppp

[Anonymous Coward]

You did.

The equipment?

[Dogers]

Nevermind the equipment needed to read it, what about the rights they'll need to read it?

And even that's ignoring the fact the CD will long since have self destructed, decaying away..

(From TFA: "Dark age ... the Powerhouse Museum's Matthew Connell with an ancient clay tablet that will probably outlive the 1980s tape in his right hand.".. Probably? Definitely more like!)

Re:The equipment?

[Anonymous Coward]

Nevemind the rights they'll need to read it, what makes you think CD-R's will last for 40 years?

Re:The equipment?

[Slashdot_Gandhi]

"Dark age ... the Powerhouse Museum's Matthew Connell with an ancient clay tablet that will probably outlive the 1980s tape in his right hand."..

This is more like what we need to preserve everything!.

Re:The equipment?

[insert_username_here]

Legally, this will not be a problem in the timescale the article implies. When restoring technology from 50 years ago, software patents (which last a paltry 17 years) don't even come into consideration. And although copyright lasts longer (thanks to the Senator for Disney), this will probably not be a problem, as the companies who own these rights will have bigger fish to fry than protecting the copyright for software that is 50 years old (assuming, that is, the company is still around - although IBM is over 100 years old and still going strong...).

I think the bigger problem, as other posters have said, is DRM; if all content is encoded with DRM, reading these DRM-encrypted formats in 50 years will be a challenge. But even then, I'm sure the computing technology available in 50 years time will easily be able to break the primitive encryption technology we have now. Although I'm sure they'll find the process easier if they have documentation of these DRM file formats.

ebay

[truckaxle]

Maybe they can buy all the necessary components on ebay!

Seriously archeologist have decoded all sorts of dead languages, decoding digital (assuming you can still pick out the bits) would be easier.

...and

[Stanistani]

In the second box is a player, if the fellow had any real fortune.

Besides, I'd have drawn the map on parchment, and tied it up with a string.

Arrr! Ye Mateys...

Re:...and

[geekoid]

A smart man would have left instructions with his lawyer.

Sheesh.

Re:...and

[$RANDOMLUSER]

OK - let's examine this concept - I have a family fortune - and the only person I'm going to tell the location to is a lawyer???????

Can't I just blow it on hookers and cocaine before I die?

Re:...and

[SpaceLifeForm]

Depends upon the lawyer. Here is a counterexample.

Re:...and

[stud9920]

As hard disks have the same interface as CDROM drives, why bother with the CD then ?

Duh

[oiper]

Don't you recall the video tombstones? Everybody will have them.

Nah

[Anonymous Coward]

While I could see the media the information is on becomming obsolete, binary is kind of timeless. Things like ASCII and the current compression algorithims I doubt will be 'forgotten' in 40 or so years.

Re:Nah

[Anonymous Coward]

ASCII in particular is part of UTF-8, which means it'll last for a long time (first, unicode must completely eliminate other encodings, then, some other unicode encoding will eliminate UTF-8, how long is that going to take? 50 years?).

Doesn't matter.

[Anonymous Coward]

This example could use a bit of revision. Some CD-Rs only have a life span of two years!

dark age

[foxhound01]

They'll take it to that crazy old guy in the corner house with uncut grass in his lawn, for he was once a great programming guru and has a ton of still functioning archaic equipment that requires insanely large amounts of power.

Re:dark age

[Frogbert]

I think its safe to say, with the number of people who showed up on Qlink using real Commodore 64s reciently, that there will be no shortage of "Crazy old guys" in the near future.

Re:dark age

[slashname3]

Damn kids! Get away from my Amiga! And leave the 3B2 alone!

Re:dark age

[shokk]

Sounds like a mission from some new MMORPG. Where can I download a demo?

Re:dark age

[Kadin2048]

This is true -- I know a person like this. (If anyone cares to read my other posts, it's the same guy who made a set of Helmholtz coils that encompassed a small room.) I know for a fact he has several 8" floppy disk drives with their associated controllers, tucked away "just in case." I'm also pretty sure he has a punch card reader. Whenever anybody had data stored on some sort of archaic format, he was the go-to person on campus: if he didn't actually have a machine to read it, he probably knew somebody who did, or where you could find one.

The only problem -- and frankly the more pressing problem as far as I'm concerned than data loss due to formats becoming obsolete -- is that the people who have real-world technical expertise with older systems and formats are retiring, and in some cases dying off. It's my sincere hope (perhaps through efforts like Wikipedia,) that people who have knowledge that could be useful to future users will write it down, or that other people will try to write it down for them, before they're gone.

It's that loss of human experience and expertise -- real working nuts and bolts knowledge -- that to me seems far more serious in terms of creating a 'new dark age' than whether or not we can access our Quicken records in 25 years.

Re:dark age

[Guppy06]

" They'll take it to that crazy old guy in the corner house with uncut grass in his lawn,"

His mom hasn't bugged him enough about mowing yet.

Re:dark age

[Anonymous Coward]

That will be me...
You can read your old stuff on one of following :

• Apple Mac
• Commodore Amiga 2000
• DEC ALPHA (3000, AS1000a, PWS 00au)
• DEC uVAX
• IBM AS/400
• IBM R/6000
• IBM PC XT
• IBM PC AT
• IBM PS/2 (several models)
• +many PC clones
  

Following drive types (disk and tape) will be available :

• Floppy 51/4"
• Floppy 3½"
• JAZ Drive
• ZIP Drive
• CD-RW Drive
• DVD-RW Drive
  
• Travan Tape Drive
• QIC Tape Drive (several types)
• DLT 4000 Tape Drive
• +some tape drives I dont know...
  

Everything in working condition...
and with the nessessary software...
Am I crazy enough to be the ol' guy ?
...then I will probably C U in 100 years.

Re:dark age

[firewood]

Don't forget that the first personal computers (Altair, Imsai, et.al.) used 8" floppies, paper tape, and audio cassette tape. Byte magazine (and others) printed code in some sort of bar code format. That's not even starting with the formats used by old, pre-PC, computers. Now where's my box of punch cards?

Let the Internet back it up for you

[Anonymous Coward]

Works for Linus.

easy

[DrSkwid]

perhaps the same way I would read a wax cylinder today

visit a specialist

a good place to start would be here :

http://www.bl.uk/collections/sound-archive/wtmcyli nder.html

Re:Easy

[Norgus]

If I carved my data into a stone tablet at the same data density a cd is burned/pressed surely a CD would last longer.

Re:Easy

[michael_cain]

Seriously, if you want to think in terms of 100-150 years, this is a solved problem, and without the need for stone tablets. Pigment-based inks on acid-free paper. Silver-based black and white photo chemistry on acid-free paper. Stitched bindings, not glue. Store in a trunk where there's neglible light. Put the trunk in the attic of a house where it's reasonably safe from large amounts of water (rain or flood). Civil War documents using these techniques have survived nicely to the present day. The Bell Labs archives have Alexander Graham Bell's original laboratory notebooks, still easily legible. To date, there are no reliable archival media for this length of time for audio or moving pictures. Write it down. Sketch it (as silver-based photographic materials are getting harder and harder to find). And you can be the source material for the historians of 2155 :^)

Re:easy

[slavemowgli]

I agree - there'll always be specialists who do this sort of thing. My aunt, for example, has an extensive collection of movies recorded by my grandfather during her youth, 50 to 60 years ago; none of us has the equipment to show them anymore these days, but she simply contacted a company specialising in this sort of thing a few years ago and had them transferred to DVD.

Unless the CD-ROMs etc. our grandchildren will find in our attics are *physically* deteriorated to the point of being unreadable, I don't think that we'll have to worry about them not being able to read them anymore. It might not be possible for them using their own equipment, just like we can't listen to wax cylinder recordings anymore today, but I don't see why there wouldn't be people or companies who offer this kind of service.

As long as there is a demand, at least, there *will* be people who're willing to do this - for a fee, of course.

Re:easy

[Casandro]

> Unless the CD-ROMs etc. our grandchildren will find in our attics are *physically*
> deteriorated to the point of being unreadable, I don't think that we'll have to worry
> about them not being able to read them anymore.

This is actually the biggest problem probably. Recordable CDs don't really work that well. It's extremely simple to destroy them by accident and they probably wouldn't survive laying on an attic for more than 5 years.

However there's a bit of chance. Theoretically you could store the image in an uncompressed 8-Bit greyscale per pixel format where you only actually use very few of those greyscales.

If you then try to read it later, you might not be able to read every single bit correctly. In fact you might even have lots of biterrors. But if your file only has 4 possible "characters/colours" it should be simple to determine the correct one. In fact it's even a bit simpler as CD-Roms encode bytes with 10 bits. So you basically read 10 bits and have to find out which of the 4 posible combinations is closest to them.

If you use an uncompressed format it's even better as single defective bytes, or even a few defective bytes in a row won't really make your file unreadable.

The problem in the future will most likely be reading the media. The formats should be possible to decode somehow, unless of course people use stuff like Word or DRM.

Yeah right

[Anonymous Coward]

Whatever is worthwhile to keep will be migrated to new media. Even if 90% of it is lost odds are 10 times more information will be preserved from this decade than the last. Digital media is cheaper to own and operate.

Re:Yeah right

[liangzai]

Nah, there are tons of movies rotting away in freezers at the *AA associated studios. There is no commercial interest in restoring them and digitizing them, but since THEM have the copyright on it, US will see it be lost forever.

The same applies to digital stuff. People have the only copies and/or the copyright, and it will one day go through the bit bucket because the owner is greedy / mentally insane / depressed / had a fire or what have you.

All the good digital stuff, like Asian 4 You, will eventually go down the same drain as the library in Alexandria.

Re:Yeah right

[arose]

Nah, there are tons of movies rotting away in freezers at the *AA associated studios.

OP was talking about digital files. Everything that is accessible, of some value, isn't digital yet and is in danger of rotting away should be digitalized ASAP. The worst are libraries and archives that don't digitalise things because they are afraid that the process might destroy their precious, unique, might-soon-be-gone-anyway, you-can't-see-them-beacuse-they're-too-fragile, pieces...

Huh?

[imbezol]

The directions to a treasure are pretty easy. What the hell are you doing? Writing it with a spray paint brush in some paint program, encrypting it, attaching it to a DRM'd Word Doc, etc?

Re:Huh?

[zxnos]

letter: HLr
number: 4743773
symbol: \/\/'$?

Re:Huh?

[Anonymous Coward]

Hellooooooo???
Mods???????
This is hysterical/insightful.....

Re:Huh?

[CajunLuke]

The point is that they can't read the physical CD.

Re:Huh?

[Anonymous Coward]

Damn... that's funny on TWO levels.

Easy

[joe_bruin]

How can they read my obsolete digital document?

The same way we do it today: emulators. Of course, your cdrom is not going to survive that long, so there's no need to worry about that. Have you considered leaving your legacy carved into stone tablets?

The times they are a changing

[Orionetheus]

Hopefully someone isn't stupid enough to store their will on a CD rom...would you?

Re:The times they are a changing

[dfjunior]

Hell no!
Zip discs are the *only* reliable way to archive digital data indefinitely

Re:The times they are a changing

[iamlucky13]

Times are indeed changing. How many family fortunes do you suppose have been lost because some nut stuck the directions of a Zip disk?

But it's the latest and greatest! Everyone will have one in a couple years.

Re:The times they are a changing

[SquadBoy]

*click* :)

Re:The times they are a changing

[the real darkskye]

Thats why I store all my important stuff on LS-120 discs.
That way even if someone does steal them they'd be hard pressed to find out what it is and finding a drive for it!

This is a touchy subject.

[empvirus]

Really, who knows what the future holds? And who says we won't be able to trace history back to these days and even further? And just because we don't use a media anymore means it is forgotten and no one will ever be able to read the media again. I mean, if one did some digging, I bet he/she would find information to be able to read punch-cards even. Just my 2 cents.

Re:This is a touchy subject.

[Kadin2048]

Actually many punch cards which were hand-entered on keypunch machines have the text printed across the top of the cards in a typewriter-like human readable form. Otherwise it would be hard to detect whether an error had occurred during the process of keying the card.

I'm not sure whether this is specific just to early punch cards, and went away with later, higher-speed systems where the cards were being produced by computers at much higher rates, but I've seen a few punch cards and they had numbers at the top which corresponded to the punched holes. It was complete gibberish as far as I could tell, but it was human-readable in the most basic sense.

I'm not sure though that this goes for all punch cards. Maybe it was a feature specific to the system for which the cards I saw were for.

Mediums change

[FahrenheitLF]

I have some old VHS movies that I can no longer watch because I no longer have a VCR. I just watch them on DVD instead. Important information will find it's way into the 22nd century. If it doesn't, it's our own fault, not the medium it is stored to.

Re:Mediums change

[zxnos]

that is totally cool. i need a DVD player that plays old VHS tapes.

Re:Mediums change

[Naelphin]

like this?

another

Old Computer?

[Anonymous Coward]

Well, someone is bound to have an old computer lying around. I know I have an old LP player, and an 8-track player, and I probably won't throw these out. So why throw my computer out?

Huh?

[tktk]

...they come across a CD-ROM and a letter...

\/\/H47'$ 4 L3773r?

Reminds me of a book

[ilselu1]

The White Mountians by John Christopher...

http://www.amazon.com/exec/obidos/tg/detail/-/0020 427115/103-3554463-5084646?v=glance

a lesson on impermanence

[puzzled]

  Each moment arises out of the moment before - call it 'dependent arising'. No object exists in perpetuity - even black holes evaporate over long time spans.

  This being said, our digital storage systems, in a collective sense, are becoming more like a brain and less like an archive. 'Memories' of some importance are in multiple locations and accessible via different search methods. They're also being changed, just as memories of our pasts acquire a patina as we age. Someone took something I wrote in the early 90s on Usenet and added it to their humor site. My flickr content is spreading if the hits are any indication, as are my contributions to YouTube.

  Public records are an important thing, but understand the other, positive things that are happening in the background as the the internet acts less like a database and more like a neural net with each passing day.

Doesn't take that long ...

[angst_ridden_hipster]

I have a bunch of old DSDD 40-track hard-sector TRS-80 5.25" floppy disks (NEWDOS/80v2 format) that I'd love be able to read.

Unless I want to build custom hardware, I don't believe it can be done...

And those are only ... uh ... well, OK, twenty to twenty-five years old.

Re:Doesn't take that long ...

[Murphy Murph]

Are you suggesting that your TRS-80 had 1% of 1% of 1% the market penetration of CDs?

Apples to oranges my friend.

Besides, what is stopping you from reading that data on an ebayed machine, printing it out and OCRing it?

Re:Doesn't take that long ...

[jcr]

Unless I want to build custom hardware, I don't believe it can be done...

There are service bureaus that will read those disks and stick the data on a secured server to download. Hell, you can even get paper tape and hollerith cards read for you.

-jcr

Re:Doesn't take that long ...

[angst_ridden_hipster]

Nope. I wasn't suggesting that.

I was just talking about how the "dominant format" of a given time falls out of dominance very quickly when it goes. Obviously, even when TRS80 was one of the big two or three formats, there were still only a few million of them out there (source: http://www.pegasus3d.com/total_share.html)

But look at 8-track audio tapes. At one time, they were quite popular. You could buy 'em in every dimestore. You can still play 'em, if you're willing to spend a lot of time and effort. But it's getting harder and harder. Audio cassettes are heading in that same direction. They're rarer and rarer.

As for the eBay/OCR situation: it might be possible. Every few months, there's a similar machine for sale. I've tried reading the disks in other machines, and haven't had much success. I think that the data integrity must be fading over time.

Re:Doesn't take that long ...

[SVDave]

  I have a bunch of old DSDD 40-track hard-sector TRS-80 5.25" floppy disks (NEWDOS/80v2 format) that I'd love be able to read.

So get yourself a PCI Catweasel MK4 controller (or MK2, if you have an ISA slot), a Linux box, a 5.25" floppy drive, and this guy's software.

Done. And it took me all of two minutes on Google to find.

And, given that for every TRS-80 hard drive sold, I would guess that somewhere around 1 million CD-ROM or DVD-ROM drives have been sold, I'd say you'll still be able to find hardware to read CD-ROMs in 2105, never mind 2045.

Whether your CD-Rs will last that long is the real issue.

Re:Doesn't take that long ...

[angst_ridden_hipster]

Really? Any you could recommend or provide a link to?

Re:Doesn't take that long ...

[Halfbaked Plan]

Besides, what is stopping you from reading that data on an ebayed machine, printing it out and OCRing it?

Besides the fact that he doesn't have the 'ebayed machine' what stops him from doing that is he isn't stoopid enough to do the second half of what you suggested.

Anything you can 'print out' by sending to a printer, you can capture digitally by intercepting the signal that would have gone to the printer. And with far greater ease than some horrible OCR kludge.

Re:Doesn't take that long ...

[jcr]

Google for it. It's been a pretty long time since I looked into those services.

-jcr

Re:Doesn't take that long ...

[Stonent1]

When's the last time you've seen a floppy drive that reads hard sectored disks? Hard sector disks have multiple holes punched in them, not just one like soft sectored disks have.

Re:Doesn't take that long ...

[orkysoft]

This guy can probably do it. He'll want some green pieces of paper in return, though.

Re:Doesn't take that long ...

[advocate_one]

excellent... now I can get to see my ascii p0rn from the 80's

SESSION #18 - SPEAK LIKE A CHILD

[infonography]

Subject of a Cowboy Bebop episode. This is why I watch anime. They actually take some time to examine an idea like where to find a Betamax player 150 years from now. http://rfblues.aaanime.net/Sessions/session18.htm

Re:SESSION #18 - SPEAK LIKE A CHILD

[Dachannien]

For that matter, where do you find a Betamax player now?

Re:SESSION #18 - SPEAK LIKE A CHILD

[Anonymous Coward]

They actually take some time to examine an idea like where to find a Betamax player 150 years from now.

Isn't that easy? Start digging, and once you hit garbage dig down to the 1980's strata and viola! It shouldn't be hard to find with all the wigs, outrageous clothes, and disco ball(fragment)s.

Re:SESSION #18 - SPEAK LIKE A CHILD

[Anonymous Coward]

eBay.

The same place you will be able to find one 100 years from now.

I suppose this means

[$RANDOMLUSER]

That I could throw away my file drawer full of 8" (DS-DD 128K) floppy disks full of 8080/8048/8051 assembly code; but then what do I do with that MDS-235 in my basement?

Give it to me

[fumanchu32]

Just give me the document. I'll print off a hard copy today, that new fangled paper technology looks promising (Assume acid free paper, no sunlight, etc, for you picky individuals). Just leave them a cd with my contact info. I will give them the directions to the family fortune, I promise. You can trust me, I'm a [insert political party of choice here].

This has come across my mind as well

[linzeal]

I put it in the form of an even more extreme question, "How do you build a 10k year machine?"

Materials science is in its infancy as reactive armor will barely keep up with advances in something as simple as an rpg. I say that their trully has not been an advancement in human culture since the iron age where since than we have used carbon steel to build dams to skyscrapers yet always staying within certain paramaters. What we need is a material that ignores all current scales when it comes to building tanks, buildings or space elevators. I am hopeful like the rest of you that such a material as carbon nanotubes may be a step in the right direction but as an engineer who merely uses alloys he knows has certain properties I ask slashdot, "What are some other novel materials of the past 10 years?" I know about fullerenes, aerogel, and magnetic plastics what else is there?

The reason I want to build a 10k year machine is to begin preparatory work on something that can reasonably be able to reach another civilization within 150 ly at less than 1% the speed of light.

Re:This has come across my mind as well

[geekoid]

easy.

Make easy to assemble parts into a can.
Carve direction on how to assemble machine.
Seal can.
Launch can into space and point it at where your object will be in 10K years.

If there is no one there that can put it together, then there isn't anyone there worth getting a messaage to.

Re:This has come across my mind as well

[PatrickThomson]

You forgot amorphous metal.

5x-10x stronger, tougher, more expensive but intrinsically better.

The format is probably not relevant

[hungrygrue]

as the CD probably couldn't be read regardless. CDs do not last forever. http://www.warehousephoto.com/How_Permanent_is_you r_CD-R.htm In fact many will be unreadable in as little as 2 years. If you want to archive, print it with good ink on acid free archival paper.

Re:The format is probably not relevant

[liangzai]

Aye. I prefer runestones, although the rune porn is less juicy than MPEG-4.

Re:The format is probably not relevant

[adrew]

I'm the tech for a school newspaper ... they started backing files up to CD in '96. This summer I backed all (100+) of them up to DVD and didn't encounter any that were unreadable, except those that were all scratched up. These haven't had any special care; they're just in jewel cases in a drawer.

I'm a firm believer in transferring files to the current format du jour every couple of years. Right now it's 4.7 GB DVD-Rs.

Re:The format is probably not relevant

[rootofevil]

wouldnt a laser print (toner) be a better choice? inkjet printer ink tends to suck for archivability.

Re:The format is probably not relevant

[hungrygrue]

Heh, compact dick? sounds like a floppy to me.

Re:The format is probably not relevant

[Anonymous Coward]

From the link, http://www.warehousephoto.com/How_Permanent_is_you r_CD-R.htm :

"Image permanence for C D-R's (Compact Dick Recordable) is an area that is not well known or understood by the general consumer or by many photo labs."

I didn't know CD-R stood for Compact 'Dick' Recordable.

Re:The format is probably not relevant

[Guspaz]

Archiving reports vary by manufacture, but 70 years would be low for a quality CD-R with the norm being 100 years. Some manufacturers of Gold CD-R's claim 100-200 years!

Obviously 40 years won't be a problem if the proper CD-Rs are chosen. Obviously there are more permanent digital storage mediums if you need extremely long term storage, but I'm uncertain as to why anyone would want to ensure that a piece of data could exist for more than 200 years. Over that sort of a time frame, any important piece of data is unlikely to stay on one piece of media. Want to store family photographs? Put them on a CD-R. If your kids care enough to pass them on to their kids, they can copy the image files to whatever the media-du-jour is.

Re:The format is probably not relevant

[alc6379]

wouldnt a laser print (toner) be a better choice? inkjet printer ink tends to suck for archivability.

You say that like an individual who has never had laser prints "flake" away. I've used numerous laser printers ranging in age from 1 to 10 years old, and the toner sometimes doesn't adhere that well.

I was using an old HP LaserJet 4+ once (which is actually a work horse printer, don't get me wrong), and I lost nearly half a page of text due to having to stack/straighten up the pile of 15-20 sheets I had just printed-- the toner just flaked off. And, before anyone mentions needing to let the paper cool, etc, I've had the same thing occur with documents that have been left untouched on a desk for a day or more.

Re:The format is probably not relevant

[picz plz]

Soon, once we get cheap portable DNA decoders. However the first versions will only accept 3 1/2" floppy dicks.

Re:The format is probably not relevant

[ezzzD55J]

Put them on a CD-R. If your kids care enough to pass them on to their kids, they can copy the image files to whatever the media-du-jour is.

The older something is, the more interesting it tends to be. So 'his kids' are not the right people to go around making that decision - their kids might be more interested than they are.

Even in your own life you (and with you I mean one, in general) can wish for something back that you'd thrown out, right?

Re:The format is probably not relevant

[anthony_dipierro]

Even in your own life you (and with you I mean one, in general) can wish for something back that you'd thrown out, right?

Sure, but not nearly as often as I wish I had thrown something away sooner.

Re:The format is probably not relevant

[Dogtanian]

Aye. I prefer runestones, although the rune porn is less juicy than MPEG-4.

Runestones are just the physical format; there's nothing stopping you from storing binary-encoded MPEG porn on them.

I have the contents of the 'College Girls Go Wild #29' DVD stored on the remains of half a mountain for example. Unfortunately, the transfer rate from rune to computer is too slow to show it at full speed. In fact, three of the guys carrying the porn from the store room to my computer died of exhaustion, as 137 runes per second was just too much for them.

Interesting - historians' concerns

[BitterAndDrunk]

I read an article about 10 months ago about the "death of history" due to the electronic age.
In a nutshell, as we've moved to more digital forms of communication (phone and email), one of the primary methods historians use to piece together older eras is going extinct - the written correspondence from one person to the next.
It was an excellent article; my google-fu sucks apparently because I can't find hide nor hair of it. Curses. No +5 Informative for me.

Re:Interesting - historians' concerns

[fermion]

The problem with this is that there is probably more documentation of what is going on now than every before. Everyone wants to harp on a stone tablet that survived. We spend millions of people hours studying it, extrapolating information, but what have we got? A list of goods sold at one time during a particular regime? I mean we assume that was is written is true, but where is coroboration? It is not like we often have hundreds of stone tablets from each year and territory. We tend to assume things and hope our results are correct.

Then we deal with letters written hundreds of years ago. That is treasure trove, but how many letters were destroyed for every letter that is found? Do historians really want to go through every letter that has ever been written? Do historians read every non-fiction book available?

Even with phones and email, we probably have a better archieve than ever before in history, at least at the official level. Phone calls are recorded, a la Nixon and Kennedy and the Katrina discussions. So much is on email. So many books are going to survive. The future historians are not going to have trouble gaining original documents, baring armagedeon, they will only have problems working through them.

One last thing. The dark ages, IMHO, were a result of certain embeeded interests that saw thier fortunes were to be destroyed by certain advances, or certain forces who wished to conquer other more advanced forces. The dark ages ended only when human curiosity won over the forces of human stagnation and starting exploring the world without hubris. Some of this was kick started by the reading of the old texts, but much of it was merely the victory of those who wanted to learn over those that thought we already knew everyhting.

Re:Interesting - historians' concerns

[Zangief]

Well, historians will move ahead, and they will plenty of search in order to reconstruct story:

-newsgroup
-email (USA courts decided that email can be inherited)
-slashdot archives! yayster!

Re:Interesting - historians' concerns

[FlopEJoe]

After reading some of the blogs out there... I'm not so sure I want our era to be remembered by that data.

Re:Interesting - historians' concerns

[mlush]

I read an article about 10 months ago about the "death of history" due to the electronic age.
snip
It was an excellent article; my google-fu sucks apparently because I can't find hide nor hair of it. Curses. No +5 Informative for me.

Hmm that anecdote makes the point rather eloquently, Both the transience of electronic records and the difficulty in finding them.

empyre

[l00k]

People might also be interested in a fairly extensive discussion about these issues that took place on the empyre mailing list in February.

Digital dark age is here...

[bigtallmofo]

We live in an age where almost everyone has gotten rid of their 35mm camera and replaced it with a digital camera. Most such people have no idea how to use them. I have a dozen or so family members with 256 meg flash RAM in their digital camera, and it's a good thing because they have no idea how to copy the pictures to their computer. And what if they did manage that? The next inevitable hard drive crash would make them lose all of them anway.

I automatically copy my digital pictures and mini-dv files from my workstation to a server on a nightly basis. Then manually once a week I copy it to a removable hard drive inside a USB controller. Then every few months I make DVD+R backups. I'm still not satisfied - I'm looking into Streamload.com as a cheap Internet backup.

Re:Digital dark age is here...

[myowntrueself]

I'm still not satisfied - I'm looking into Streamload.com as a cheap Internet backup

Right, so as in the example in TFA, you'd be leaving what in the attic?

A URL printed on, of course, acid-free paper?

Nice try...

Re:Digital dark age is here...

[merreborn]

Nice job slipping the referal link in there.

Commission junction, eh?

Re:Digital dark age is here...

[Devistater]

LOL! One of my aunts and one of my uncles is that way. They had no clue how to get pics from camera to computer, they were complaining that thier 512 meg XD card was full and asking me what they could do. They thought they had to buy another card to take more pics!!!

The tools are not the problem.

[glwtta]

You'll have no problem obtaining a device capable of reading a CD in 40 years, software to read any file format that ever existed can be obtained from this thing we call "the internet". If it's a particularly esoteric format you might have to spend a little time with it, though with the family fortune at stake that doesn't seem like much.

The main problem is that in 40 years the organic dies on that CDR (I'm assuming) will long have degraded and the disc is completely and utterly unreadable. In fact that only needs about 2 years.

Re:The tools are not the problem.

[glwtta]

"dyes" is of course what I meant there...

Re:The tools are not the problem.

[limabone]

Why do people keep saying CD's die in 2 year/5 years/x years? Has anyone actually had a CD die on them? I have CD's in front of me at this very moment that are over 10 years old and still work great (yes I did in fact test them). Is there some conspiracy by the blank CD manufacturers to make you think all your CD's are going to die so you need to keep transferring the contents from one disk to another forever?

Re:The tools are not the problem.

[6*7]

Yes I have dead CDs, they were recorded using a cdd2000 some 10 years ago. Offcourse they were noname elcheapo disks. Still perfectly working CDs (which are even a little older) were "produced" by Sony, Kodak, Philips etc. These were a little more expensive offcourse.

But that reminds me to really really take some time to read the old stuff and put it on DVDs, which offcourse should be repeated every 5 years atleast (which the obvious upgrades to the then current media).

Re:The tools are not the problem.

[mikael]

I have some CD's from the 1980's which are not playable on current players, because they have over 40 tracks each (Ruby Trax Charity CD). Another CD (Billy Idol's Neuromancer) claims to have digital images between tracks which could be read by computer. Although there is no explanation on how to do so.

Use an emulator

[Anonymous Coward]

'nuff said.

More importantly

[Lord_Dweomer]

More importantly...even though they might be able to find whatever software/info they would need to configure something/buy legacy hardware to run it...there's most likely not a chance in hell that the CD would still work after that length of time due to how quickly the information decays on them.

The big long term problem with our increasingly digital world is data decay from all our archived information.

The person would be better off inscribing the information in stone for their descendants to find because at least we know that stuff can last thousands of years.

I think that..

[slapout]

..a more likey outcome is that patents and DRM will lead to a digital dark age.

Re:I think that..

[Gibsnag]

Thats what I initially thought that the article was going to be about when I saw the article title.

It seems that digital archiving is a small problem in comparison to the possible dangers of DRM, considering that there will always be some kind of format which most likely will be contingent for years (for example .txt for plain text).

Its most likely that there will always be a crazy specialist somewhere who deals with 10-20 year old hardware and could get your archaic data into a modern format for you, assuming that your media hasn't disintegrated over time (which is probably a harder problem to crack).

Directions to a treasure?

[Anonymous Coward]

Yarrrr!! On this CD be th' directions t' me booty an' some o' me best porn.

2045 is a bit soon.

[llamaguy]

2045 seems a bit soon - after all, we can still get pretty detailed information on the mainframes of the 1950s. Also, CDs are a -very- well known format, and have been for the past 10-15 years at least, so they'll hardly be marked down as insignificant. Now, the formats on said CD could be more problematic, but 8/16-bit Unicode or ASCII are again very well known protocols and text written in them should survive. If my descendants can't figure out how to read a popular hard storage medium or what's on it, I don't want them to have my family fortune.

Re:2045 is a bit soon.

[Anonymous Coward]

OK, thats theory on your part. So I present a challenge. As an exercise, can someone place any extant 1950's mainframe operating system code, probably assembler, on current media (i.e. ASCII on a DVD)? Would be good to have the a compiler for that ASM. Then outside archival, can that compiler be emulated in a subset of a current operating sytem? Finally can that revived mainframe OS run COBOL, or the like, and print 'hello world'?

Also, do you recognize that this is like one of the exercises in an Andrew S. Tanenbaum book?

Thanks,
Jim Burke

Easy, just pull out your laser key-fob computer

[myth_of_sisyphus]

Hold the CD-ROM in front of it, the laser will read the bits--even degradaded--figure out the compression scheme and project it onto the back of your retina.

easy

[Anonymous Coward]

you could hide the data in an internet meme, like the time I took a picture of my unusually large anus so I wouldnt forget my passwords.

Besides the media incompatibility...

[Jugalator]

I guess most average CD brands would have deteriorated beyond rescue long before 2045 anyway.

Heck, that's something I have to remind people using CD's for digital photography even now: never buy CD-RW's, always burn new ones. They're so cheap anyway, and you get some redundancy, and there's less risk of them simply going bad from a brand of worse quality than you expected.

As for the article, yes, it's quite important to make the transitions and not miss out more than say 3-4 generations!

Not really a problem

[geekoid]

people migrate their data to new technologies all the time.

Think about it. A person gets a new computer with the latest technology, then they transfer their data to the new machine.A contant upgrade cyscly.

Same with lerge businesses, they may be using a tape library, but they upgrade there tapes regularly. And if some came out with a 1000 terabytes in a cubic inch of crystal storage device, they would also ahve a way to migrate there clients data. If they didn't they would have a hard time selling any.

Re:Not really a problem

[joel48]

The largest problem isn't the current data that is often migrated, but static archived data that no one bothers to migrate, either because it's in "archive" or gets forgotten because no one uses it anymore.

Re:Not really a problem

[L.Bob.Rife]

Well, you could call it a "problem", but the name "Digital Dark Age" is a bit alarmist. Not all information survives, and it never has. We have at the absolute most, 1% of the documents from 100 years ago, and in a hundred years, due to digital archiving etc, we'll probably have 5% of todays docs. Sure you can bemoan the loss of 95% of data, but you'd be ignoring the 500% increase in how much we've saved.

The numbers are of course all made up, but the point remains valid.

Re:Not really a problem

[Geoffreyerffoeg]

Indeed. Think about non-computer works from the ages.

Take Shakespeare - the standard editions have their spelling fixed, and there are many editions paraphrased into modern English.

Take Dante - the Italian he used, though it practically created the language, is most definitely not the common modern vernacular. Or as another example, I know modern, standard French pretty well. With some effort I can read Le Petit Prince (1943), but it's not enjoyable. I can kinda get the gist of Les Miserables (1862). I looked at the original Song of Roland (11th century) once and the only words I recognized were stupid stuff like "le" and "est" (and I think even those weren't in their modern forms) - the vocabulary looked very different. But these works are still read today; they aren't lost.

Take Beowulf, which was written in English. I'm assuming everyone here knows English. But "Beowulf is min nama" (Beowulf is my name) may be the limit for modern English speakers trying to read it. Can you understand (without cheating) the meaning of "his modsefa manegum gecyðed, wig ond wisdom"? And even though there's only one manuscript, and that bound, burned, rebound, and now missing letters, we still have modern versions, and the manuscript remains as intact as possible in the British Museum.

What about The Epic of Gilgamesh? The thing is carved in a Sumerian stone tablet, yet there are translations today.

If words on stone tablets from three or four millennia ago by people who didn't even suspect forward-compatibility problems is still readable as an e-text, I'm fairly sure there will be a mechanism to get important files back in 50 years. There will be some important material that will be continually transfered to more modern materials - think of the US Constitution, which was written on sheepskin, afterwards published in books of the period, and now available on the Internet. Of course, not everything will be saved, just as Shakespeare's doodles from age eight haven't been recovered. But just as we can analyze parchment and stone tablets, people will analyze CDs - even deteriorated ones - so long as the content is sufficiently important. Just as always.

Re:Not really a problem

[Anonymous Coward]

It's when you swap platforms that you get problems. For instance, I can't see any stuff on my ZX Spectrum tapes any more (not that I'd want to), can't play any of my old Amiga games that are on disk as I don't have a drive that'll read them (I can get ROMS though, so that's a partial solution).

Most annoyingly, I can't retreive either my old Amiga Wordworth files, or my old PC Amipro files - nothing seems to like them at all..

Re:Not really a problem

[Anonymous Coward]

For disks, get a Catweasel Mk4 controller and a 3.5" and 5.25" drive, and DISK2FDI.

For tapes, play the data from a tape recorder into your audio input and then have it decoded to binary. Check emulator-related software, it's there.

CD Rot

[Malicious]

Don't forget about CD Rot. While you'd like to believe that if you put that Treasure Map on a CD so you can find the treasure years from now, chances are... your map will have disspeared on you.

This is why I still get my digital photos developed. Last thing I want is all my treasured memories to become suddenly un-readable someday.

Re:CD Rot

[pharwell]

And as a musician, this is a concern for me. How long will my CD's last? I hope to someday be able to purchase a record cutter. Vinyl LP's will last a very long time if stored properly. Look at the vintage vinyl industry. Beatles records from the 60's are still playable 40 years on, but CD's made in the 80's may already be subject to CD rot.

This is how they read it...

[kage.j]

A text file. They can open the thing with a text editor!...I don't think .txt will ever be obsolete.

Re:This is how they read it...

[Anonymous Coward]

actually, even if the file was in a proprietry word processor format, a "strings"-like program easily extracts the ASCII text, which carries most of the relevant information (except for mathematics - one of the reasons I use LaTeX for mathematical documents)

Let Google worry about it

[rarewire]

Make your problem Google's problem: Mail yourself all your archive files to your Gmail account

The Short Answer

[Caraig]

The easy and short answer is to not rely on any middleware: use printed word. If you pack any sort of digital media, it will either degrade or it will not be able to be read. If you pack, say, a PDA, or even a laptop. there's no garauntee that the storage media will survive the decades, either, or that the same electrical power setup will exist then.

On the other hand, a written message on non-acidic paper (probably some kind of vellum,) properly cared for, can last for a long, long time. And you don't need to run it on a computer to read the message. All you need is at least one Mk.0 Eyeball. Of course then you run into the problem of having someone to translate it... but it seems to me that this is a much easier task than trying to find decades-old hardware and trying to reconstruct magnetic bits which may or may not be in the right order.

Here

[liangzai]

People in China still rely heavily on diskettes and other gadgets found in computers from the previous century, and I am sure this is more true in India and in Africa. Thus, in 2045 we need look no farther than the poorer parts of the world to find older equipment. Just look at all the 50s cars found in abundance on Cuba.

The question is rather if the USA exists in 2045. There are other, more important questions as well, and this is a non-issue. People who update technology usually transfer their stuff to their new medium. If they don't it just means it is not worth preserving anyway.

An interesting drawback to digitalization

[kerohazel]

Reminds me of a discussion I once got into about analog vs. digital storage. Some of the people on the analog side argued that the myth of digital media being everlasting is false -- which it is. Digital media, on their own, should be seen as temporary storage. The true virtue of digital media isn't even the media itself -- it's the content. Content is what can be copied over and over again with no degradation.

Like oral traditions, the chain of copying needs to remain unbroken for any information to truly last forever, outliving "mere mortal" media. As long as P2P networks continue to exist, I can die happily knowing that the sum of mankind's knowledge will be floating around there somewhere... even if it is buried under millions of terabytes worth of lesbian porn. ;P

Offtopic: Poor rendering of Slashdot in Konqueror

[Anonymous Coward]

Wow, Slashdot editors, what did you do to the slashcode with your CSS conversion? I am trying to view it using Linux 2.4 kernel and Konqueror browser, with Java, Javascript, animations, Flash and cookies all disabled the way I usually surf. In the past the Slashdot site always used to render nicely. Now, at work on Internet Explorer it still looks fine, but here at home using Linux/Konqueror there are things like huge blocks of empty or dark space (is some animated advertisement supposed to be running in that space or something?), and weird extraneous lines cutting across the numbers on the page (e.g. if it says "280 out of 360 comments," there are little annoying lines through the numbers).

Weird that the site renders better in IE. And before you ask - it is not my browser - other sites look great as always. Sorry to be off-topic, but thought you guys might want to know.

Who cares?

[Chysn]

So we're going to lose our information. Who cares? Proton decay will eventually destroy all of it. Sure, that's a long time in the future. You know how things go: it's 10^1032 years away today, but before you know it the kids have moved out and the end of the universe is right around the corner.

Just try and keep those bits in line.

This is important.

[LordSnooty]

This is a very important topic, has been mentioned for years, and of course it has already been demonstrated in real life... the BBC Domesday Project of the 80s, where kids across the UK were asked to submit their own descriptions and images of their local area, much in the way of the original Domesday Book of 1086. It was collated on the default school computer, the BBC Micro, and packaged up & sold back to schools on huge laser discs. Except as the 21st century arrived, no-one had any readers. A basic example, perhaps, but one that actually happened. As it turned out, they managed to get to the data, after appeals on the web & in the press, and it can be browsed here.

Strangely the article barely touched on physical degradation. This is a bigger problem. We don't know how long these cheap late 90s CDs will last. However that's the same for any media, from paper and photos. The advantage we have is we can easily run off an exact clean copy on fresh media - this why I've started to date every CD/DVD I burn.

Dupe?

[minus_273]

Man i thought i saw a similar article somewhere before

Re:Dupe?

[Anonymous Coward]

Silly me. I immediately thought it might have been a lead in for a dupe too, but I was apparently going back a few years farther. (It doesn't even show up in "search" anymore here, but...)

In 2002 the BBC found they would soon have trouble reading an LV-ROM Laserdisc created in the 80's since the format is completely out of production. They ended up having to read and decode the disc seperately using an emulator for the original hardware, which, if the DCMA weren't around, might someday be legal here.

http://en.wikipedia.org/wiki/BBC_Domesday_Project

  -- dut

well, if you are looking for

[geekoid]

a tech story from the past, just go to slashdot...It will be back.

Digital Archeology!

[kbahey]

This is a topic that I thought about a while back, and even wrote an article on.

There are also some success stories with old media.

I hope our data does not meet the fate of Hieroglyphs: undecipherable for two millenia.

Already happened

[slashname3]

I have a box with several dozen reels of super 8 movies that were taken by my family many many years ago. Last time I found someone that might be able to convert those reels to digital format the cost was very high.

It is very likely that all of those films are lost at this point.

Re:Already happened

[Dwedit]

There's LOTS of people who will convert that stuff. Shop around for a better price.

Re:Already happened

[Halfbaked Plan]

Actually, those reels of film will still be viewable 75 years from now. If you'd had them digitized, the VHS or DVD media they would have been transferred to would be long gone by then.

Re:Already happened

[slashname3]

But where do you get a projector to view the film and the splicing material to fix the film when it breaks?

In the current format no one can view it. If I can get it converted to DVD format I can not only view it but currently easily duplicate it and trasfer it to other media formats.

Last time I checked it was going to run something like $1000 to get this done. At the time was to much.

Re:Already happened

[Halfbaked Plan]

In the current form _anybody who cares_ can view it by making an effort. There will continue to be projectors, and in fact, fifty years from now projectors will probably be more accessable and servicible than any of today's digital media. The film is an archival format. It doesn't matter that joe sixpack, right now today, can't go to WalMart and buy a machine to view the films on.

Paper

[nurb432]

That is why paper books are so important.

If stored properly they will last a thousand years..

Re:Paper

[antifoidulus]

Heh, thats something I didn't understand in the scenario mentioned in the summary, why would someone create a paper explaining a document on a cd, but then not bother to print out the document itself? Seems a bit weird to be combining "formats" like that if you will. More than likely what would happen is that the grandchildren would find a spindle of cds that may contain old family photographs and throw them out not knowing what they contained(priceless family memories or they could just be leisure suit larry games)

Re:Paper

[nzkbuk]

The hard copy could easily be the inlay card. Most CD's have them.
So there you've got a very short paper / thin cardboard document, but you're not about to print reams of paper (the content of the CD).

Anyway if you want the content to last have any text aspects in ascii, and pictures in an uncompressed bitmap format.
Probably also a good idea to include a program that can display the pics with instructions on what sort of computer & OS it ran on (they might be able to find an old emulator).

Re:Paper

[MikeFM]

What we should do is record several copies of every slightly important data to alumnium plates that could be read by an industrial-age culture written in simple binary. Include an etched howto picture guide to build a reading machine on a similar plate in every storage facility. Include an etched picture guide explaining the basics of binary. Include an etched picture guide to the language the directions will be written in. Include plates that gradually teach how to convert this written text into binary and vice versa. Then, as binary written on the plates, gradually build up directions for reading various other media types from binary and converting them to something usable by humans. THEN store your data.

I'd suggest being a little nuts and etching the most important directions into a large land mass that will likely be around for thousands of years at least. Maybe on the moon in large enough print that a telescope would make the directions readable? Maybe the aliens have had the right idea with their crop circles. It sounds silly but for truely making backups it'd be great. :)

Re:Paper

[ChrisMaple]

Aluminum is not a particularly good choice. It's soft and it oxidizes.

Re:Paper

[MikeFM]

Probably so. I'm not especially an expert in metal plates. I just have experience at how easy it is to carve in aluminum. Some sort of cheap, easily carved metal plate then..

not an issue

[real_smiff]

reading CDs will be trivial for evermore.. unless something really serious happens to the human race, and then we'll have much bigger things to worry about, like how to build vehicles, or houses. i really find this kind of scenario bizzarre. 2045 is only 40 years away. when was the LP invented? could you build a rudimentary LP player in 10 years? i think so. its only going to get easier.
now, reading old hard disks could be more difficult because both the reader and media are combined, i,e, the interface between the two is not standard, ironically as that should make it simpler?

So true!

[HexRei]

...because, you know, nobody has record players, dat tapes, or 5 1/4" drives anymore...right? Come on. At some point, yes, it will become a niche item, but short of a world-wide holocaust it will never be impossible or even preposterously difficult to recover data from old formats.
At worst, you'll send it to a specialty studio to transfer to another format, at best, you'll call up your friend who loves those retro CD's (they just SOUND better than quantum cubes!) and have him transfer it for you.

Re:So true!

[wertarbyte]

...because, you know, nobody has record players, dat tapes, or 5 1/4" drives anymore...right?

Indeed. It's hard to retrieve data from a 5,25" disk, and it won't get easier with the years going by - 3,5" FD is walking the same direction, many new computers don't ship with that old and small drive anymore (Thank you iMac). NASA as well has tapes where no readers exist anymore - IIRC from the viking probes

Re:So true!

[HexRei]

What type of tapes were those? I'm willing to bet they were decidedly less wide-spread than 5.25" was. Sure, it won't be EASY to pull the data, but the idea that there will be a digital dark age- that we will permanently losing massive quantities of important data- is silly.
Like I said, you might have to walk it into a local shop, or send it in to a studio somewhere, but there will be someone willing to accept money in return for retrieving the data, even if it means repairing old drives by hand.

Similar issues with old movies

[Alien54]

All too often these are literally rotting away in storage, because the originals are decaying, and the movie companies are unwilling to invest money to rescue them, even though they would sue you for millions if you published these on your own.

Re:Similar issues with old movies

[Cylix]

That's why we have companies that will track down a reel that is in public domain and recover it.

Sure technology that is even 10 years old gets lost.

It's the nature of the beast.

There are ways to store data so that it lasts. It's just a little expensive.

Someone should burn a cd, lock it away and come back and tell us how it works in 5 years. Do it again in 10. I bet you can get 5 or so mod points out of it.

Re:Similar issues with old movies

[jimmydevice]

I have 10 year old CDRs, Gold backing with the dark purple / blue phenol? dye, burnt on a 1X scsi plexstor in 1995. Those still read flawlessly. I also have some cheap Al / yellow dye, They lasted about 3-4 years before starting to generate checksum errors.
It all depends on the media and storage conditions. Conditions here are very dry 20% Humidity most of the time and stored at room temp.

Re:Similar issues with old movies

[log0n]

I've got CD-Rs burned in late 1995 that still work reliably today. No problems with the data. Yamaha media (yellow/golden discs - very funky compared to a normal CDR) simply labeled as 'CD Recordable'. A college roommate went crazy and splurged ~ $1k on a SCSI Burner (I think it was also a Yamaha - I know he had a Plextor at one point later). Ahhh those were the days - 2x CD drives were almost the norm but still highly regarded, 4x were luxury beyond measure.

That being said, I've also got CDs after that point that no longer even mount. Those 1995ish Yamaha discs (around $20 each iirc) must have been special.

Re:Similar issues with old movies

[Steve Franklin]

Interestingly enough, despite the commonly accepted wisdom that the loss of the material stored at the Library of Alexandria was the result of successive burnings, an analysis by Luciano Canfora ("The Vanished Library") shows that it simply crumbled to dust because the ongpoing process by which it was continuously copied and recopied was interrupted. If you really want data to survive, you need to put it on something that will physically last, like the clay tablets from Mesopotamia. The notion that civilization somehow began at Sumer is a direct result of the fact that their documents survived burial in the ground for thousands of years. This is the standard to which you need to aspire.

Not to worry...

[PrntlUnit27]

In 2045, the debut of Longhorn will take care of this.

not a new concern...

[beefguts]

I refer to this as digital amnesia

Media Evolution and Digital Photography

[TFGeditor]

Media evolution and subsequent obsolescence is what keeps may photographers from adopting digital cameras. Slide film images, though not "forever," are certainly more enduring and readily adaptable via scanning to whatever digital storage medium is the current state of the art.

Hardcopy backups

[gregmckone]

Write the location of the paper backup on the CD.

Problem solved.

In any system never build it more complicated than need be, never build a pyramid of dependencies that will have its base knocked out.

This is why depending on a centralized electrical grid instead of decentralized sustainable development is such backward thinking.

Greg.

Transfering

[chriso11]

There is one way to go - rely on the HD manufacturers - they have been able to continually improve storage density. Now, the only data I store on optical media is my Ghost files, so that I can rebuild my PC. Everything else is stored on a set of three HDs, which are sync'ed on a pseudo-random basis. One HD is stored away from the home, in case of fire.

I expect in 20 years, everyone will store their data on the internet. In that time, we will trust the internet to hold our data. Why keep local storage, when you would have a fast connection to internet all the time? We will probably be complaining about the evil Google monopoly, which owns the storage on the internet or something.

Pfft.

[po8]

First of all, you can still buy 60-year-old wire recorders. What are the odds that you can't buy a vintage CD drive and enough vintage hardware to bridge it to the present day 50 years from now?

Second of all, any competent engineer with a scanning digitizing optical microscope and a copy of some books from the library on formats could put together a workable CD reader in about a week today. Think how easy it will be in 2045.

Yes, the CD may have degraded hugely by then. But if there's any redundancy in the underlying data, there should be enough left to hit the high spots. Ironically, the scanning microscope plus clever image processing should be way better than a standard CD player at this.

I think all this "legacy data" scare is just hype. I have an 250KB 8" floppy at my house right now that was written around 1975. If I could possibly bring myself to care about its contents, I'm pretty confident I could get the data off without much trouble or expense. But oddly, almost all data I actually care about has migrated right along with me to my 500MB of local storage. I see no reason to expect this situation to change.

Data Worth Saving...

[Fortress]

...should be migrated as formats/media change. Got directions to a family fortune? Don't burn a CD and hide it in the attic. If data is important to you, you should be backing it up regularly anyway, so the dying media problem should take care of itself. As for data formats, just make sure you can access all your data in the programs you use for that data type. That way, when you change programs/formats, presumably some form of converter will translate your important data to the new standard.

That said, I think there will be a burgeoning market in the near future for data-archaeologists that specialize in data recovery from old media. Hold on to your floppy drives!

typo

[gregmckone]

Sustainable generation (sorry not development).

Like solar & wind generation in every town.

Special software to read files?

[pherthyl]

Serves you right for making the treasure map in MS Word!

just reread summary (heh)

[real_smiff]

oh you're talking about degradation of media? oh well if it's gone, it's gone. maybe you can recover some, but thats current tech, and it's digital, so nothing magical is likely to happen there. anything that people care about is naturally being preserved through copying.. p2p could save the world, who'da thunk it ;)

there is going to be no digital darkage...

[3seas]

... except for stuff that has copy protection on it...

why? because anything anybody wants to preserve they will either copy it over to newer larger space media or the archiologist will build the device to read the old media.

if there is any concern its with teh ability of the media to hold data... but we were all told how much better cds are to tape and floppy at holding information....

so its on the media industry to be sued when the truth is exposed....????

cd's are to last at least 100 year???

of course there is always writing it out and storing it in some cave at the dead sea site...

Securly?

[Anonymous Coward]

Yabbut Linus is sharing information freely. I don't think you would want to publish the location of the family fortune this way.

way offtopic

[Anonymous Coward]

But do you realise that anime only seems good because the rest of TV is so very, very bad? I suggest some hard-ish science fiction stories.

OTOH, the giant robots fighting the demons is pretty cool.

More ontopic: Isn't it great that reality TV is most likely "archived" onto DVD, if even that? At least we're saving ourselves that embarassment in front of our grandkids.

Re:way offtopic

[infonography]

"But do you realize that Anime only seems good because the rest of TV is so very, very bad? I suggest some hard-ish science fiction stories." Why did you AC that post? It's a fair post?

I was thinking some of the old "Pulp" shorts were on the same level. Some were dreck but a lot of clever ideas that should get a wider audience. Anime is a good place, some made it to comix. I think the quality writing is still alive, it's just hiding from the main stream. ( I would too.)

Re:way offtopic

[Anonymous Coward]

OTOH, the giant robots fighting the demons is pretty cool.

Cowboy Bebop has nothing to do with giant robots or demons.

But do you realise that anime only seems good because the rest of TV is so very, very bad?

Granted, but even the best TV sci-fi doesn't approach the best anime (Cowboy Bebop, Witch Hunter Robin, Ghost In The Shell: Stand Alone Complex, etc.)

The Singularity Is Near

[Anonymous Coward]

In 2045, humans as we know them today won't even exist, let alone be rummaging through attics.

Moore's Law + 30 years incubation time ensures this inevitability.

Slashdot == Luddism at its best.

LaTeX

[Compholio]

LaTeX

The damn spec hasn't changed in ages and is designed especial for posterity. If you have a textbook (you know, those expensive things you have to buy for school?) they're all written in LaTeX.

CD Rot will perhaps destroy it long before...

[PocketPick]

Should of thought harder Gramps. CD rot may have taken care of coating on the disc long before the kids get access to it. Optical formats, though much more long lasting than magnetic tape, do not have an infinite life span. Over the course of say, 50 years, it's not feasible to think that all the data on the CD will still be non-corrupted.

Here's an example

my kids will figure it out

[digitalsushi]

I fully expect that in 30 years when I have 18 year old kids living in the basement, that they will have the inherent know-how to reassemble any such thing. I.e. our 50 year old parents look at us and go "How did they figure that out?" I likewise expect to someday look at my children and wonder how they know such a useless trick. :D

Emulation?

[highfreq2]

What popular computing platform (or even rare platform) of the past isn't currently emulated on a standard PC. Obviously, the hardware to read old media is and will continue to be a problem, but software is definitely not a problem.

Much ado about nothing!!!

[ferrellcat]

I have in my possesion a commodore PET computer and a commodore PET 8080 floppy disk drive than I can use TODAY to read media written to in 1979, more than 25 years ago. Hell, if someone had written a file to a disk with instructions to the family fortune, I could find the location with ease!

Now the Commodore 4040 is not a common animal. Production was in the low 1000's, I believe. Compare with with the number of CD-ROM devices made thus far. TENS of millions? HUNDREDS of millions? I'll eat my hat if you couldn't come by one in 2045, or even 2100.

BACKWARD COMPATIBILITY!!!

[Anonymous Coward]

THis article doesn't take into account well-documented open sandards, and backward compatibility with past technologies.

CDs can be played in CD players, DVD players, and Dual Layer DVD recorders. In 2045, if there are newfangled " Disk" players out there, I'm sure that they will play today's CDs. ...unless they're DRMmed.

Anyone remember 8" floppies?

[glowworm]

I believe the equipment to read the media will probably not be around... How many of you have old 5.25" MS-DOS floppies? 8" CPM floppies? Cobol programs on punched cards? Paper tape? CRAM? QIC-20? All very valid media in their day and now long obsoleted.

Don't forget 8" floppies were de rigueure just 20 years ago, so in 35 years time what hope do we have that a CD-ROM or even Blu-ray drive will be able to be used?

LOL - Just had to install a 5.25 drive today.

[Graemee]

I had to find and install a 5.25 HD floppy today for someone to retrieve old contract files from floppies. The easy part was the drive, from the tickle trunk under my desk. The hard part was the cable. To find an old floppy cable was tough. All-in-all it was a sexy looking beast, a Compaq Deskpro SFF 866 with floppy instead of the CDROM.

BTW if you don't know what a tickle trunk is, google Mr Dressup

Oh, I don't know

[tsotha]

How many people are really archiving data these days? Backups, yeah, but archives? I have, on my current hard drive, every bit of data I've touched in the last 15 years. Not because I'm desparate to save old data, but because my hard drive space increases each time I buy a new computer.

So my drive has a directory named "old computer", which in turn has an "old computer directory", which has one as well, etc.

So as long as my grandkids are willing to sort through all the dross (or have a reasonable AI to do it), they'll pretty much have my complete digital life.

That's not an issue, really

[zecg]

Data that gets left behind on obsolete media is data no one thought should be moved to newer media. CDs are not an ideal archival media as it is really not smart to leave anything on it that you would like your grandchildren to find in more than a decade...

In 2045 grandad knew about this stuff called. . .

[kfg]

paper. Use it. Maybe give a copy to the family lawyer who handles grandad's will and shit, also on paper. The stuff isn't going away any time soon. Like, ever.

In 2045 there will still be functional CD drives around, because they are a mass market item. There are still Edison wax cyclinder "record" players around too. Data loss due to format changes are the biggest problem where rare or custom machinery is used, such as at NASA.

There is a huge difference between losing some data and losing all, or nearly all, of it, as in a Dark Age. Yeah, losing the key to the family fortune is a bit tragic for the family, but it already happens all the time without any reference to digital storage, and nobody declares a "Dark Age" over it.

In any case, dark ages aren't even defined by the loss of data, per se. They are the defined by the loss of data because reading and writting itself is lost and/or denigrated. It was not so much the burning of the library at Alexandria that created a dark age, it was the lack of social importance placed on recovering and preserving what had been lost.

People ceased to backup.

At the time backing up was labor intensive and expensive. Now it is quick, easy and cheap. Even, comparitively, for obsolescent/obsolete data storage devices. If the family fortune is really anything substantial it will be recovered because the knowledge of how to recover is maintained and the CD itself still exists.

Nothing has actually been lost, it's just a cost/labor issue to recover.

KFG

Yeah, but so what?

[ottffssent]

The example's contrived. I don't like contrived examples unless they illustrate an important principle, which this one doesn't really do. Such data loss has already started happening even in my own life, but I don't think that's a bad thing. The fairly minimal effort required to keep data up-to-date is a natural impediment to a policy of keeping everything. Data which isn't worth a new hard drive and an rsync dies. Data which isn't worth the effort of importing and re-saving in a newer format dies. This isn't bad. It's not new either.

Data goes the way of the dodo not because of technological obstacles, but because of a decision made or not made to preserve it. We don't know how the great pyramids were built, the obelisks shaped and erected, etc. not because there was no way to preserve that information, but because it wasn't important enough to justify the effort. The same is true of 10-yr-old WP documents I made to bill people when I mowed lawns for spending money, or a million other things that get saved or trashed every day.

If you're serious about the problem, then it's not a technical hurdle. Data storage is cheap. Emulators are good. Batch document conversion is possible. The problem, if you're willing to call it that is that the benefit has to outweigh the cost. Lowering the cost of data preservation only increases the cost of data searching and real information retrieval. And very quickly it becomes a philosophical argument about the value of preserving irrelevant knowledge in a world that has moved on. Yet the argument is couched in terms of data storage and manipulation which is really the tiniest corner of the issue.

Re:Yeah, but so what?

[Anonymous Coward]

Heh. I'm actually kind of glad that obelisks went out of style. I'd really hate to see them be a significant part of, say, your average chrome-and-steel corporate colossus in your local down-town business district... Fucking ancient god-phallus symbols.

(Though I'm sure that there are enough anthropologists on slarshdot who disagree most emphatically.)

Re:Yeah, but so what?

[Yehooti]

Not contrived, as I've felt this pain. In my programming days (Fortran), I wrote many programs that I thought at the time were monumental. I stored them on seven track tape and had a tape of them sent to me for backup. I stored it in my garage along with many selected printouts of the contents. When I remembered something that I thought was choice and could be used for a current application, I attempted to retrieve that code. I took the tape to work and tried to find something that would read it. After searching for a week or so I found that we did have one remaining seven track tape setup left just for this purpose. It couldn't read the blasted thing. Dug into the printouts and found that they had been smeared due to moisture plus faded due to age. The company archives of this data had been deleted after less than 10 years. This is from work done in 1982 or so. It wasn't even 20 years from the origination until total loss. It doesn't get any better it seems.

Re:Yeah, but so what?

[wamatt]

Couldn agree more! The article is stupid and contrived. I can't believe it made it on to the front page. We will always be able to read old media in the future. Right now we can read punch cards and magnetic tapes etc.

You actually made a far more interesting point. I too have debated for years whether to keep all my old emails and arb documents (10 yer old) if only for nostalgic value when I get old. This is a conscious decision (or lack of decision in the case of a backup strategy) on my part and not anything do with a "digital dark age".. gawd.

Re:Yeah, but so what?

[randalx]

You're assuming people can accurately judge what information will be important to somebody in the future. Also, the producer/custodian of the information might not necessarily care about preserving it. But again this does not mean it won't be important at some future date. Lastly, utility != importance. It might not be useful to know how the pyramids were constructed, but it's most definitively an important piece of information.

Re:Yeah, but so what?

[Anonymous Coward]

"A world that has moved on" is an enormous oversimplification. Have you considered that people might some time later develop a new interest in the lost information? In the digital world, information survives only if every single subsequent generation considers it important. If some particular subject fades from popular interest for a single generation, then it will be gone for ever, regardless of how much future generations would have liked to have seen it. At least with stone tablets and papyrus it is possible for the information to be buried and survive by accident, for the benefit of interested future generations.

RE: Speak like a child

[Donniedarkness]

Agh! You beat me to it!

I actually just got theough watching that episode about 20 minutes ago.

Anyways, while the example that TFA is a bit...far-fetched (who the hell would leave their will on a CD-Rom?), this is a really good issue... I don't think we'll ever have to worry about this, though..... at least, not until we no longer have the internet...

I doubt there will be a digital dark age

[Kat0325]

It's difficult to refute that digital storage has a short life-span. But there are a few reasons I don't believe in the digital dark age:

1. The Internet is here to stay.
   These days, the Internet is the new medium for digital storage. The Internet has significantly evolved in the past several years, but the Internet will always remain accessible. 50 years from now, when IPv256 is in common use, I'm willing to bet that I can still access documents living on some IPv4 server assuming it's still connected to the Internet, because whatever future technologies are used on the Internet, I'm sure web browsers / networking technology will employ backwards-compatible technology that will allow me to access to servers running older technology.
   
   
2. The most treasured knowledge will always be migrated to newer technologies.
   Let's take some terrific story as an example, to which society assigns great value. It starts off as being passed by storytellers, the most primitive form of knowledge management. Then the stone and chisel comes along, and people are like, hey! this story is great. We need to preserve it by carving it onto a stone. Then paper comes along, and people think, stones suck, it's not portable, let's put it on paper. Then the digital revolution comes along, so people decide, digital is great! So of course, people put the story into digital form to preserve it and make it more accessible. Then Internet comes along, and without a thought, it's put on to the Internet. As new technologies emerge, people will always make sure that our most important knowledge in preserved in the most common technologies.
   
   
3. If there is a will, there is a way.
   Directly addressing the question posed, some kids 40 years from now find a CD, and have no idea how to use it. Let's put this concept into today's terms. Let's say I go into an attic and find a 5.25" floppy. (I haven't seen one of those in over a decade.) Now the 5.25" floppy drive is an uncommon thing these days, as is finding some old version software to read it. But I'm sure SOMEONE out there has an old 286 or something that still works -- and with the Internet around these days, it won't take me long to find it. (12 seconds to be exact: typed 80286 into eBay and found a ton of mint-condition 286s.) If the supposed bazilli-jilli-tillion dollar family fortune was on that 5.25" floppy, I would definitely make the effort to stick that floppy into a mint-condition 286 purchased on eBay to find out the details.

Re:I doubt there will be a digital dark age

[Kalvos]

Okay, I'm glad you took the time for this reply so late in the ever-decreasing slashdot interest curve...

#1: The Internet is here to stay.

Remember the Dark Ages. The real Dark Ages, and the reason for them. It was a political choice based in religion and power. Future technologies, for some period of time, simply may not exist. We may indeed have a kind of Dark Ages that will remove this option. As the Internet is politically segregated today, it can be further segregated, then destroyed. These scenarios have been proposed, and they are not impossible, just unlikely.

#2: The most treasured knowledge will always be migrated to newer technologies.

Here's a problem -- knowing what that 'treasured knowledge' may be. If Bach or Mahler lived today, for example, most of their works will have been permanently lost because their work was not treasured as they grew old when they did live. We tend to treasure what's personal or what's popular, but rarely have the cultural vision to understand what is ultimately culturally lasting. Imprisoning cultural vision in technologies that quickly fade is a mistake. Sure, I may migrate my material (I believe I'm a very good composer), but who will do the next generation of work?

#3: If there is a will, there is a way.

This is a kind of American myth, like the unending frontier. It's very positive, but ultimately it fails. The forces of nature (including those involved in the deterioration of technology) need only be applied intermittently to break the chain of possibility. And that does not even include the question of cost, which every archivist faces.

These three points you make can be multiplied: there must be a technology, there must be cultural vision, and their must be will and funding. What in the world qualifies for all three, even now?

I covered this topic three years ago, by the way: http://maltedmedia.com/books/papers/sl-archv.html

Dennis

Um..

[blake213]

If he left a CD AND a letter, why didn't he just leave the directions to the family fortune on the letter? You disappoint me, Jeff Rothenberg.

extra durable DVD-/+R?

[E8086]

What about one of those "super" DVDs, the ones claimed to be extra scratch resistant and needing nothing less than a hammer to break? If stored in a cardboard box(away from light) inside a tupperware with a decent air seal it should last many years. Since most encryption from present day will probably be able to be broken in about 5min 40yrs from now you're better off storing it in plain text.
My plans for keeping data around for the next 50yrs is to make new copies on the newer medium. I still have some files from 5.25" floppies from the late 80s. They're now on DVD and when the DVDs replacement is eventually made they'll be copied to that.

Touchable media in '45??

[Hurricane78]

Lol. I'm pretty sure we will have internet everywhere by then. And why should wikipedia dissapear because of bad removable media??

Old problem

[phocuz]

It's not something new. There are lots of projects around the world trying to solve it. As with all issues of preserving knowledge, you'll find that even the simplest things can turn to great pains. For instance - you have your great hole in the mountain for storage of nuclear waste. Now, you don't want people in the future walking down there. Assuming they dont speak your language, what do you do? Paint something, like pic 1: human outside, pic 2: human going in, pic 3: human dead. Sounds good eh? Try reading it backwards, as some cultures do. Human dead, goes inside, comes out alive? Not really the same thing, but it's an interesting problem. What will we leave behind? Not many things are written in rock any more.

All this makes me think of Stanislav Lem's "Memoires found in a bathtub", which is a grat book for those of you who haven't read it.

We'll be OK.

[WoTG]

Remember, back in the ancient times there weren't that many copies of anything printed (printing presses didn't exist). Today, we've got so much more data that we're saving, that if even a small percentage survives 100 years, future archeologists will have more raw data to work with than they'll know what to do with it.

However, to help those in the future, maybe a repository of emulators, hardware designs, RFC's, and source code should be created. Perhaps we could make a fresh collection of relevant data files every 10 years and put several copies in cold storage around the world.

unwarranted paranoia

[lysergic.acid]

unless there's a nuclear holocaust or some other kind of global disaster that destroys all records of the past, it's unlikely that these scenerios of pandemic technological amnesia could occur. new technology is often built with foundations in older technology, so just because new digital mediums or data formats for information storage are developed doesn't mean that old media formats will simply be lost forever.

and the way that technology develops and progresses is always with increasing complexity. so even if we lose all the documentation and records on an old media format, it shouldn't be too difficult to derive the architecture/specs for older media formats from modern digital mediums, which are likely built upon those older and more simplistic architectures. it's kinda like when you're in advanced calculus and one day someone asks you to help them with a basic algebra problem. even though it's probably been a while since you've solved that sort of problem, because you've moved onto more advanced maths, it shouldn't be difficult for you to recall how to solve those types of relatively simple math because you had to learn those basic concepts in order to get to the higher up maths.

aside from that, people tend to migrate vital/critical information from older storage mediums to newer ones as they are developed. look at the music industry for example. with the advent of CD's, people didn't just toss out all music that was recorded during the era of vinyls, 8-tracks, or tape-cassettes. as long as human society maintains this level of common sense, this digital Dark Age scenario is unlikely to happen.

Not the only possible future.

[sbaker]

I don't archive stuff on CD-ROM.

Hard drives are the only things big enough to archive my 'stuff' - so everytime I upgrade my system, I copy all of my files onto the new drive (which is probably 5x larger than the last one - so I have plenty of space and no need to delete things).

So - I still have files that were on a TRS-80 cassette tape - then on an 8" floppy - that I copied onto 5" and then onto 3" and then onto a 200Mb hard drive - then 1Gb, 10Gb, 200Gb, 400Gb hard drives. The idea of keeping stuff on a large pile of CD-ROMs or even DVD-ROMs is ludicrous. The only way to archive my stuff is by putting it onto another hard drive.

With the rise of networked storage, the physical media becomes less relevent at time goes on. I can get files back from 'The Wayback Machine' in the state they were in YEARS ago - provided they were on my Web site.

Why wouldn't those Internet archives continue to be maintained in purpetuity?

The backward reach of Internet storage increases daily with people like Google scanning ancient paper media in order that they can be searched and perhaps one day viewed fully online. Blogs, Gmail archives, Forum systems mean that even our personal communications are saved for our descendants. The only personal mail my late Father left was a couple of letters to my mother before they were married that she kept as memento's. My grandkids should be able to find and read this very post 100 years from now.

How can anyone say that the trend is for data destruction? We're hoarding ancient (and mostly useless) data by the petabyte.

Walmart are reputed to have computer records of every credit card sale they ever made...archeologists will find those most interesting. Imagine if we knew the exact and detailed buying habit of every citizen of ancient Greece? Find Aristotles favorite brand of soap powder?

The larger problem (and it's potentially VERY serious) is the rise of closed file formats, encryption and DRM. There is a real possibility of designing systems that our ancestors may yet be unable to crack. When Moores' Law hits the endstops, our encryption may be truly impossible to break - even theoretically.

THAT is the thing that'll frustrate the efforts of future data-archeologists.

Is there a point where our privacy needs extend past our own lives? How about a thousand years from now?

Re:Not the only possible future.

[paradizelost]

There is a real possibility of designing systems that our ancestors may yet be unable to crack.

I hope you mean our descendants. as our ancestors won't be cracking anything, other than maybe their skin or someting....

Re:Not the only possible future.

[sbaker]

Oops! Thanks for the correction.

2D barcodes

[TheSHAD0W]

You folks might consider 2D barcoding for long-term document storage. It's not AMAZINGLY compact, but is a lot better at holding information than text, is potentially lossless, and given the availability of a page scanner or even a high-res digicam, means that everything else is a software problem - which isn't completely trivial, but is better than having to build custom hardware.

Old news with Analog

[Ironsides]

This is already happening with analog recordings. The Piano Paper that you put into a piano and the piano plays the music. The old drums that were originally used to record sound. Records (45s, 33s, 78s), 8 Track Tape, Reel-to-Reel, dictaphone, Cassettes (becoming this way).

Want picture/video? My father has some negatives that are 3 inches by 5 inches. Back before the days of 35mm film. Then there are those old home movies that predate VHS.

The only difference between that and digital is that digital is newer.

Re:Old news with Analog

[Apotsy]

To play back an analog motion picture, you need a lamp, a shutter, a motor (or handcrank), and a lens (optional with a pinhole aperature). All of this can be made from scratch fairly easily. Even if a motion picture were found thousands of years from now (which with certain types of film stock is actually possible), it wouldn't be that hard to figure out, even with no prior knowledge.

To read a data DVD, you need electronics that take billions of dollars worth of factories to produce, a considerable amount of software to go with it, and lots of prior knowledge.

There you go ... another difference.

Re:Old news with Analog

[Ironsides]

To play back an analog motion picture, you need a lamp, a shutter, a motor (or handcrank), and a lens (optional with a pinhole aperature). All of this can be made from scratch fairly easily. Even if a motion picture were found thousands of years from now (which with certain types of film stock is actually possible), it wouldn't be that hard to figure out, even with no prior knowledge.

You forgot about the optically encoded stereo soundtrack that is also on the film. This also requires prior knowlege, among other things.

This is infantile.

[concurrent.ca]

I still have the files I had on my MS-DOS computer. Even the ones that were on 5 1/4 diskette. If there were anything tricky about moving bits around, (like when you had to use a floppy) this would be a problem, but with the expansion of portable media, and networking, my bits roam from computer to computer, OS to OS, media to media with the greatest of ease.

We are in greater danger of people forgetting how to read.

Dark or Bright?

[Anonymous Coward]

If it so happens, I call it bright age or I prefer to call it "Fresh Start"

If learning from past is ture, we are all great saints and scientests by now.

the other side

[Eric604]

a letter, which explains that the disk contains a document that provides directions to obtaining the family fortune.

Write the directions on the other side of the letter.

Besides, this scenario won't easily happen with important information because every sane person living in this fast changing world knows that floppies, cdroms, videos.. etc might not be around in 40 years.

If Grandpa was smart he'd leave a PC too.

[Anonymous Coward]

Couldn't Grandpa leave a PC that would read the CD? Maybe even a 110V power supply hooked up to an excercise bike? "Pedal fast children, the information about the treasure is only 35 mph away!"

No problem...

[Davorama]

and, even if they found a suitable disk drive, how will they run the software necessary to interpret the information on the disk?

If you saw the first NerdTV interview with Andy Hertzfeld you'd have heard his story of having to write a custom piece of software so that he could read an ancient Mac disk with the source code for MacDraw on it. Fun stuff, but I doubt my grandkids would be that resourceful.

Another reason for the information dark age

[slashname3]

Another reason for the informatin dark age that we are approaching is the insane patent and copyright laws that are being passed. Fairly soon most ideas and concepts will be controlled by a few mega corporations. At that point innovation will become extinct. Anyone trying to produce something new that infringes on one of these patents or copyrights gets sued out of existence. As a result no new technologies get released except for those the mega corporations want released. And since the mega corporations are held to a three month cycle by the stock holders they only push out the same old thing that people buy everyday. No sense in taking a chance on spending R&D money to try and develop something new that might not sell and impact the bottom line for the quarter. Thus resulting in a dark age where no real progress is made, just stagnation. As a result no one is able to adapt as the earth warms up then cools down into a new ice age. This causes massive evacuations of coastal cities as they become unlivable due to increasing frequency of storms and rising water.

Oh, wait, I just watched this on the nightly news.......OH SHIT!

Why's Gramps burying his fortune in the backyard?

[nunchux]

Seriously, is this the best example they could come up with? Grampa leaving a treasure map on a CD Rom in the attic? Is he a pirate, or just insane? Will this ever happen to anybody?

How about a circumstance someone might actually deal with... Perhaps you're audited, or selling a small business, or a former employee sues you for funds owed years ago, and you need to access files from a tax or database program that's no longer made. Or you need to access old letters or contracts written in a word processing program no longer made (happened once to me, I had to access some info written with "WriteNow" on a Mac in the 80's.) I can tell you from experience that reading old television (and movie) scripts are a big problem, since the big two programs-- Final Draft and Movie Magic-- don't play nice with each other, and often can't even open files created by a different version. Non-standard software is the problem... And one solution is to backup everything in multiple formats, especially PDF. Just about everything can be saved as or printed to PDF now. That standard will eventually be replaced, but considering how much government and business has invested in storing just about everything in the format the ability to read it won't go away in our lifetimes. I would bet there will be also be jpeg and mp3 readers for a long time to come.

The article also omits emulation-- which is how I solved the WriteNow problem. Is there an outdated computer or OS that hasn't been emulated yet? I can't see this going away, either.

As for hardware-- say, reading old disk formats-- if the data is still there, and if the device was mass produced, it can be done. Maybe not cheap, but it can be done. I guarantee some packrat somewhere will still have a machine... There are still working kinetoscopes out there, for God's sake. The problem isn't that CD, tape or disk readers won't exist-- the problem is that the mediums themselves (especially CDs) are a crapshoot. It's really only common sense that important info should be backup up a few places-- on a hard drive, to a server, and to CD/DVD/whatever's next.

CD is as good as a universal format at the moment

[PhunkySchtuff]

CDs have been around for some 20 years now. The basic CD is still not obsolete - half a gig of information is still a useful amount. With all the higher capacity optical disc formats being proposed and/or introduced - none of them break backwards compatibility. You can still buy music on LP and Compact Cassette today - I'm willing to put money on you being able to purchase at least music on CD in 20 + years time - failing that, it is such a ubiquitous storage medium today that there should be backwards compatible drives to read it for the foreseeable future.

Well...

[Mechcozmo]

First thought was, "Hey, DRM will do this!" But then I RTFA and thought, "Well, that's pretty stupid considering computers have been storing data for at least 30 years now and this is already happening kinda."

DRM will do this quite a bit more. "Kids, the family fortune would have been yours but we can't read grandpa's format... and according to Wikipedia, Microsoft was a huge company that is gone now but nobody but them could read their software."

Business opportunity!!!

[peter1]

I smell a potential business here! You can startup a business where you keep old systems and software around to read old data, transfer it to a modern computer and burn it out to the portable medium of the day...

Wait, everyone ignore that - I don't want the competition...

Seriously, even today if you have an old record in some weird format you can always find a company that can transfer the data off somehow and put it onto a newer format so you can use it. In fact, wasn't there an article out here a year or so ago about a couple of university kids that were working on system by which they could put an old LP onto a flatbed scanner, scan an image of it, and then feed that image into some software that could "read" the grooves and turn it into WAV files? Basically this is going to be a viable business for the future, where sucess is measured by how old your equipment is...

photograph it and write a program

[harlows_monkeys]

Well, by 2045, it should be easy. First, they will take a photo of the CD with the digital camera in their iPod/PDA/cellphone/tricorder (we'll finally have device convergence by then). The digital camera will have trillions of pixels--enough to resolve the pits and lands on the CD.

Then they'll do a search on Yahoogle to find out what the format of a CD is, and write a program to scan that photo and reconstruct the data.

Then the RIAA will sue them.

Re:photograph it and write a program

[CrosseyedPainless]

Attention moderators: +5, Funny-'cause-it's-true

Float to the top

[spentmiles]

First, let's eliminate the few cases where grandpa has buried the family fortune on an encrypted USB drive. USB won't be around anyway, and the grandkids will have to continue to hovel around in the nuclear haze, their tails brushing aside old Kids in the Hall VHS tapes.

Let's get to the more likely possibility. Let's just assume that's grandpa's fortune isn't cash-money-yo! or the original season of MST3k, the Crow Tapes. Let's assume that grandpa pulled his head out of his ass just long enough to write the key to happiness. The map to this euphoria is expressed in exquiste detail, though its embedded in the bits of an old episode of Degrassi Junior High, hidden on the recently unearthed CD. Chances are, the kids will try to eat it. They have, of course, been scrounging the Wastelands looking for the Oracle, and it's only by chance that they stumbled onto Grandpa's stash... They will, of course, have to wade through his three terabytes of child pornography (he found a way to download the family's intranet). But once they've shuddered at their youthful exploits, they will certainly be starved nearly to death. Finding the CD inedible, they will have no choice but to take it to the leader of the tribe, Bill.

Bill will insert the CD into his rectum/DVD burner, and using his telekinetic powers, will burn the contents onto the newfangled media of the day, Puffs Plus nasal tissues. The kids will each blow their snouts, and then Bill as a sign of thanks, and then ramble off with the information stored on their platter minds. They will follow the map exactly, only to find the key to happiness. They'll wonder at the notion of a "cubical" and "work" and "fat free preztel" and "coffee". They'll be sorrowfully disappointed, though, when they try to point their browers at the prophetic URL, http://slashdot.org./

They'll be thankful, though, that they had the adventure.


Perhaps it's better to encrpt nostalgia... Life stopped getting better when they shut down Inn of the Last Home.

Very true

[gilesjuk]

Indeed, since not only will they have to find a method to read the data, figure out the encoding etc, they will also have to decrypt it.

Here you can read about the problems encountered without DRM in recovering data in weird and old formats:

Firstly how a modern day Doomsday book project became unreadable:
http://observer.guardian.co.uk/uk_news/story/0,690 3,661093,00.html

Project who managed to salvage it:
http://en.wikipedia.org/wiki/CAMiLEON

Wikipedia info:
http://en.wikipedia.org/wiki/BBC_Domesday_Project

Don't Worry - Don't Leave ANYTHING

[NotQuiteReal]

When I die, my will states that all assets are to be liquidated and converted to cheap Mexican Fireworks.

Then, they are to be ignited, en mass, along with any surviving pets, in a collosal bonfire in an environmentally sensitive area.

Everyone knows that everything important is always written in stone.

Hah! What Family Fortune?

[Anonymous Coward]

Spoken like a true bullshitter!

Before considering a problem, determine whether the problem truly exists. If there were a "family fortune" you could afford to leave evidence in some universally-readable format like, say, plain text in a will.

analog to the rescue

[hobo sapiens]

I found this interesting article on the aptly named Rosetta Project. It's nothing too new, but it shows that there are ways to preserve data. New tech isn't always the answer. Those Babylonians were on to something...

P.S. CDs are optical storage

[kfg]

At sufficient magnification you can actually see the raw, digital data. That means that if need be you can actually work out the the meaning of the raw data with pencil and paper if you had to, assuming you had the spec of the code.

Use published open standards. That is the important part with regards to maintaining knowledge. The actual meaning of symbols, not how they are recorded. See the Rosetta Stone.

Of course what you'd really do is photograph/scan the CD and use a computer to recover and interpret the data from the visual image, then save to whatever modern formant currently holds sway. If the issue becomes so pressing and widespread as to the threaten a "Dark Age" I'll simply set up a kiosk at the mall to recover your CDs for you, for a nominal fee, of course.

I suppose I should run out and file a patent on the method right now, eh? So, who's got venture capital left? It actually seems one of the more logical fields for Google to invest their pile of cash on.

KFG

Anyone remember the BBC Domesday Project?

[mwooldri]

It was a wonderful piece of work - with great detail. It came on two Laserdiscs, which ran on a special reader linked up to a BBC Microcomputer. There aren't too many laserdisc players let alone BBC Microcomputers around anymore. As such, all the information that was available on two 12" laserdiscs was in danger of becoming unreadable and unaccessible. Fortunately people who remember about the 15 year old project and care for it have preserved the information and whilst it hasn't been converted to DVD format for people to play in their PC's... there are people who know how to access the information.

As other people have said here, if the data is important it will be preserved.

Re:Anyone remember the BBC Domesday Project?

[craXORjack]

As other people have said here, if the data is important it will be preserved.

You are not answering the question posed in which the information was purposely kept secret to protect the family fortune. And even if we were talking about the preservation of widely distributed information in general, you have not considered the possibility of an event which could disrupt civilization globally such as a large asteroid impact, a supervolocano eruption, runaway greenhouse effect, nuclear winter, etc.

How to archive a digital document

[AB3A]

The thing about digital information is that you can continue copying it as many times as you like without degradation. This is a unique thing about digital data. It's not about the media. No, there won't be a digital dark age. Anything important will be archived in multiple places with regular backups.

I know, because I have had this very problem at work. At the water utility where I work, we have to archive years and years of data from our SCADA system. Right now, we have the last 18 years of data at our fingertips. It's stored in a lowest common denominator form: Text files of CSV data on CD. In addition, we keep the CD data online (hard drive storage is cheap now). Keep in mind what the technology looked like back then...

The file formats will change eventually, but we are always seeking the lowest common denominator. We move the data to new media when it becomes feasible.

As for those older documents... Who keeps memos forever? How many paper documents have been lost to the mists of time? We aren't writing literature here. This is common communication. I'll bet historians of the future will be drowning in daily trivia from our time.

Those CD media may rot after a decade or two. But who cares? If it is important, there will be copies online somewhere. And some important things will be lost. It's sad, but it happens. Even libraries burn sometimes. Shall we start trying to develop fire-proof paper?

It's not about the media, folks; it's about using well known file formats and placing the files in a reasonable, well kept and well known server.

old story

[SomebodyOutThere]

This scenario is an almost exact ripoff of a Jan 1995 *Scientific American* article [sciam.com]

We're talking 2040, people!

[ConceptJunkie]

2040! Twenty-frickin-forty! Can you even imagine how far technology will have advanced? Can anyone even imagine the capabilities we'll have to decode old media, assuming that we could possibly forget how to use it? I've got floppy disks that are pushing 25 years old that I can still read if I hook up my old 5-1/4" drive. Sure a couple are unreadable, but most of them are fine (at least as of about 1997 when I archived them all). We're only talking 35 years from now... and important data will probably will periodically upgraded to new media as it becomes available. I migrated from floppies to ZIP disks to CD-R to DVD-R, although I still have the capability to read all those formats, if I needed.

Assuming you come across an optical disc format you don't know how to read, just throw the CD into an electron microscope and feed the picture into your computer and let the A.I. work on it for a few hours. Presto! Instant data. I'm sure data recovery services will be cheap, plentiful and highly capable for all kinds of obsolete formats.

By that time, you'll probably have services where you can stick a box of old papers or an old book into some kind of high-resolution MRI or PET device and have the computer reconstruct all the contents without having the risk of destroying them by taking out the contents.

Truth and Memory Need Freedom.

[twitter]

..patents and DRM will lead to a digital dark age.

Digital technology makes it far easier for me to record, store, copy, transmit and share information. My family has been in this country for more than 250 years. What's left of that is stories, a small library, furniture, a few scrapbooks and the odd film. In all that time it was much more difficult to record thoughts. Paper, the cheapest and easiest, is also a bulky fire hazard that's difficult to search trough. Do you really have room for your own correspondence or the time to sort out what's "important"? How about someone else's? Things get lost because you can't tell what's and old bill and what's a love letter. Now compare that to email in a reasonable format. I've gone from file cabinets to one or two hard drives. I've got every paper and email I've written since 1989, I'm digitizing old photographs, sound recordings, you name it. When Katrina came, I ran with the hard drive. It's much easier than loading up the horse drawn cart. The same thing can be said for businesses, churches and local government. Digital media is a panacea if it's not done is some stupid, commercial way.

Now enter "trusted computing" and DRM in the most distopian way. Boom, some one else thinks they own my computer. Even if the media is preserved in formats me and my friends do know how to work, we can't. Instead of being able to run free software that does everything I want, I have to pay some turd to store, search and share my files. The machine itself won't let me do it and it's owners know just how much they can hit me up for before they erase everything anyway. In this world, I can't afford it and I'm left with a subset more pathetic than the one I've been handed.

How can I avoid this? I can't, without decent laws preventing the nighmare. Judging from my first, still working computer, I can keep DRM free computers going for 20 year or so. But that's the dead end discussed in the story. Only by standing up for our digital rights and avoiding bad hardware will we prevent the next dark age.

The last dark age reduced the entire Greek and Roman library down to a few hundred volumes. You can fit it on one good sized shelf or a few hundred megabytes of text. Archaeologists are currently digging through garbage in Egypt to get more. That is what paper media will do for you. A hundred or so churches, museums and universities hold the physical remains. The remains of our own culture, as described above, are more numerous but infinitely poor next to the digital records free people can pass on.

PPM, rather.

[Grendel Drago]

I think you mean PNM. XPM is kinda limited-use.

Old Problem, New Twist

[Blind_Io_42]

The fact is that newer data storage techniques are more resistant to the ravages of time than many older mediums. Take for example the early movie industry: films were shot onto early concoctions of celluloid and emultions which quickly started to decay. Chemical reactions started to take place in archived film cannisters stored in the Hollywood vaults, so many that there are documented cases of spontanious combustion of stored footage. There are an unknown number of films that were lost when the fire department destroyed them because they had become fire hazards. Paper, even with the most tender care will yellow and turn to dust or rot (depending on the environment) and even my beloved film will turn into a sticky mass with enough time.

In this way, digital technology has a distinct advantage over traditional media. Digital information can be duplicated countless times without any degredation to the data. As long as we continue to archive our data on current forms of storage (and make redundant backups) there is no reason to ever loose data. As for the story in the article, there will always be people with old machines. If nothing else the Smithsonian would be one place to look. There is also a private technology museum I read about some time ago that maintains obsolete machines in working condition. An enterprizing person with enough motivation could find a machine to read the data somewhere.

Re:Old Problem, New Twist

[The Master Control P]

Just nitpicking about early films:

The first film-based negatives used a nitrate as the base of the film. As this base is chemically 1 step away from guncotton, it's extremely unstable and can indeed spontaneously combust (more like explode) at room temperature. The next step was film with a base made out of an acetate, which is still used. When this degrades, the film turns into vinegar.

The only way to make a photographic negative last forever is to either cool the plastic to liquid-nitrogen temperatures, which will completely halt the degredation or make the base out of something else. Metal is good (daguerreotypes, tintypes, ambrotypes), as is glass (Lumiere plate).

Personally, I feel that the greatest problem to long-term storage and retrieval of data is changing and dying languages. The Japanese could never decrypt Navajo during WWII because they'd never heard it. If it weren't for the Rosetta stone, we would have never figured out ancient Greek. What will happen when we write something down, and in the distant future there are no English speakers left and nothing written in multiple languages?

Easy answer

[Maljin Jolt]

How can they read my obsolete digital document?

Keep your obsolete computer as an antiquity. I do that for 20 years now.

A great joke to play on the grandkids

[Anonymous Coward]

As I age, I come to hate people younger than me more and more.

By the time I get old enough to have grandkids, I have no doubt this will have become an obsessive, burning hatred, which I will deal with by making their lives a living hell, combined with threats to disinherit them if they tick me off.

Then, I can croak and leave them all nothing!

Then, they will frantically search through my left-behind possessions, where they will find a cryptic letter and a CDROM.

After months of searching, at great expense, to find equipment to decipher it, they will either be unable to do so, or they will do so, and find that it contains a letter instructing them to send money to an account in Nigeria....

Bwahahahahaha!

if you expect to have to reverse engineer it

[toby]

bmp would be preferable to jpg

Only if you expect to be in the situation of having no software to read JPG, and no specification. That's a slightly extreme scenario? Since your data has been, obviously, carried forward. You could always carry forward source code or specifications too, along with your JPG corpus. Or am I missing something?

Re:if you expect to have to reverse engineer it

[Kadin2048]

This is a pretty good point.

If you're thinking that your data will be carried forward electronically, then there's no reason why a set of specifications or source code in a commonly-understood language (I can't imagine that any reasonable programmer of the future wouldn't be able to at least puzzle out some well-commented Pascal or C) showing how to decode your data. However, you'd have to hope that whoever is 'carrying forward' your data isn't lazy or cheap, because this would be the kind of thing that would get cut off or removed for being redundant if you started running short on storage space. It would be unfortunate, but not hard to believe, if some space-consious bureaucrat somewhere were to keep only the data and not any of the sample decoders, especially if it was in a repository with a lot of other people's data, and each one had its own set of example decoders. I guess the decision of whether to trust the archivists in question would vary with each situation.

In the case of physical media, I think it's important to bundle each set of data-containing artifacts with either an actual reader device which produces some kind of easily understood output, or schematics for same. For instance if you were going to bury a vault of CD-Rom type discs, it would make sense to put at least a CD-Rom drive in (actually in a vault, you might as well put a whole computer in too). In something smaller, at least include a decoder schematic, or at least some kind of minimal diagram. I'm thinking the bare minimum would be something like what NASA put on the top side of the Voyager probes' "Golden Records."

Actually if you want a good example of a data-storage 'time capsule,' I think the Golden Records pretty much are the 'best practice.' Engraved into metal, built to last practically forever, each one comes with a cartridge and some minimal schematic instructions. If I'm not mistaken too, the very beginning of the recording is a triangle-wave test tone, which is represented on the front, so you know you've gotten it right. I've also read that they included analog slow-scan TV images on there too, although how they expected people to puzzle those out I don't know.

Re:if you expect to have to reverse engineer it

[drownie]

And don't even think about it if this is in a post-technological or post-apocalyptic scenario. That's when you want hardcopy! Old-fashioned printouts and photographs... with all their attendant preservation headaches. That should be in the bunker too. The basic data that defines our civilization along with a lot of technical data is in fact stored on microfilm, in metal containers in a salt mine. The german Bundesarchiv does this, the Swiss archive does it and I think there are some other countries with these kind of national archives.I think the library of congress has some of these kind of storage capacities. It's absolutely safe, there has never before been a safer storage for information. The microfilms are produced to last some thousand years. A microfilm reader is essentially some kind oflense with a light and the salt mine was here for the last 50.000.000 years. It will survive a nuclear war, an asteroid impact, it will probably survive human civilisation. I wouldn't be too worried.

video formats...

[Kenshin]

Video IS a tricky thing.

Back when I worked at a multimedia company, in '98, I had to encode stuff in "Vivo" format for the web.

When was the last time you heard of THAT? Know anything that can play it? Probably not.

Re:video formats...

[paedobear]

Actually, Real have a player available for download on their site.

Re:video formats...

[Kadin2048]

Frankly I think for video, your best bet for archival purposes is some sort of analog format if you want to store physical artifacts. Actually, if you can get film stock that's archival and not unstable like the old celluloid is, I'd use film. I'm not sure if they still make K-14 process movie film, but if they do that's the stuff you want. Not E-6, it's dye based and fades. Otherwise get some silver-halide black and white. I know that's still available.

Put all your video on film, with the audio in the visible-light modulated stripes on the side, and pack that up in your time capsule with a few projectors and a bunch of spare bulbs, and I think even 1000 years from now -- provided the film base doesn't degrade, which is a big IF -- the people who find it ought to be able to figure out how to play it back.

I suppose that solution might not appeal to geeks, but if I was really going to build a time capsule for the future, I think that's the sort of solution I'd be looking at. Simple, reliable, obvious (to comprehend on the far side).

I'm Glad

[lord_sarpedon]

Personally I don't want my grandchildren or archaeologists 1,000 years from now being able to read my CDs or hard drives. The automated destruction of outdated software and useless documents isn't really so bad...but the catch here is they REALLY don't need to find my pr0n collection. "Mommy, why didn't women wear clothes 100 years ago?" HOWEVER: It would be a service to humanity to preserve Wikipedia and the good digital dictionaries and translators every so many years in the most reliable and long-lived storage medium of the time, including reference documentation in multiple languages about how to read the data. Think of what we would have to include: HTML, CSS, ASCII, Unicode, the concept of bits and bytes... If at all possible, a bootable reader machine would be nice. Solid state of course.

I totally agree

[obarthelemy]

1- burned CDs and DVDs do NOT have a very long lifespan (some company in germany guarantees theirs for 100 years, but I'd guess most are chemically stable 10-20 years, and subject to scratching)

2- physical formats evolve. I recently came across 5"1/4 disks with all my grad. student stuff on them (that was wayyyy back in the 90s)... Assuming the data is still readable, it won't be easy to get the right drive. I could say that I don't care, but for my 30th BDay my sister exhumated my first writing exercises from school... It DID bring tears to my eyes.

3- logical formats evolve. MP3, JPEG and .DOC are prolly as time-safe as WordStar and 123 were back then (I'm pretty sure I can still import that into Office), but I doubt that 50 years on, those formats will still be popular. A case in point is mailstores, whose format I really don't trust over the long term.

So, same as you, I keep ALL my documents on a HD, mirror it every week to another HD, and burn DVDs every month or so (which I then store at friends') in case of a major disaster (fire...).

That doesn't work for "media" files (films, MP3s...), that take up too much space, but just for my office files, mail (and any other documents I create myself), and photos.

So, there will be no equivalent to grandma's old musty books, letters, and notebooks for our grandchildren (or toys for that matter, I doubt a gameboy will last 50 years). That's a BIG change.

On the other hand, will be able to IM with them, phone them, teleconf them..., which beats a monthly visit. Maybe we'll still be able to watch Star Wars and Princess Bride together ?

Re:I totally agree

[Kadin2048]

I was thinking of how you could store data that would really stand up to the test of time. History provides us some examples: things cut into stone seem to do pretty well. Paper isn't bad, providing you store it well. Animal skins, not so good. Celluloid isn't either (evidenced by the old movies and cartoons that are degrading).

However glass is really good, and while it might not have the proven track record that stone tablets to, it can also support a much higher data density. For example, Ansel Adams original glass plate negatives are in some cases just as sharp as the day they were shot, and they should stay like that for the foreseeable future providing they're well taken care of. But even they are dependent on the chemicals used in processing -- whether the silver sticks to the glass over time, etc.

So here's what I was thinking: what if you used some sort of photographic process to physically etch a pattern of bits into glass: use a fairly strong acid and get the etching pretty deep, or maybe etch the bits at the bottom of phonograph-like grooves so that light surface touching wouldn't destroy them. If you could make something like this that could be read with a regular CD Rom, that would be even better.

I think some sort of process like this is used on metal (or is it actually glass?) to make the dies for stamping CDs. Basically I'm suggesting just make and retain the masters, but don't degrade them by stamping anything.

Answer - document custodian daemons.

[jlseagull]

Idea #1
What about a semi-intelligent expert system daemon that, given two document formats, could figure out how to convert one to the other?

Consider this: I would like to archive a set of CAD documents, but they're in archaic format X. Modern CAD formats are A, B, and C. CAD programs typically have ancestors that can convert from past versions for migration purposes.

So consider an interlinked set of CAD converters:
#1 can convert formats F, G, H to formats D and E.
#2 can convert formats W, Y, X, and Z to formats I, J, K, L, and F.
#3 can convert formats D and E to formats A, B, and C.

Consider then a daemon that continuously monitors a filesystem looking for documents that aren't in a current format. It then fires up the converters and performs the conversion while archiving all past versions.

So in the example, the daemon fires up converters 2, then 1, and finally 3.

It could also cryptographically sign the files to provide a chain-of-custody.

It also maintains a set of applications and an emulator for different operating systems. When one needs to open an archaic dataset, one can either look at the converted files or call the daemon directly to seamlessly pass an emulated application session to the user if you want to look at it in the original form.

Idea #2
Documents could contain their own viewers. Yes, I know that's a bad idea making document objects executables, but hear me out. The document custodian daemon could also maintain a sandbox for document viewers to run in - it could even be a standardized virtual machine written in something like Java. This is getting a little out of my area of expertise, but I'll ask my girlfriend about it. It would get interesting after several levels of emulated virtual machines.

This year, hard drives became cheaper than tape for the first time in terms of $/GB. RAID with NFS should be way better than tape backup in terms of retention and nearline access, but I'm not really an IT guy.

I'm sure there's a business model in there somewhere.

No problem

[HungSoLow]

The way I see it is, in 2045, they'll have computers powerful enough to look at any binary data and accurately assess what the data represents, and how to extract any useful information from it.

Easier

[Sycraft-fu]

Copy it to a new format. That is the real beauty of digital. Since it can be perfectly duplicated easily and quickly it's no problem to move it to a newer format. I have data on my drives now that was orignally on 5.25" floppy. It has just been recopied many times. Some of it has been converted to new formats, some of it is unmodified. Either way, it's still here despite being decades old.

I don't know where this silly idea comes from that somehow digital is really fragile and we'll just lose all of it later. Sure, we lose tons of it all the time, but it's worthless, by and large. The by product of the information age is that we produce so much of it, it is not only impossible to archive all of it, it's undesirable. To have more information than you could ever sift through would be almost as bad as having none at all.

Also what's the this stupid notion that we'll forget how to read things? That's like saying that we'll forget how to build sailing ships, now that we have motors. Of course that's not the case, the knowledge is preserved, in the case of sail boats, they are still made.

This is even more clear for computers since emulation is a major protect for many people. We have emulators for all kinds of old systems. Means if you find data for one of them, you just load up said emulator and it'll get at it.

Digital actually seems to be the ultimate prevention against a dark age. The ease of copying information and archiving it in multiple spots means that it's difficult for a single catastrophe to wipe out large amounts of data forever. There was a lot of work in teh past, for example the Mayan Codexes, that was destroyed and is totally unrecoverable. It was fragile precisely because it was hard to copy and thus there wasn't much of it around. Now, of the orignal hundreds of thousadns of Codexes, we have but 3.

I think it's just a bunch of alarmism.

Re:Easier

[Dionysus]

Copy it to a new format. That is the real beauty of digital. Since it can be perfectly duplicated easily and quickly it's no problem to move it to a newer format.

Don't worry. The movie and recording industries are working hard to make this next to impossible. All for your pleasure.

Re:Easier

[Antique Geekmeister]

You've obviously never tried to recover data from paper tape, 5 year old floppies, or 15 year-old half-ing magtape. People forget the weird formats they kept things, especially proprietary backup formats. When you couple that with companies that wrote the software closing their doors and never releasing their source code, and with the physical fragility of these media, and the unwillingness to spend the time and money to transfer them to the latest readable format, you face a tremendous loss of original scientific and historical data.

Even old SCSI drives can be a nightmare to recover data from if they've been sitting on a shelf for 5 years.

Re:Easier

[deaddrunk]

But how much of their junk is actually worth saving?

Re:Easier

[iwbcman]

Amazing how your post got modded +5 considering just how little thought you put into righting what you did.

While it is true that digital technology makes it easy to reproduce digitial content(ie.copying, transfering etc.) this very same functionality is dependant upon innumerable things which themselves are not digital and hence not digitally reproducable.

Sure I can take any given piece of data from a supported device and transfer it to some new supported device. Sure I can transform any given piece of supported data into any new form of supported data.

But the key word here is 'supported'- do any such devices still exist, if not is it possible to remanufacture such, does there exist any program which can acurately decipher the format of the data, if not is it possible to sufficiently reverse engineer the application which produced that format. 'Support' means *currently* in the state of production/manufacture- and when talking about the future what we can say for sure is that what will then be 'current' is *not* the same as it is now.

Open Source software goes a long way towards ensuring that future generations will be able still access data produced with/by such software. Of course with propietary software this is not the case -once the company which produced the propietary software vanishes, taking their propietary product with them, it is impossible to access data stored in the format of the defunct applications shy of reverse engineering of that format-but the ability to reverse engineer is not *simply* a given, and most propietary formats are never reverse engineered.

The possibility to reverse engineer a format does not mean that some format will be reverse engineered-and in all cases where some format is not reverse engineered there is palpable loss, ie. irretrievable loss of data. Beyond the software dimension all software is dependant upon the pre-existance of hardware-and hardware no longer produced must also be reverse engineered. Of course one can also reverse engineer hardware-but the same providos also are valid regarding hardware-and let us not forget the equipment/techniques/process methodologies used in producing said hardware.

To state this simply:

• data is dependant upon the format utilized by the application used in it's production

• data is dependant upon the medium of storage

• applications are dependant upon OS's

• applications are dependant upon the medium of storare of the OS

• OS's are dependant hardware

• OS's use certain mediums of storage

• the hardware which constitutes computers is dependant upon the hardware used in it's production.

IF at some point in the future we longer have computers or devices which are sufficiently similiar(technologically speaking) to what we now call computers -if I wished to have access to some data found on/in some technological ruin I would need to reconstruct the conditions of that data(for data is nothing more than the codification and storage of itself(ie. binary self-representation))-which means I would need to reproduce the following:

the production methodologies, techniques, equipment and resources used in the production of the hardware which constitues computers which run specific OS's which in turn run specific versions of specific applications which in turn support specific formats of spefic kinds of data.

None of the above are *given*- their pre-existence is fundamental-if such does not already exist good luck trying to reproduce them-particularly given the fact that our hardware designs are not open, nor are the production methodologies, techniques, equipment used in the production of that hardware. Notwithsstanding whether the resources required to manufacture such things still exist.

The fact is that the vast majority of data produced and recorded in the last 30 years has already been lost:

1) the

Which would be fine, unless

[jwigum]

the specialist dropped said cylinder on live TV, and then proceeded to swear...

Re:Which would be fine, unless

[DrSkwid]

that story sounds unfortunate, though I'm not familiar with it, got any more info ?

National Archives

[Krimsen]

Wasn't there just a story ahref=http://slashdot.org/article.pl?sid=05/09/12/ 201234&tid=126&tid=103rel=url2html-7034http://slas hdot.org/article.pl?sid=05/09/12/201234&tid=126&ti d=103> about the National Archives choosing Lockheed Martin to handle this very problem?

Re:National Archives

[Krimsen]

Let me clean that URL up: http://slashdot.org/article.pl?sid=05/09/12/201234 &tid=126&tid=103

Pish posh

[TheCabal]

The article said that a letter accompanied the CD-ROM. Written notes have been around for centuries, and apparently are still human-decipherable in the near future. What makes you think your CD won't be usable in 50 years, other then from rot or something? If yuo really wanted to make things difficult, you could choose a format that was never really big to begin with (or at least not mainstream) such as Beta tapes, but a determined person could still find a way to play the media.

Nothing New

[Anonymous Coward]

I recently had some ancient (1930/40s) UK standard 9mm film converted to DVD. There were two companies in the US that could dot it. As long as there is financiel incentive, there will be a solution.

DRM - the end of mankind?

[MikeFM]

Just think what we'd [not] leave for future generations if some massive fall of civilization were to happen right now. Even if they managed to climb back up to the modern day they'd then hit a road block thanks to DRM. Books, movies, music, software, and most every other record of our modern history totally locked up beyond recovery if you didn't manage to retain a copy of an original player and all the associated equipment that is needed. BANG we slam the door in the face of human kind as it tries to climb back up the ladder after disaster.

People laugh at this idea. It seems so impossible that our civilization could fall. Isn't that what every other civilization has thought before it happened? Funny.. every single one has eventually fallen! War, natural disaster, social changes, etc all eventually pose the risk of destroying even mighty civilizations. We climb forward and then fall back and then climb forward again. That is how we progress. With DRM they destroy a great number of the records that could help a future generation climb back up the ladder using what we've already learned for them. IMO that is nothing less than tragic.

DRM combined with laws that make it illegal to archive material or reverse engineer DRM are dooming future generations from recovery. Even if we could organize an effort to store this data for future recovery we couldn't do so because it's against the law. It's all very short sighted. Everything in our society is about instant gratification rather than planning for the future.

It's pretty well accepted that if mankind falls now and isn't able to recover quickly that it may never recover at all because we've used up more natural resources than we could afford to and we are only now starting to discover the possibilities that will let us save ourselves. If we suffer a fall and can't access our modern learning and culture we may simply never recover.

A related thought.. the best way to backup data in this age is to make lots of copies and spread them far and wide. Make new copies frequently so that no matter what the current medium is there will always be a recent copy. At least us open content fans can rest assured that long after the MPAA and RIAA has faded from memory our work will continue to live on somewhere in somebody's files.

Until the universal eternal format is developed ..

[RustyPelican]

Periodically rolling over data from its existing format/media to the latest new fromat/media before the media or format life expires (whichever comes first) is the only way to perpetually save data. The problem is not in having a reliable method to preserve data. It is an unwillingness on the part of the guardian of the data to spend the resources (operational overhead) necessary to do it. The perceived value of historical data is often viewed by how often it is accessed as opposed to what future insights might be derived from it or what commercial value it might have in markets that do not yet exist. In other words we lose data because operational management doesn't see its contribution to the bottom line on the profit and loss statement. The reality is that these judgements evintually do have to be made because we just don't have enouth bandwidth to store all of the data that ever existed even if we wanted to. Until everyone recognizes that maintaining your data is no different than maintaining any other valuable asset such as highways and city buildings the situation wont change.

So, if you want to have the map to the family fortune readable in 50 to 100 years, store it in the highest linear (uncompressed) resolution available to you and every time there is a paradigm shift in digital media or formats transfer the data again to the new format linearly at the same resolution is the original. If you compress the data and then transcode to a new compression format some data will be lost in each iteration and eventually the data will be unusable.

Check this link regarding the Electronic Records Archive http://www.archives.gov/era/index.html .......Just my 2 cents worth

Simple

[Coop]

Just store it in Word format.

Truly long lived data storage?

[rthille]

Walking thru a graveyard the other day, looking at the grave stones from less than 100 years ago rotting away I was thinking about what it would take to come up with something that really would last for thousands of years.

It wouldn't store tons of data, but I was thinking that plate titanium with holes drilled in it. Expensive as hell, but it might last quite awhile, especially if the plate was about an inch thick.

What is the problem exactly

[Anonymous Coward]

say you want to show you grandson that game you played back in the day, just "layer" your way back to whatever platform you require ex: boot your fancy os which in turn runs linux in a vmware type app emulating windows running dosbox

hardware is much, ah, *harder* than software

[toby]

I think it's important to bundle each set of data-containing artifacts with either an actual reader device which produces some kind of easily understood output, or schematics for same. For instance if you were going to bury a vault of CD-Rom type discs, it would make sense to put at least a CD-Rom drive in

Yep. That's my worry. It's going to be much tougher to actually find the data and read it than interpret the data. Imagine trying to read a CD-ROM, or hard drive, or NVRAM, anything! in a world where the complete integrated systems aren't available. Even if you had specs to say, and IDE interface, you'd have to do man-millennia of engineering to get at the data. We already know this from the impossibility of recovering data stored during the last 40 years (NASA's Viking probe data is the famous example but there are thousands of other cases). The hardware gets decommissioned and scrapped; and even if the media survives, it has a limited shelf life.

And don't even think about it if this is in a post-technological or post-apocalyptic scenario. That's when you want hardcopy! Old-fashioned printouts and photographs... with all their attendant preservation headaches. That should be in the bunker too.

Writing software or even reverse engineering formats looks much easier by comparison.

Re:hardware is much, ah, *harder* than software

[Ythan]

If you're worried about people being able to read your data in the future, why not just include a computer for their convenience? Any old rugged Pentium laptop with a CD-ROM drive would do. I think it would last as long as the media, right? Assuming you remember to remove the battery. :)

Re:hardware is much, ah, *harder* than software

[Kadin2048]

I think you've got a good point about the interface. I didn't really think about that when I was thinking about including the hardware, but you're right, including a IDE or ATA CD-Rom would be pretty frustrating for some reader down the line, if that format no longer existed.

I was thinking about old hardware that I own, and what I'd put in there if I had to build a time capsule. I'm pretty sure I've got a parallel port CD-Rom drive somewhere, that would probably be it. Unfortunately I don't have any documentation on the control commands for it, but it got me thinking.

What would be really nice for a time capsule would be some kind of ruggedized reading device, with a very simple interface and simple machine-readable output. I'm imagining a CD-Rom drive with a little panel (or knobs, even) where you could put in a filename and press "Go," and it would begin to spit it out a serial or parallel interface, one bit or character at a time. Maybe an 8-bit parallel interface, with additional pins for clock, logic high, low and ground, at some fairly low bitrate. Even if you didn't have any other old equipment, it'd be pretty elementary for someone probing around with an oscilloscope to figure out what was what. You could even do some pretty rudimentary pictograms for clock (square wave symbol), high (1), low (0), that would help the interpreter. Unless they've transcended electricity and binary logic altogether, they ought to be able to puzzle that out, and route it into whatever data storage equipment they'd rather use.

If all you had on the storage discs was 1-byte text code like ASCII (heck 7-bit BAUDOT would be OK too), it ought to be pretty simple for someone to do a frequency analysis and figure out what's what, even if (the horror) they've lost or forgotten ASCII. After that, figuring out our language would be up to them and the Rosetta Project people.

Re:hardware is much, ah, *harder* than software

[InvalidError]

Well...
1- lamps (backlight) do 'breathe' a tiny bit so, over a long period, the backlight can stop working.
2- lubrication (HDD, fans, CD/DVD drive, etc.) will evaporate or be absorbed by materials
3- some components may rust or be damaged by their environment some other way
4- capacitors lose capacity over time
5- atmosphere can contaminate semiconductors
6- etc.

But a dead system still leaves complete system specifications behind... then again, possession of a non-"DRM du jour"-compliant device with an equally non-compliant OS/firmware may be illegal and punishable by death penalty before the data can be transcoded.

Re:hardware is much, ah, *harder* than software

[Rekolitus]

Yes, you just said it. Who knows if they're going to know what bit represents 64, etc, etc. The alphabets may even have changed, and, while they may still be able to read English, are they going to have the ASCII specification memorized?

Re:hardware is much, ah, *harder* than software

[toby]

Writing software or even reverse engineering formats looks much easier by comparison.

Of course, I was assuming that 'our' brains, with our education and knowledge intact, would be looking at the problem. Now imagine some kid with no education in numeracy, let alone mathematics or computer science, having to operate a computer and reverse engineer the file formats on it... So maybe we would need to carry forward libraries, and enough materials for people to learn the sciences necessary too. Oh, I know, let's just not have that apocalypse.

Re:hardware is much, ah, *harder* than software

[Anonymous Coward]

this is why you need john titor to come to the past a bring a pentium laptop to the future.

Perhaps the Pirates will save us.

[cwsulliv]

To the extent that DRM and the Law succeed in eliminating "Piracy", we are without question in for a Cultural Dark Age.

Although some maintain we will always have books and libraries, improvements in e-book readers and general acceptance of them, plus the increasing cost of hardcopy publications, will in all liklihood lead to most literary works being released only in electronic form.

How much money can we expect publishers to spend transferring works to current-technology media when 97% or more of these works lose their market value after just a few years, despite the fact that some of that 97% will many years later be recognized as classics. Or would be if someone had the ability to read them at that later time. But with DRM that hasn't been cracked, most scholars won't have the chance.

DRM & Out of Control Intellectual Property Law

[salesgeek]

While file formats and media have presented a problem, there is plenty of evidence to suggest that digital information has an extended lifetime, and the most valuable information will be converted into newer formats as well as more simple and fundamental formats. Simple formats like ASCII text have handled the test of time. I'm more concerned by the potential lockdown of information through overzealous use of DRM technology backed by overbroad intellectual property laws. Just like the last dark age, the next one will be the result of people trying to control other people.

Technology

[SEWilco]

Because a CD-ROM is an optical technology, you can include a definition of the data storage method. They may be able to put it on their home office optical scanner and decode the image of the data dots. Or include a printed copy of the treasure map.

For optical media, it's very easy...

[tlambert]

For optical media, it's very easy... assuming the media actually survives, it's the same way this guy plays vynil LP's using a flatbed scanner:

http://wired-vig.wired.com/news/digiwood/0,1412,57 769,00.html

Obviously, in the future, ultra-high resolution optical input will put the current scanning/video technology to shame; they will just need to scan the thing in and run a program against the data to get the contents of the media back.

Re:For optical media, it's very easy...

[Anonymous Coward]

Using the scanned album as an example isn't a good idea. It's essentially saying that in 100 years, we'll be lucky if the "recreated" version of a printed page is rectangular, with some blurry things on it. Nevermind that you won't be able to make out what it actually used to say.

The coming digital dark age: The greatest threat.

[windowpain]

A far more serious problem is the degradation of the English language. Many college graduates can barely write. This is particularly true in the engineering disciplines. These people understand the need for redundancy in data processing and storage but don't understand its vital role in natural language.

Anyone who believes that knowing "its" from "it's" is not important or that spelling and grammar are not important because "people can still understand" poor writing don't understand that proper spelling and grammar are an important part of natural language. The reason "people will understand" is that other signals in writing can usually take up the slack of conveying meaning.

English is about 40 per cent redundant. That means a message can get screwed up pretty badly and still be understandable. But when we stop using seemingly unimportant cues like proper spelling and grammar we lose (not loose!!!) some of the information integrity "insurance" they provide. Every lapse we come to accept weakens the ability of our language to communicate clearly and unambiguously.

I recently read a poem that illustrates my point rather poignantly. I wish I had written it.

Windows is shutting down, and grammar are On their last leg. So what am we to do?
A letter of complaint go just so far,
Proving the only one in step are you.

Better, perhaps, to simply let it goes.
A sentence have to be screwed pretty bad Before they gets to where you doesnt knows The meaning what it must of meant to had.

The meteor have hit. Extinction spread,
But evolution do not stop for that.
A mutant languages rise from the dead
And all them rules is suddenly old hat.

Too bad for we, us what has had so long
The best seat from the only game in town.
But there it am, and whom can say its wrong?
Those are the break. Windows is shutting down.

--The Guardian Review, 30 April 2005

humbug, it's evolution!

[the0ther]

Software is subject to the laws of evolution, like any other meme. It will continue to reproduce across any sort of storage medium that comes along. Don't worry. You underestimate the number of copies that exist as well as the ease of copying. Perhaps the biggest (and most plausible) worry about digital-rights-management technologies is the scenario where a particularly excellent application goes extinct because the media it exists on is no longer accessible. I say this is highly unlikely because the fittest applications will always be copied onto the newest media by users. Seems like a "yeah, duh" to me.

Are we overlooking one inportant fact, internet?

[jimmydevice]

The internet, Google and persistant archives are saving this history now. The idea that CD roms will not be readable, or cd drives that read this media won't be available in 40 years seems laughable. I'm sure somebody is going to produce legacy CD media readers, the market is there. As for 7 track mag tape, I can't see anybody coming out with a home version. The only dead tech is un-archived tech. JimD.

It won't be that dark

[kilodelta]

I work with a bunch of library science and archvist types who worry about this all the time.

It's such a pain taking care of books that are a few hundred years old. But they miss the point when it comes to digital.

For example, data I had on 5.25" floppies was moved to 3.5" floppies, then to a 20MB hd, then to a CD-ROM, then onto my current system.

If it's that important you transition it to new media.

batteries

[toby]

Assuming you remember to remove the battery

If they can't power it (post-apocalypse?) then it again might be best to fall back on hardcopies. Which may even last longer than the computer hardware.

This topic has been discussed more thoroughly in older threads; I'm not really an expert on data longevity. I can just see that the problem is much trickier than it looks. :)

Re:batteries

[BrokenHalo]

If they can't power it (post-apocalypse?) then it again might be best to fall back on hardcopies

This seems to be one of my days for pet peeves; or maybe I'm just getting so old and cranky I have more of them than most. ;-)

You don't need an apocalypse to bring down the power. Here in the so-called civilised world we can usually take it for granted that 240V (or 110V) AC will be available, but it doesn't take that much to take it down. New Orleans is a case in point. Or a well-considered terrorist attack.

I'm not trying to say OMGOMGWhatAboutTheChildren, the point I'm clumsily trying to make is simply that we are probably allowing ourselves to become too dependent on computers. If a document (say) is important to us, we should make damn sure it is accessible without having to depend on a gadget to read it.

Re:batteries

[toby]

Which was more or less my point. "Post-apocalypse" is shorthand for a very long future without power...anywhere. Of course there are shorter interruptions but data retention doesn't enter into them so much (apart from ensuring the hardware isn't destroyed before power is restored).

Re:batteries

[gronofer]

It doesn't take that much to take it down, but there seem to be plenty of people who can get it back up again. You would have to assume that they were all killed, or something.

Re:batteries

[Blkdeath]

It doesn't take that much to take it down, but there seem to be plenty of people who can get it back up again. You would have to assume that they were all killed, or something.

Yes, mainstream, large-scale power generation is fragile and has relatively limited people who can manage it. However, the knowledge required to create a battery from something as simple as a lemon carries down from generation to generation. I'm sure we'll always be able to create some form of electricity even in a post-apocalyptic future.

Re:batteries

[trentblase]

Include a generator....

OpenDocument

[Anonymous Coward]

Isn't this precisely what the State of Massachusetts is attempting to solve by using OpenDocument? http://www.dwheeler.com/essays/why-opendocument-wo n.html

There's a really easy answer actually

[jonfields]

Throw your computer in the atic next to the disks. That way they'll have everything they need...except perhaps the power plug converter.

There'll be someone with an Amiga...

[LoadWB]

...lurking around with a Catweasel X and various drive equipment. Come see us.

XML

[Devistater]

nobody has mentioned XML type docs? WTF? Its basically pure text with some tags. If you had to you could remove the tags if they didn't apply to anything anymore and just look at the pure text.

The file format is the easy part...

[bgspence]

The decoding of a file's bits from the bits on a CD would be the hard problem.

If they could read the files, they probably have a system that can read the file format.

Why not just leave an entire system?

[Anonymous Coward]

If i was going to leave some kind of information, to be discovered at a later date, wouldnt it be best to take something like, say a laptop. Weather it be a video message, whatever, install your OS, and set it up to play whatever message, or open whatever file immediately on boot. then, seal it in a vaccum package.

the only potential issue o see is the death of the battery. the best thing in this case, would be to leave a power adapter with it, with printed instructions, and detailed explanation of the voltage and amperage requirements.

That is something they will know about in the future, because, power, although it may be different voltages, etc, will still be used in some form.

Thats my 2 cents.

And patents do that

[Eunuch]

The patent system collects all this information and will be very useful in the future. Unless we've transcended humanity.

WordProcessor Recovery Possible

[Kadin2048]

I'm not sure about Wordstar, but I'm fairly certain with MS Works that if you open up that file in your favorite text editor that your data will be in there, and you should be able to just copy it out. Naturally you'll lose all the formatting, and probably have a lot of crappy unreadable characters introduced, but I'll bet you'll recover the bulk of it.

I had a similar issue once with a very nice (but very dead) word processing program that I used to use called WriteNow -- where the developer has stopped work and sales on the product but refuses to release it even in binary form to the public -- I had a bunch of disks of documents, but none of the original program disks and no computer with it installed anymore. I was pretty stuck until I just opened up the files one day and searched through them until I found the plain text. A few minutes of cleanup in BBEdit later, and I was ready to go.

Needless to say, they're all stored as .txt files now. Anyway, I wanted to say that I can definitely feel your pain with a problem like that, but that there are solutions to at least one of the problems you described.

Re:WordProcessor Recovery Possible

[WinterSolstice]

I will take a look. Last time I checked, both DeScribe and MS Works were pretty severly managled, but perhaps a perl script or two could be made :)

The wordstar ones would require a 5.25 disk reader (and valid disks). Those may be harder.

Thanks for the reminder!
-WS

Including interpreter source code?

[tendays]

What about archiving along with the data the software that's able to read it? (i.e. the source code - looks like C is a format that's going to last for a while)

MOD PARENT UP

[andrewbaldwin]

I wish I had mod points to give this submission.

It's lamentable how the English language is mistreated these days. As the writer said the great strengths of the language are its inherent redundancy and the human ability to deduce maening from context. Contrast this with most (all?) programming languages - we'd never defend sloppy coding and demand that the compiler 'knows what we mean'.

English has been described as a living language. The language has exised and evolved for hundreds of years, it has accomodated new ideas and 'forks' (eg English and American 'strains'). Nobody is advocating a complete freeze on changes; they are advocating avoiding misuse. English remains in good health BUT like a living organism there's a limit on the amount of abuse it can absorb.

Before the flag waving starts - I'd agree that the writer's views apply both sides of the Atlantic

Re:MOD PARENT UP

[andrewbaldwin]

maening ==> meaning

Lesson: -- use Preview :-)

Simple...

[Anonymous Coward]

Use Google III...

The Dark Age

[Anonymous Coward]

So you're siding with Tracer Tong too huh?

Is this a bad thing?

[shish]

All the stuff people care about'll be copied back & forth onto the latest media anyway, and probably in several locations via some sort of network; what's left is stuff that's been implicitly declared "not important enough to update"

Hey, I've read this before

[aiken_d]

Dateline: February, 1890

New "photographs" may be useless for archival purposes

Scientiests say that the dyes used in so-called "cameras" may not provide the kind of lasting record that traditional stone tablets have provided. In fact, left in bright sunlight for 50 years or more, photographs may be completely unreadable by even the latest 1890 technology.

This will surely mean the demise of modern civilization, since future generations are very unlikely to care enough about the past to devote any energy at all into preservation and reclamation of older information. Anything that can't be read by 1900 is likely to be lost forever.

It's yet another sad commentary on the state of modern civilization, and one more reason why manufacturers of stone tablets and chisels shouldn't throw into the towel too soon.

Cheers
-b

Put it on the net

[stesch]

Aren't we living in a digital age where everyone can go online and every piece of data can be online?

For small applications we still have to carry around some storage system. Like memory sticks in digital cameras. But software can be bought online and there's no need in shipping a CD anymore. And we will sure get rid of the memory sticks in the cameras, too. Maybe as soon as UMTS gets widely available, maybe later.

Dupe from Scientific American 1995

[paulatz]

I'm sureI have read the same article several years ago,I cannot remember were, maybe on Scientific American or such. After a search on sciam.com I have found this dated January 1995, more than ten years ago. Are we reading the older news ever posted on slashdot?

Mass solves the problem

[drownie]

What people are forgetting is the amount of data we have, one hundred years ago there weren't that much photos around in the first place. Today ... well if we would loose 98% of all data available at the moment we would still have to much data in 100 years. Imagine in 2100 we will have 100 years of satellite images for some areas, we will have homevideos, TV shows, Internet archives... and thousands of private foundations and archives. At the moment there are more books printed than ever before. Here in germany we have the http://www.bundesarchiv.de/ it stores all interesting books articles and all laws and speeches in parliament on microfilm in a salt mine. Just to get the history facts right: My personal libary is ten times the size of all books we have from the dark ages. Maybe we will have some problems with lost data in 100 years but compared to the problems we have today finding data from 1900 it won't be a problem at all. And of course we will have infinite number of technical collections ( the history of the cell phone ... ) and if we loos the development data of the Ipod or some of the data of the Space Shuttle... I don't care, who needs it ?

You time travel...

[Mendy]

...back and find a computer to read it on. I mean that's what they'll do when they need an IBM 5100 isn't it? :)

What about all the horse manure in 2000?

[Fredge]

This question is akin to somebody in 1900 asking what the world would be like in 2000 when the population kept growing and everybody had horses on the street - "think of all that manure accumulation - how will we walk without stepping in crap?"

The point is - the question is irrelevant. In 100 years, assuming the continued growth of storage mediums, the average personal user will have access to terrabytes, if not more, for personal use. I imagine that the most basic of ISPs (if such an entity continues to exist separately from other existing utilities) will provide users with gigabytes of personal space online to keep store/back-up their data. The only reason to put things on physical mediums will be for short-term backups.

I think a more pressing question is "will we be able to find the needle in the haystack?" Sure - Google does a decent job of indexing the internet now but even they are not 100%. Also the fact that while they may not be 'EVIL' today, it only takes 1 CEO change for them to become what most other companies are and then it's up to the next do-gooder to start an index from scratch. Then, assuming you can find stuff, you'll have to break the 200Mb encryption key. Luckily, the local Kinkos will have a quantum computer that you can use for $7.50/hour.

Re: Flaky laser prints.

[deimtee]

The only times I have had problems with toner flaking are when using coated art papers in machines that weren't meant to take it.
The old hp laserjets were beasts of machines (in a good way, built like tanks and ran forever).
If you had problems with flaking toner either you were using clay-coated paper or your printer's fuser was under-temperature (probably a faulty temp sensor on it, reading high)
For archiving you should use rag or hemp uncoated paper.

Open Standards

[bagatonovic]

I believe that open standards are a good way to make sure this probably will not happen to you. The person who said XML was right on. URL:http://perens.com/OpenStandards/Definition.htm l

Junk shops, anyone?

[Exzyle]

Well, put it this way. Let's say from my generation, someones father put instructions on an 8 track on how to gain the family fortune. How would you play one of those ancient things on the devices of today? It's very simple, really. You'd just make your way down to the local pawn or junk shop, wouldn't you? I'm sure that there are more than enough CD-ROM drives to go around. And with how they're making DVD readers and writers backward compatable with the good ol' CD formats, it is highly possible that several installments of future media readers and writers will be able to at least read a regular ol' CD. This, of course, all provided that junk shops are around in say 40 to 50 years. I don't know about how old the rest of you are, but I don't plan on kickin' it before that happens. Of course you can't expect death, can you? Not that I have a fortune to give to my future children anyways.

No problem

[deaddrunk]

We just need that Apple laptop from ID4, that thing can interface with alien technology, a mere IDE drive should be easy.

Likely untrue.

[JabberWokky]

I don't buy it. My dad has old reel to reel tapes of him performing half a century ago. I can take them to a variety of companies that specialize in transfer. Same goes for 8mm family film reels, VHS tapes, old records, there are even companies that transfer 8-tracks and radio carts. Ditto for 5 1/4" and even 8" disks in a wide variety of formats (I saw an ad for 8" data recovery a year or so ago). For some formats, it's not cheap, but heck -- there are still companies that recover punch card data!

When they see the CD, their first thought will be along the lines of when I discovered my Dad's reel to reel spools. Their second thought will be "Hunh... we should Google somebody who can transfer this to the whiz bang tech of today". Me? I had to call around. It will be easier for them.

--
Evan

Dark Ages?

[Liam Slider]

Dark Ages? The period of the Middle Ages commonly refered to as the Dark Ages (the whole Middle Ages certainly can't be called that) is distinguished by an end to trade, a near end to literacy and education, and nearly constant warfare. How is "not being able to read old CDs" a "Dark Age"?

Also... Do you know how many documents from ancient times survived? Very few. What we know of today, the documents that tell us of our ancient history....is a small fraction of what actually existed. Most were destroyed thousands of years ago. Not just at the Library of Alexandria either. So somehow, the idea that most of our records won't be avaliable to people in the future...that somehow means a new Dark Age? I don't really get this.

Permanent storage

[bb5ch39t]

That's why I'm working on a USB connected granite chiseling machine. Hey, it worked for the ancient Egyptians! Of course, I will need some sort of "master slab" with multilingual messages so that the future can decode the language. My own rosetta stone.

Dead Media Project

[qengho]

Bruce Sterling has been thinking about this stuff for some time now.

Hardware, OK, but software?

[anthony_dipierro]

But they've never seen a CD before - except in old movies - and, even if they found a suitable disk drive, how will they run the software necessary to interpret the information on the disk?

I've never understood the software argument. These formats are well documented and short of a global cataclysmic disaster, they're not going to be lost. A programmer could whip up software to read the format in less than a day, or more likely by that time a computer can be fed a standards document and figure out the format all by itself. Even if the format is proprietary, if it's well distributed then someone will have software that can read it, and this can be reverse engineered to find out the algorithm.

It's not like software formats are that complicated. If you can read the bitstream, it's generally going to be trivial to read the content. The only problem in this regard might be an exotic proprietary format which was intentionally designed to be difficult to read.

Codecs?

[Anonymous Coward]

So maybe if I find the rest of the Codexes, I'll finally be able to find one that plays this video file I got off the net!

Not really an issue

[Anonymous Coward]

By then, the CD will be so scratched and smudged, it'll be useless for data, anyway.

Are they?

[porkchop_d_clown]

Think for a moment - substitute "7 track tape" for "cd rom", now tell me how you would find out what file format was used.

The redbook definition is certainly more widely disseminated than the encoding methods for 1960's tape storage but you're still underestimating the skills a person needs to read the CD is also capable of finding, reading, understanding and coding to the redbook standard.

Re:Are they?

[anthony_dipierro]

Think for a moment - substitute "7 track tape" for "cd rom", now tell me how you would find out what file format was used.

I would assume you can use any file format you want on a "7 track tape".

The redbook definition is certainly more widely disseminated than the encoding methods for 1960's tape storage but you're still underestimating the skills a person needs to read the CD is also capable of finding, reading, understanding and coding to the redbook standard.

As long as we're talking purely of software, I don't think I am. In fact, you probably won't even have to write any code, someone somewhere will likely have kept and up to date standard which runs on current machinery. Alternatively, there will almost certainly be an intel processor emulator (software, a la bochs) available, and you can just plug current software into that. For something as ubiquitous as the CD I just find it impossible to believe that someone somewhere isn't going to keep around software to support it. The big problem is communicating the bit stream.

Give me a bitstream from a 7-track along with a rough idea of what format it's in, and I'm sure I could extract the majority of the information. And this is with today's technologies - I fully expect in 50 years we'll have the natural language processing technology for computers to read and understand the specs themselves.

Sigh

[Deliveranc3]

If by 2045 they can't just take a picture of the disk and have the patterns/data automatically analyzed.

ex: Graphics through facial recognition and OCR, Text through a spell checker, code through an assembly language checking algorithm. Then a CS major becomes a joke, don't underestimate the profession, it's producing more progress than any other industry.

Maybe because

[porkchop_d_clown]

in 10 years we haven't solved the problem.

I worked for $BIG_PHARMA_CO for a while, and they considered the issue a tremendous problem. They are used to saving research data for a century and are deeply worried that 50 years from now they won't be able to read their current data, because of format changes and media degradation.

Internet makes this article pointless.

[Anonymous Coward]

nt

Future digital dark age? It's already here

[rfc1394]

• You can't buy 5 1/4" floppy drives any more.
• Few places are selling 5 1/4" diskettes for the occasional use in places that still have a need for them any more
• 8" diskettes died off perhaps ten years ago despite that I used them for several years
• As is noted, mag tapes are dead (a 1600" mag tape only holds about 60 meg of data and is much more expensive than even
• overpriced zip disks, which are also going away as did
• 2 GB Sparq disks when IOMega bought out the format and killed them off, and
• now people are moving to DVDs because one DVD can hold as much as 6
• CDs, except that, of course, more machines have CD readers/writers than DVD readers but the cost has come down so much that CDs are starting to become old hat.

A lot of machines either aren't networked or didn't have networking capability and thus using the Internet or anything similar to transfer files isn't necessarily an answer. Plus, the problem is many of these machines have data locked in proprietary formats that may not be accessible. (Anyone have any programs that use the RMS database format from Digital Equipment on their machines?)

There's also issues for cash-strapped institutions having to spend money to constantly move data from format to format as it changes. I've lost stuff I had because it was on older formats I can no longer read. I've been fortunate that many of the messages I wrote in the past on various mailing lists were kept by archive sites and I've been able to recover some of them. But a lot of stuff I otherwise saved is gone because I can't read some older disks.

There's also the possibility of corruption. I had an important set of programs that the source files were damaged on the CD and I can't read them; I may be able to find the person I sent the original tape from which I obtained them but if I can't, I'm out of luck.

Keeping some of this material on paper isn't always an option; if I was to keep paper listings of everything I have, it would cost a small fortune and would probably require room for thousands of pages of storage. And putting program listings on paper of a program that's 20,000 lines in length wouldn't help much as I'm unlikely to type that program back in if it were to be lost.

Important stuff now I keep multiple copies around but there's always the possibility of loss or damage, and if one doesn't keep count of what one has one can find one has data one has irretrievably lost due to format obsolescence or software obsolescence.

keep it live

[petermgreen]

copy it from one media format to the next BEFORE the old one dissapears or keep your data on your hard drives and copy it to your new ones each time you upgrade.

Low capacity removable media like floppies and to some extent CDs is the enemy of data preservation because it makes the job of copying stuff to fresh media require far more human labour.

Re:keep it live

[toby]

copy it from one media format to the next BEFORE the old one dissapears or keep your data on your hard drives and copy it to your new ones each time you upgrade.

That obvious procedure has been mentioned by other posters, and is commonplace, but it still leaves many possibilities for loss open (see ensuing discussion).

For instance, software obsolescence. On a small scale, this happens frequently today: How many files (e.g. obsolete and/or proprietary word processor and graphics formats) do you have in your archive that your current applications can't open? Likely quite a few, unless you store everything in ASCII text. Do you still have the older applications? Can they be installed on your current hardware and software setup? Will they successfully run? What about in 2 years' time? 5? 10? 30? 100? So you see you have to carry forward the means to interpret that data as well as the means to physically store, read and display it. Not trivial at all.

This issue is also increasingly raised, of course, in connection with statutory requirements for transparency and longevity of government documents (see recent Massachusetts story).

Re:keep it live

[petermgreen]

yes that can be a problem but shouldn't be too bad as long as you stick to formats that are either very simple or very well documented (e.g. i don't think the deflate algorithm will get lost any time soon for example).

and you can generally rip the raw text out of most word processor formats quite easilly without needing the app that created them. Its messy but you can get the raw information.

Re:keep it live

[toby]

you can generally rip the raw text out of most word processor formats quite easilly without needing the app that created them. Its messy but you can get the raw information

Prevention is better than cure. We're human beings. Can't we do better than this? :)

All it akes is one!

[mattma]

All it takes it one person to have the equipment and the floodgates open. Today when you go an visit some dinky 100 ppl town in the middle of Wyoming you can find museums filled with working contraptions during the wild west / gold rush settlements. The key to this comment is that those contraptions still work. The people in the museum actually know how to use them. So while the oh-so-big metropolis might not have these at their disposal, you would be surprised at how big the world actually is if you stepped out of our metro areas once in a while.

Think about this from another angle

[petermgreen]

Suppose you decided you'd give each english letter a randomly assigned 8 bit number to send your secret messages how long do you think it would take a cryptographer to crack it with a reasonable chunk of text. (hint not very long)

given that how long do you think it would take them to figure out ASCII which puts the entire english alphabet in order?

Old movie footage..

[oliverthered]

Well, there are a number of places near where I live that will transfer old cinifilm onto DVD, I expect the same will be the case for people trying to recover data from old media in the future

Easiest

[SiliconEntity]

They just turn their nanobots loose on it. These eat the disk and simultaneously create an atomic-level scan of the device, so that the whole disk has an exact representation in software. Using the known specs for CD-ROMs, along with error correction algorithms to account for degradation in the CD-ROM medium, they can turn the encoded data back into raw digital form.

Copyright and DRM are the big issues

[WareW01f]

I thought about this as I heard that the local public radio station was going to archive all of its old recordings to digital. Even with NPR (a *public* radio station that I give money to) they are putting audio on the Net in Real Audio. This was OK as long as I had NetTransport and a Real Audio player burned into the ROM of my Treo... But when I picked up a LifeDrive and finaly had gigs of space to carry audio, I was rudely awakened to the fact that Real *has* no player that works on the LifeDrive. Makes you wonder *what* format they are going to archive things to

Using Linux and dealing with my wife's Mac, I've sadly come to be used to this, but the bigger issue here is both that in the future, people will find that the media they find is not playable, not do to the fact that they can't read the media (which is still and issue) but because the media is in an undocumented, proprietory format. Look at DVD. They mention Laser Disc in TFA but I can't (legally) play a DVD *today* on my Linux box, much less years from now after the "trade secret" knowledge of how to decode the info is gone. The same for project Gutenburg vs all the e-book formats. I think it's a good idea to get *everything* in digital, but with all the crappy DRM ideas that will only let an e-book work on *one* reader, that info is doomed. Info needs to be digital so that the people *today* have affordable access, but in something more concrete for tomorrow.

It's not tech that is the issue here folks, its the IP laws that will kill the media. The sad part is that it's probably safer to put my will on a USB key than a CD (both stupid ideas if you want people to read them) and paper makes both look fragile in terms of time. But what would you say if the records were in a bank in a safe depost in New Orleans? (9th ward maybe)


To make a short story long. Yes, digital will pose and issue, but only if we don't think about both the medium *and* the format. The sick joke of the industry today is that while we could now probably throw a portable DVD player in a time capsule with the DVD of our message, if the RIAA and MPAA have their way, the people of the future won't be able to copy the data. That would be piracy!

Don't Panic!

[elseedy]

Just head over to any public school and you'll be sure to find a PC with a CDROM (perhaps even up to 2x read!) until the end of time.

Snort.

[porkchop_d_clown]

You're oversimplifying. I can't use "any format I want" I have to know the format the tape was written in 40 years ago. And, given your obvious ignorance of even the existence of 7- and 9- track tapes, I defy you to lay your hands on the specs and tell me how to read the data off them.

I'll even give you a hint: They don't even use ASCII, and may not use EBCIDIC.

Re:Snort.

[anthony_dipierro]

You're oversimplifying. I can't use "any format I want" I have to know the format the tape was written in 40 years ago.

You misunderstood me. What I meant is that the person who encoded the data 40 years ago could have used any format on that tape. Knowing that it's a 7-track tape doesn't help very much on the software end of things. It's necessary information for the hardware, but that's about it.

And, given your obvious ignorance of even the existence of 7- and 9- track tapes, I defy you to lay your hands on the specs and tell me how to read the data off them.

Reading data off tapes isn't the job of software. What I said above is that once you've gotten the data off the actual media, it's not going to be a problem to read the formatted data, certainly not for something as ubiquitous as for instance ISO-9660. There will almost surely be software 200 years from now which can read ISO-9660, and if not it'll be relatively trivial to write some.

You seem to be talking about the physical disk formats here, which is something altogether different. I agree it's probably going to be a pain in the ass to get a CD-ROM reader even just 50 years from now. Even if you have the CD-ROM reader, IDE and SCSI will likely be hard to come by. But let's say you have a working computer with a CD-ROM drive and a serial port (I think support for null-modem serial connections is going to be around for a long time to come, at least on specialized hardware, and the technology is simple enough to build in your garage if it really came to that). Once you've got that connection to transfer the raw bit streams, I just don't see how it's going to be difficult to read the data, regardless of the file format, so long as you've got a spec.

I'll even give you a hint: They don't even use ASCII, and may not use EBCIDIC.

A quick google search suggests the standard usage was 6-bit bytes plus 1 bit parity. Unless these tapes were written to by some really exotic hardware, that's what format they'll be in (reading of this data is probably handled by the hardware, though). Of course it's not ASCII or EBCDIC, as they use 8-bit bytes. You can't really say much more about 7-tracks in general. The character set might be BCDIC, but then again, it might not. If it is BCDIC, it's trivial to convert that to ASCII. Of course that just gets you a character stream. Now you've still gotta figure out the higher level formatting. But none of it is difficult, so long as you know what the format is in the first place.

Ah, but which BMP format?

[Richard Steiner]

Windows 3.1, OS/2, and Windows 95 all have their own BMP variants.

http://atlc.sourceforge.net/bmp.html

Re:Ah, but which BMP format?

[ratsg]

Richard's right about the BMP files. TIFF is the way to go.

I wouldn't worry

[siim04]

I mean by that time they would have processing power to crack any format in no time. And even if they do not have CD-players any longer thay could make a detailed 3D scan of it and crack it then.

  And why would anyone use DRM on archives? I mean DRM is made on other purposes. Oh, but if they do use it, the computing power available will crack it in no time as well (After all, there is no uncrackable encryption - even one-time pad can be cracked).

  Of course we must make sure we do develop higher processing and computing power by then.

I can imagine...

[gamer4Life]

...the grandkids finally being able to play the digital "document"

--and it ends up being a porn video shot by their grandparents.

"Wow grandma, you looked really good back then!"

Imagine getting the box in the attic now

[Anonymous Cowpat]

from your native-American (is that the PC term now?) great-great-great-uncle "I've recorded the instructions for getting to the family fortune in navajo onto this wax cylinder...".
Just one example of a format that isn't commonly accessible today. You COULD do it, but you'd need someone who speaks a very rare language (I have got that right, haven't I? if not replace navajo with a best-part-of-the-way-to-dead language) and an extremely dated piece of audio reproduction equipment. It would be possible to get the two together, but pretty difficult. I imagine that it would be the same with a CD of images, the CD specification and the jpeg specification would in all likelyhood still exist (in the same way that 100 year old scientific papers still exist) but you might have to recreate the equipment to access them. I suspect that wax cylinders, when they are played, are played on new machines. 100 years on people can still figure out a wax cylinder or follow an originary design well enough to make a modern player, unless something catastrophic happens to the human race, the technology and skills required to a make a CD drive will likely be commonly taught as basic skills. When the CD drive was new it was brand new and cutting-edge (ish) now 1st year physics students are being taught how the laser diodes work, and people in clean-rooms all around the world can churn them out to their heart's content (the laser being the hard part of a CD drive, belt, accurate-speed regulated motors and a decoding chip that follows defined standards aren't that hard - not having tried it myself, of course).

This will not be a problem at all

[phision]

I think the people in the future will be intelligent enough to read information no matter what format it is written. The same refers to us - we can decode and read ancient texts written hundreds of years ago, even though they are badly damaged. Our successors will be able to take the dust remaining from a buried CD and recover the information written on it.
And, they will be able to make and use a suitable hardware the same way we can make and use hammers and bows today :)

data retention failure

[icbkr]

I glanced through the 341 comments and didn't see my response, but my apologies if it's buried in there below my threshold.

The example is contrived, but as some have mentioned, this is already happening. I work in healthcare, and we try to maintain electronic patient information forever, which, unfortunately, is too expensive to do in most cases. This is some of the most important information one can preserve over a person's life span, and yet we are unable to do it due to costs and bureaucracy.

The first problem you must address is media. That's been dealt with elsewhere, so I'll move on to the bigger issue which is exotic hardware. Most people think exotic HW is a mainframe. That's nothing. You can obtain specs on a mainframe. What you can't get specs on is super-proprietary medical systems. By the time we retire a system, it's manufacturer is out of business or has been repeatedly sold and purchased to the point that nobody can do anything except board swap for repairs, and when we sunset a machine, it's because even boardswapping isn't possible anymore.

Now I work at a high end organization, one of the best in terms of digital capability. But we repeatedly have opted to sunset machines and cross our fingers rather than migrate data off to a newer format. It's a simple risk management exercise. What are the odds I'll ever need that data versus the odds of the machine being usable. We might get one or two requests in the decade after sunsetting, and it is unlikely there will be any legal exposure, since all healthcare law requires us to "try really hard" to preserve the data. Only if we were negligent or malicious could it cause trouble, and frankly, that's not going to happen, although the ambulance chasers may try.

Nonetheless, as a 21stCenturyDigitalBoi, I have dealt with this problem in other organizations (i.e.: for profit) and we had little difficulty pulling data from goofy hardware to current media. It can get tricky, but rarely would it exceed $50,000 in T&M. It's just not that big a deal. But in healthcare, you're dealing with people who usually haven't come up as rocket scientists, and are verrrrrrrrrrrrry conservative. So they weigh the perceived risks and make a call that it's not going to happen, we'll just cross our fingers. I will probably die an early death from the frustration this causes, but at least I know my medical records won't be found by my grandchildren in the attic, since they won't last that long.

As with all issues involving judgment calls, it's about education and argument. Anytime you want to spend money for something that's not operational, especially in a not-for-profit, you are going to face the challenges of Hercules to get the bureaucracy to move it's ungainly butt into the current decade.

We've migrated lots of other stuff to keep it up to date, I see no reason why we shouldn't be migrating all the data we want in any industry. I've still got PHRACK issues from way back that went from email to 9-track to floppy to DVD.

So the penultimate point is that this is a management issue. If management doesn't perceive the loss, and doesn't know how to get around the risk, you're going to lose the data.

already happening: i REALLY want my Apple][ data!

[ClioCJS]

Seriously... I have 500 Apple][ disks. I have an Apple emulator on my PC. I want to get the discs converted into image files on my PC.

But..... No apples anywhere. The solution involves finding an Apple2GS, and an older MAC. Both are hardware I don't have access to.

I would pay, but even at a dollar a disk, this would become pricey.

I would also share all data with anyone who would help me. There are probably literally over 1000 games and such.