Slashdot Mirror
The Myth Of The 100-Year CD-Rom
Toshito writes "Are we putting too much faith in the ubiquitous "recordable CD", or CD-R? A lot of manufacturer claims 100 years of shelf life for a CD-R. But in real life, it can be much less. Expect failure after only 5 years... Personnaly I just discovered 6 audio cassettes with the voice of my late grandfather, talking about old times. These tapes are copies of reel to reel recorded in 1971, and they are still in excellent shape.
I was thinking about digitizing everything, do a little noise reduction, and burning this on CD's, for my childrens and great grand-childrens enjoyment, but it seems that old analog tech from the '70 is more reliable than digital. The full story at Rense. Other links about the subject: Practical PC, Mscience, and an excellent reasearch by the Library of Congress (warning! PDF): Study of CD longevity, html version (google):Study html."
Or just burn to multiple cd's, that way the chance they all go bad is low.
The danger in "old" storage formats is lack of machines to read them. Those tapes may be in good shape, and so might the data on an 8" floppy I have, but the 8" floppy is effectively lost to me because I don't have easy access to a drive that can read it anymore! The paper tape programs I "printed" out from a VAX PDP-11 are probably good (if I hadn't lost them years ago) but I can't get to a tape reader, etc.
You almost have to make dozens of copies of data on a modern cheap format, and keep moving it forward.
How do you know there is no loss with analog?
Analog quality loss is acceptable, because it results in static. Digital loss isn't acceptable, because (at least practically) it is a binary property...the CD works or it doesn't. Scratch the hell out of a record, and at least you still have something.
We could build acceptable redundancy into digital backups, its just that most people think of it as wasteful. You know what though?... I have everything worthy of backup "backed up" in at least 3 places, one of which is always CD stored somewhere out of reach. Digital is better. Once you convert to digital, you can have zero quality loss with near 100% efficiency, you just have to want it that bad.
In the wrong conditions, such as sunlight, humidity and upper surface damage, your CD-R will slowly turn into a coaster. "CD-Rs should never be left lying in sunlight as there's an element of light sensitivity, certainly in the poor quality media," says Stevenson. "I wouldn't rely on CD-Rs for long-term storage unless you're prepared to deal with them as recommended."
Surely storing cd's correctly is the key, if the dye on a cdr fades after being kept in a jewel case at a room temperature fr 2 years then that is obviously very bad (and there could be some lawsuits in the future).
Whether CDs last a long time or not is really missing the point. The benefit of going digital is that the data can be backed up.
If you're oriented on the media you're forever on the upgrade path. Should you move the collection to DVDs? But wait, blue light DVDs are right around the corner. It will never end.
120Gbyte hard disks are getting cheap. This trend will continue. What you store something on will literally become unimportant. The only important thing that will remain is still: how well is it backed up?
I still have perfectly working music CDs from the late 80s.
I have data CDs from the early 90s that are fine also
I just dug up some CD-Rs I burned from 1998 and they were fine also.
I think CDs can last a long time, but just like everything else...you need to take care of them. If it's something you use all the time, make backups and use those.
It's not time that kills CDs...it's scratches and wear.
"Some things have to be believed to be seen." - Ralph Hodgson
I think you are doing the right thing. Who has time to dick around backing up to CDs, tapes, etc? To me, any backup solution that spans multiple tapes, etc is severely broken.
I have a big honkin hard drive 120gig with all my stuff at home. I have a 2nd big honkin 120gig that has USB2. I take the USB2 drive to work once a month and leave it there. Bingo--off-site backup solution. (Yes, encrypted file system so co-workers can't browse my comprehensive porn collection.)
The stuff that changes more often (like photos) that I couldn't really bear to lose I rsync to my linux box over the net.
Everything fails, redundancy is the way to go. And it has to be easy.
slashsearch.org - slashdot search. powered by google.
I wouldn't even allow 100 per month, maybe 10 times a month. Or even completely offline altogether.
Better get some great insurance, I wouldn't want someone to have their 1850's relatives diary destroyed and then find out that I also lost their only digital copy!
Umm, of course the kids didn't magically know this.
But,
did YOU happen to know that the dye was toxic?
What happens when the amount time it takes to transfer all the data from one medium to another is longer than the life time of the media on which it currently resides?
Then obviously you couldn't have copied all the data to the "current" medium in the first place.
I'd like to add that with the current progress of storage technology there is really no need to copy CDs to other CDs every five years. Instead you'll probably want to copy CDs to DVDs to HD-DVDs to whatever in order to save physical storage space more often than every five years. Thus refreshing will happen automatically as long as you do not lose the media.
Losing and then finding media is of course the real problem as lost digital recordings do not get refreshed and may be destroyed.
The best is Mitsui Gold.
It is the same dye system that Kodak used in their Gold Ultima that is unfortunately no longer manufactured. Kodak licensed the technology from Mitsui.
Burning at slow speeds is still a good way to ensure more players can read a CD. This technique does work, and I used it again just last night. I burnt an SVCD of a *cough* movie I found umm, somewhere in a cupboard, at the rated speed of my medium which was 10x (rewritable). The DVD player rejected it, unable to read the data. I burnt it again from the same .bin file at 4x and the DVD player read it perfectly. It may not matter when you read it back on the same drive you burnt it on, but it sure can matter when you want to hear it in your car or watch it on your DVD player.
All those moments will be lost in time, like tears in rain.
What about offsite disaster recovery storage? I understand that disk is cheap, but I actually have more faith in a 2 year old CD (or DVD) in storage than a hard drive that's been in storage for two years.
;-) But 2-5 years is out.
As for archiving, where I work data needs to be kept for 7 years and then can be destroyed. If I could get the media to last for 7 years and then be unreadable, that would be ideal!!
Well, yes, but every so often, a format comes along that works, is cheap, and enjoys widespread use and support. CDs were the last one. Eventually, the multiplicity of DVD options will coalesce around one of them. After that, we can bitch about the next gen of data storage here on Slashdot.
Linux, you magnificent bastard, I read the fucking manual!
I absolutely agree.
CD-R's are for daily/frequent use.
For serious archiving, keep copies on multiple hard drives. Tools like
rsync make this very easy.
La via sola al paradiso incommincia nel inferno
Burning at slower than optimal speeds will result in more low level errors. Burning at lower speeds may produce a slightly greater phase change (or darker burn spots). However with more low level errors that disc will wear out faster. The type of dye used is usually a better indicator of lifespan. The dark blue dye lasts best. The lighter colored silver-green that is used by most cdrs is crap.
Informal testing shows that the silver-green dye lasts about 6 months in a hostile environment (namely the visor in my car- and I live in FL). The dark blue has lasted upwards of 6 years.
Just a Tuna in the Sea of Life
Just digitally remaster it, then record it on the best analog you can find. You get most of the quality, and all the durability...all you lose is the convenience, really.
Not quite. The difference is robustness.
The negative may have had a small crease, or off-color spot or three (i.e. "bit" decay), or even be torn in half, but the basic information was intact.
The problem is that for many electronic storage formats, copy fidelity is strong but robustness (tolerance to a few corrupt bits, eg. in the FAT, or a plain an simple crack) is low.
So what's a robust way of storing gigabytes, so that the corruption of a few makes a few "off-color" pixels but doesn't destroy the image overall? Give me a format that I can still read most of it, with no crucial weak spots (eg FATs) even if a few words are smudged or faded. That's why papyrus works.
So what if a CD doesn't last a hundred years? It's still a digital medium, which means that as long as there is one good copy, you can make an unlimited number of exact duplicates with no degradation in quality.
But another points is, why would you want to keep something on CD for a hundred years? You can't walk into a Walmart and buy a record player. 100 years is probably more than the lifespan of the medium, regardless of how long each disc is expected to last.
Punctanym: alternate spelling of words using punctuation or numerals in place of some or all of its letters; see 'leet'
Par, it works.
Link
The actual life of any specific item depends on many factors: manufacturing quality, manufacturing materials, storage, number of plays, etc.
So, while it is fair to say that "audio tape" is a relatively short-lived, fragile medium (based on the average "audio tape") it is not unusual to have tapes that last 30 years without noticable degradation. I've had tapes that didn't survive the first pass through the recorder, because they were made with crappy glue holding brittle magnetic bits. I have tapes that I've kept in a box for twenty years that are just fine.
You'll find the same thing for CD's. If you use good quality CD-Rs, and store them correctly, I have little doubt that 100 years is a reasonable expected lifetime.
And as others have already pointed out, if the recording is really important, make multiple copies, and then make new copies from the old before they degrade. In this case, CD-R has it over tape, because each generation of tape gets worse, while each generation of CD-R is identical to the parent.
Not true. The data could have been collected and recorded on the current media by multiple field sites, which may no longer exist, may have upgraded their recording equipment, or be too busy with current data collection projects to dupe media. You can easily end up with many thousands of tapes in a warehouse and insufficient equipment and time to copy them to new media before they rot. That's assuming you can get the funding for the work in the first place.
Mea navis aericumbens anguillis abundat
Not just the format of the file system (your example), but the format of the individual files. Does anyone believe that, outside of a handful of people in museums, anyone will be able to read GIF files in 100 years? Or MPEG-1 compressed video? Or documents stored as Microsoft Word 97 files? I've worked with computers for the past 25 years, and have encountered all of the problems that people have mentioned in this discussion: tapes for which there are no drives available, tapes and disks which degrade to the point that they are unreadable, file systems on disks that are not supported by contemporary OSs, and individual file formats for which no software (or specifications) exist. I also have a box filled with the paper copies of 25 years' worth of writing, and even the oldest are in good shape and WORK. If they were especially valuable, I'd make another paper copy and put it in the safe deposit box at the bank.
Audio and video are more difficult, since there's nothing as good as paper for them.
That's an engineering tradeoff driven by cost, not an inherent truth of information theory. A CD can be stamped from a glass master in an instant, while any drive ever built will take far longer to read it.
Exactly right. But when the value of the media (CD-Rs, HDDs, etc.) is less than the value of the data stored on the media, it's not hard to justify. This is the difference between bits and atoms. Atoms are mostly worthless -- bits can be invaluable.
Correct. With analog media, each succeeding copy represents a breakdown in quality of the data, but there is no difference between an digital original of data and its digital copy. After a digital copy is reliably made, it is just as accurate as the original, and can be used as such.
Whos to even say that the file format will still be supported long into the future as well.
Makes me wonder about thoses old Apple//e games I used to have on 5.25" discs.
I miss Hard Hat Mac!
Tape seems to tolerate degradation better because if a few magnatic "bits" flip, it won't make much difference, mostly just more background hiss. However, if a digital stream loses a bit or two, the current algorithms cannot recover very well.
What is needed is special encoding, reading, and algorithms that are more tolerant of degradation. However, it will probably take up more storage space, but that may be the tradeoff for longevity.
Tape tends to have redundancy in the lower frequencies, and this is partly why it seems less fragile. Perhaps something similar on the digital side can be done.
I notice that our VCR tapes are more kid-proof than DVD's. The kids play with both innappropriately, and the VCR tapes have about a 3-to-1 survivle rate over DVD's. I would have never guessed this without seeing it in action because VCR tapes have seemingly fragile moving parts and more parts. Go figure.
Table-ized A.I.
Unfortunately, paper doesn't last as well with holes in it, and it tends to develop holes as it degrades. If, on the other hand, you print out 2D bar codes, you could probably get it to last thousands of years. You might want to be sure to include documentation of the file format in a couple of languages, though.
"Hey, if you make 8-dot chunks of the dots on these pages, there is a 256-element field such that every 255 chunks, when considered as a polynomial and evaluated at a particular set of elements, gives zero for all of them, on every page." "Eh, it's probably just an incredible coincidence."
But... That implies there will always be somebody there to move the data from one medium to the next. What about the CD-R that gets found 75 years from now by your great great grandwhatever and nobody knows what the hell to do with it.
The human brain?
Shop as usual. And avoid panic buying.
Ah, I'll answer my own question.
The quick reference is not really a standalone document. It is the summary of the longer PDF that assumes you have already read the longer doc.
What the longer document says is that my cherry-Kool-Aid-smelling Sharpie would be in the "aromatic organic solvent" category that should not be used.
Alcohol substitutes are also a solvent that should probably not be used, but aren't nearly as bad.
The recommendation is for the use of water-based markers.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
Comment removed based on user account deletion
It is normal, and to be hoped for. Retaining historical knowledge doesn't cause us to stagnate. It gives us a base to build off. Can you imagine if every school year you had to start over because you didn't retain the information from the previous year? An extreme example, but I think it gets the point across. You stagnate when theres no growth, and there cant be growth if theres no history to grow from. Whether its printed media, or digital, the concepts the same.
...about 5 years ago. it bombed. there was also the superdisk. it bombed. now we have usb flashdrives with generic USB bulk storage drivers.
they may not be any longer lasting though - the only answer is archive and periodically read and rearchive to the latest storage medium.