Slashdot Mirror
CDs, DVDs Eyed For Long-Term Archival Use
Alien54 writes "Computer scientists at the National Institute of Standards and Technology (NIST) are launching an effort to develop specifications for 'archival quality' CD and DVD media that agencies could use to ensure the procurement of sufficiently robust media for their long-term archiving needs (i.e., 50 years and longer). See the press release at the NIST site." The research involves "...enclosed chambers that use temperature and humidity changes to artificially age the media some 20 years in only six weeks."
For this, WORM's have been invented. Currently at 9.2 GB per media, put larger versions are in the pipeline. They are still readable 10 years after, and have been guaranteed to be readable for 100 years, given the software exists.
Just you can't burn them with your run-of-the-mill software, you need some professional software for the whole document and jukebox managment as well, else you'll have some problems to find you archived data in a decade or so when the audit comes.
I remember reading about guy who's profession was accessing legacy media. He had all this old legacy equipment which he had accumulated over the years. Companies deciding to move data over to newer media often had to contact him waving their 12" disks and tape cartridges asking for access as the old computers along with the readers had long been discarded.
There's a number of CDs which have already experienced 10 years of mistreatements, I wonder if any mass producing company has already learned something valuable and if they modified their production accordingly. Polycarbonate-eating fungi were already mentioned here and on Nature as well. Add the aluminum layer oxidation problem and my trust on cd-r as long term storage is reaching zero. I also own a couple of 10+ years old CDs (original, shush RIAA) that don't show any surface problem, but no player I have tried can play it anymore.
What I meant to say, is:
You write to whatever media is in vogue, and then periodically you read it back, checksum it to make sure it's not corrupted, then write it back again to a new media. Repeat ad nauseum.
Now imagine you've got to do this for whole applications and infrastructures to support those applications, and have them instantly viewable by the FDA at any point over a 25 year period.
This is what working in pharamceutical IS is like.
They tested 30 different brands of CD that had been recorded only 20 months earlier. Here is a picture of one of the CDs.
The red area can no longer be read.
This is pretty hideous.
I use CDs for archival storage.
It looks like it will become necessary to copy everything to new media every year or so, lest it become lost, forever lost, never to be seen again by the eys of mortal man.
It's very annoying.
Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
Slashdot had an article on the subject some time ago (original salon article).
bundaegi is good for you
If you're a sysadmin, this problem has already been solved. RAIDed hard drives, always on, read occasionally to check for errors, and drives replaced as they fail. Replace the drives with new models every so often (or as they fail perhaps). Replace the controller and system it is attached to as necessary.
That is to say, that no digital storage that exists outside of a lab is suitable for long-term archival. Luckily, digital data being so easily copied (how easily people forget this!) makes this an easy problem to ignore. If you're developing new types of media, great. Otherwise, there is only one practical solution.
Yes, "my" solution - the solution used by anyone who has digital data they want to store long-term - requires someone to babysit the data. Sorry, most things in this world need some kind of human maintenance.
If you're storing the data on hard drives attached to a working computer, you can mirror the data on the other side of the world to protect it against any catastrophe that humans will survive.
If you don't care about a practical solution that has by far the highest chances of success, feel free to speculate about how long CDs will last based on completely invalid lab testing. (Accelerated aging? Hah! How can they possibly account for every variable?) If you truly care about your data, keep it online and make sure someone is around to maintain the system. If you want something less, it's because you don't actually care about the data that much.
Kodak did some accelerated longevity tests on CD-Rs and found that many disks degraded rapidly when exposed to sunlight, due to the UV components of sunlight.
Mea navis aericumbens anguillis abundat
Magnetic tapes leak, don't they? I have a pack of cassettes recorded with old stuff I wrote for a C64, 20-odd years ago, and even ten years ago they were already unreadable. This was not even high-density recording, just normal screeching. My understanding is that each layer of tape has a small effect on its neighbours, and after some time the entire tape is reprogrammed to noise. Presmably if the tapes are played and rewound the effect is less dramatic.
But magnetic tapes do not strike me as particularly stable. Hard disks may be more stable than tapes.
Ceci n'est pas une signature
one way to preserve data longer than 50 years is still microfilm. Guaranteed to las at least 100 years, if processed and stored correctly. Currently several companies offer the service to store 'digital dots' on microfilm, instead of typeface, improving the data-density considerably. Of course density is still waaaay below that of DVD or CD, but at least you're sure it'l be there in the future, and relatively easy to read (optical-scanning...)
Some things I've discovered so far are:
The biggest problems seems to be that the CDs come and go, so it can be difficult to get the tested products. The tests that has been done has used "accelerated aging", which is just a simulation. That is, there is no real experience in aging CDs.
My advice would be to store valuable information on as many different formats as possible. Continually monitor the quality of these, and transfer to new backups when they start to degrade too much.
Hope this helps!
It's almost 20 years ago since the first CD's came along.
Are they still working?
I have several CD's from 1985/86 that are still perfectly readable, even in secure mode with EAC (Exact Audio Copy).
I know this because i ripped my entire CD collection (about 2000 of them) last year and very few had errors, only 1 CD was unreadable.
The difficulty is that these disks cost a dollar or two each. Compare this with the el-cheapo ones that sell by the billions. Few mass consumers bought the good CD's, and Kodak stopped making them entirely and the Mitsui's are now specialty products that are not widely available.
There's also been a major shift in where and how CD-R's are manufactured. At first they were high-spec products, made in a few select factories in the US and Japan. Then manufacturing scaled up and cheaped out as the plants moved to Taiwan. Now a lot of those plants are going even lower-budget and moving to Mexico and mainland China.
The point is that consumers rarely buy for longevity... they go for neat packaging or cheap price or high burn speed or something else. The CD manufacturers have learned that lesson well. That's why it's so hard to buy good archival CD-R's anymore.
But CDs are an interesting case. You could argue that, unless we lapse into complete barbarism or some rejection of science, recovering old CDs should be possible for any future civilisation if the bit pattern is preserved. Provided the encoding and protocols are stored safely somewhere, it should be possible to construct a reader if anything is considered important enough to read. Unlike tape or punch card, the mechanical handling needed for a CD reader is very simple. Small lasers are made in ever greater volumes, and anything that replaces them is going to be more, not less capable. They use little power and there is no environmental reason why they are likely to fall into disuse.
Even so, my best photos are printed on archival grade non-resin coated acid free stock that should last a couple of hundred years. As if anyone is likely to care.
Panurge has posted for the last time. Thanks for the positive moderations.
The longest lasting picture medium is good old B&W silver halide film, printed onto silver halide paper. You can still buy the film and develop/print it yourself in a home darkroom. If done and stored correctly, they should be good for at least 150 years. A lot of the early film pictures are still around today.
The longest lasting color process, AFAIK, is pigment-based ink, such as the prints from an Epson 2200P inkjet printer (about $700). Printed onto archival matte paper, it should be good for about 70 years if stored in museum conditions (sealed behind glass, proper temperature, humidity, and not exposed to light).
C-prints (standard photo developing) and color negative film probably won't make it past 40 years without significant fading, and Cibachrome (the best option before pigment ink printers were available) isn't much better.
Why not archive on a high-quality film?
If documents are important to archive, especially for long periods of time, transferring the data to a less technical medium such as film is a much better alternative to the CD.
20 years ago, a great many people owned 8-Track tapes and players, along with record albums on vinyl. Very few of these items are readily available. I certainly know a few people with turn tables, but no one who owns an 8-Track player.
Whos is going to gurantee that the technology in use in 2103 can read a CD created in 2003? By storing data on film, even as a series of light/dark bits, requires very little technology for retrieval. Think about that, a lamp, a lense, and a wall to view an image. Data encoded as a string of bits could easily be read into a recording device.
Many types of film can be stored for much longe periods than CD's, and can be easily copied and in some cases restored.
Why does no one take a Tyrant like approach to this problem?
Actually, a glass master is a physical object... it's a glass disc etched with the negative "cast," which is pressed into the polycarbonate layer to form the pits and lands of the CD during manufacturing.
A glass master is very fragile, fairly expensive to make (~$500 depending on the pressing plant), and obviously won't play in your CD player. ;-)
One common criticism of the use of vellum is that of animal cruelty, although it's worth noting that the goatskin used is a by-product of the leather industry, and comes from goats that had already been slaughtered to make, say, shoes.
I was glad to hear that this latest attempt at pointless `modernisation' (for the sake of appearing `modern' rather than for any deeper and more sensible reasons) was defeated. Not least because it's really cool to be able to say ``This is actually an Act of Parliament from the 16th century, and that's actually Henry VIII's signature - not a photocopy, not a JPEG, but the real thing.''
A related matter concerns the increasing prevalence of digital photography. As this BBC News article explains, digital photography may cause problems for future generations of local or family historians. Proper (printed) photographs tend to get stored away in shoeboxes in attics, and are still more-or-less as legible after a hundred years as they were when they were taken. Whereas an entire collection of digital photographs can be wiped out by one hard-disk failure.
Or maybe in fifty years' time nobody'll know how to display a JPEG. Somewhere I've got a tape with the very first program I wrote, recorded on it. It was a simple bullseye game for the Video Genie II (a TRS80 clone) - it wasn't particularly sophisticated by general standards, but not at all bad for an eight-year-old kid. I have no means of retrieving it on any of the small menagerie of computers currently in my house.
Another, related, example: in the mid-1980s a (for the time, pretty damned impressive) multimedia project was launched - the Domesday Project. This was a laserdisc containing digital reproductions of the original 11th-century Domesday Book itself (a census survey of the entirety of England ordered by William I) together with (I think) the 1981 national census data. All very innovative (albeit rather a costly system) but ironically, 15 years on, the laserdiscs are not readable by current technology - but the original 900-odd-year-old Domesday Book itself still is.
I guess the point is that it's all very well saying ``this media is guaranteed to last for fifty years'' (although personally I'd be happier with something that'll last several hundred) but also you have to guarantee that the data format itself is going to remain readily decodeable. This is not a problem for 1000-year-old documents on vellum (as long as your Latin or Norman French is ok, and you've done a basic course in paleography).
nicholas
Once I accumulated 5 years worth of backups, I copy the expiring data to new media and throw the old away, this gives me a little protection from 1) aging media and 2) aging i/o devices. I also make sure that as I get rid of older devices, that I convert all backups it might use before getting rid of the device.
Simple, but it works well for me.
The problem :
:
A paper based collection of 100,000+ maps dating back to 1886 that are slowly decaying with use. They require digitisation for long term ( 200 years +, at least as long again as they have survived already ) archiving but need to be available quickly and easily for viewing on demand. Total storage requirement is in excess of 150 Tb.
The Solution
An archiving setup using Magneto Optical technology in managed jukeboxes in a controlled environment. MO has been around for nearly 20 years now and is a highly refined and proven technology. Current capacities are up to 9.1 Gb per media item with 30Gb coming online in the near future. Jukeboxes handling up to 10Tb per unit are readily available now.
MO doesnt use dye at all. The laser melts a magnetic substrate that is then manipulated by the write head to impart the data in a similar way to a conventional disk, the sustrate cools and the data is permanantly stored. There is no degradation of the media by sunlight, heat etc as compared to DVD or CD formats. It's more accessable and requires less management than tape, its cheaper than conventional disk, off site storage of duplicate media is easily achieved, data throughputs are faster then DVD or CD and capacity is as good or better than either.
The MAM-E (former Mitsui) Gold Archive and Verbatim Datalife Plus are the best CD/DVD blanks you can get in terms of longevity. In general, these brands along with TDK are what I recommend to folks almost exclusively because they have the best quality control. I've heard good things about Taiyo-Yuden and Mitsubishi's high-end blanks as well. Sadly, Kodak Gold blanks are no longer made, but if you can find some, go nuts on them.
Longevity is the last thing that people are thinking about. That's why crap blanks like Princo and Memorex are usually the best deal price-wise. Whether it's your music collection or home videos, you can't afford the cheap blanks. Of course, I also recommend many of the same things that the original poster (environmentally controlled environment isolated from light). However, using CDs with less storage space really doesn't make a big difference if you've temperature controlled them. The size of the "bits" is not going to be the deciding factor when there's already in-line error correction and redundancy. Just use good archival blanks.
One last note...for the truly paranoid, you should actually go and get a CD mastered by a manufacturing house that specializes in this type of thing. This may not be the cheapest option (probably on the order of hundreds of dollars), but what's your data really worth if you lose it and can't get it back?
These pits aren't likely to degrade for some time.
In contrast, any current CD-R/RW of which I am aware stores information photochemically; i.e., the information is stored in a dye that changes state due to the application of light from a laser beam.
These chemicals are likely to degrade much more quickly than are physical pits.
I don't believe that this situation will change any time soon.
Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
The idea that name brand does not imply quality in the CD-R market is quite correct. There are only something like 14 companies in the world that make CD-Rs but there are hundreds of brands out there that buy and then resell them under their own name.
So therefore it's important to know which company manufactured the disc as opposed to which is selling it. It's also important to know that some name brands buy from more than one manufacturer. For example, my 'archival' burning is done on FujiFilm discs made by Taiyo-Yuden, one of the best manufacturers. But I have to be careful because Fuji also sells Ricoh media. Typically the "Made in Japan" mark identifies Fujis that are from Taiyo-Yuden. This is how I find the quality.
So how do you tell if some disc on the shelf is good or not? Buy a single disc, bring it home and use an ATIP-reader utility to find the manufacturer name. If it's a good manufacturer (i.e. a manufacturer you have carefully researched and is known to make very good CD-Rs) then go back and buy a truckload. Otherwise keep searching.
I think you are really heading in the wrong direction with this. Instead of expecting everything to be destroyed by the elements, and adding in more redundancy to (do a poor job) compensate, you just need to make the housing material much more durable, so it can sheild the data from damage.
Stick a few DVDs in a thouroughly padded steel case, then drop it down a mountain, and you won't see any damage done. If it's padded well enough, you could throw it off a cliff, and not damage the DVDs even slightly. Waterproofing would be necessary, but you would get data lasting much longer, and in a much more effecient form.
Why make it so a DVD can be read after being shattered, why you can just prevent it from being shattered in the first place?
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Sounds like a perfectly normal SNOBOL deck. (Each card does in fact end with the next card in the program, the sequence number.)