Slashdot Mirror
CDs, DVDs Eyed For Long-Term Archival Use
Alien54 writes "Computer scientists at the National Institute of Standards and Technology (NIST) are launching an effort to develop specifications for 'archival quality' CD and DVD media that agencies could use to ensure the procurement of sufficiently robust media for their long-term archiving needs (i.e., 50 years and longer). See the press release at the NIST site." The research involves "...enclosed chambers that use temperature and humidity changes to artificially age the media some 20 years in only six weeks."
For this, WORM's have been invented. Currently at 9.2 GB per media, put larger versions are in the pipeline. They are still readable 10 years after, and have been guaranteed to be readable for 100 years, given the software exists.
Just you can't burn them with your run-of-the-mill software, you need some professional software for the whole document and jukebox managment as well, else you'll have some problems to find you archived data in a decade or so when the audit comes.
What I meant to say, is:
You write to whatever media is in vogue, and then periodically you read it back, checksum it to make sure it's not corrupted, then write it back again to a new media. Repeat ad nauseum.
Now imagine you've got to do this for whole applications and infrastructures to support those applications, and have them instantly viewable by the FDA at any point over a 25 year period.
This is what working in pharamceutical IS is like.
They tested 30 different brands of CD that had been recorded only 20 months earlier. Here is a picture of one of the CDs.
The red area can no longer be read.
This is pretty hideous.
I use CDs for archival storage.
It looks like it will become necessary to copy everything to new media every year or so, lest it become lost, forever lost, never to be seen again by the eys of mortal man.
It's very annoying.
Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
If you're a sysadmin, this problem has already been solved. RAIDed hard drives, always on, read occasionally to check for errors, and drives replaced as they fail. Replace the drives with new models every so often (or as they fail perhaps). Replace the controller and system it is attached to as necessary.
That is to say, that no digital storage that exists outside of a lab is suitable for long-term archival. Luckily, digital data being so easily copied (how easily people forget this!) makes this an easy problem to ignore. If you're developing new types of media, great. Otherwise, there is only one practical solution.
Yes, "my" solution - the solution used by anyone who has digital data they want to store long-term - requires someone to babysit the data. Sorry, most things in this world need some kind of human maintenance.
If you're storing the data on hard drives attached to a working computer, you can mirror the data on the other side of the world to protect it against any catastrophe that humans will survive.
If you don't care about a practical solution that has by far the highest chances of success, feel free to speculate about how long CDs will last based on completely invalid lab testing. (Accelerated aging? Hah! How can they possibly account for every variable?) If you truly care about your data, keep it online and make sure someone is around to maintain the system. If you want something less, it's because you don't actually care about the data that much.
one way to preserve data longer than 50 years is still microfilm. Guaranteed to las at least 100 years, if processed and stored correctly. Currently several companies offer the service to store 'digital dots' on microfilm, instead of typeface, improving the data-density considerably. Of course density is still waaaay below that of DVD or CD, but at least you're sure it'l be there in the future, and relatively easy to read (optical-scanning...)
Some things I've discovered so far are:
The biggest problems seems to be that the CDs come and go, so it can be difficult to get the tested products. The tests that has been done has used "accelerated aging", which is just a simulation. That is, there is no real experience in aging CDs.
My advice would be to store valuable information on as many different formats as possible. Continually monitor the quality of these, and transfer to new backups when they start to degrade too much.
Hope this helps!
It's almost 20 years ago since the first CD's came along.
Are they still working?
I have several CD's from 1985/86 that are still perfectly readable, even in secure mode with EAC (Exact Audio Copy).
I know this because i ripped my entire CD collection (about 2000 of them) last year and very few had errors, only 1 CD was unreadable.
The difficulty is that these disks cost a dollar or two each. Compare this with the el-cheapo ones that sell by the billions. Few mass consumers bought the good CD's, and Kodak stopped making them entirely and the Mitsui's are now specialty products that are not widely available.
There's also been a major shift in where and how CD-R's are manufactured. At first they were high-spec products, made in a few select factories in the US and Japan. Then manufacturing scaled up and cheaped out as the plants moved to Taiwan. Now a lot of those plants are going even lower-budget and moving to Mexico and mainland China.
The point is that consumers rarely buy for longevity... they go for neat packaging or cheap price or high burn speed or something else. The CD manufacturers have learned that lesson well. That's why it's so hard to buy good archival CD-R's anymore.
But CDs are an interesting case. You could argue that, unless we lapse into complete barbarism or some rejection of science, recovering old CDs should be possible for any future civilisation if the bit pattern is preserved. Provided the encoding and protocols are stored safely somewhere, it should be possible to construct a reader if anything is considered important enough to read. Unlike tape or punch card, the mechanical handling needed for a CD reader is very simple. Small lasers are made in ever greater volumes, and anything that replaces them is going to be more, not less capable. They use little power and there is no environmental reason why they are likely to fall into disuse.
Even so, my best photos are printed on archival grade non-resin coated acid free stock that should last a couple of hundred years. As if anyone is likely to care.
Panurge has posted for the last time. Thanks for the positive moderations.
The longest lasting picture medium is good old B&W silver halide film, printed onto silver halide paper. You can still buy the film and develop/print it yourself in a home darkroom. If done and stored correctly, they should be good for at least 150 years. A lot of the early film pictures are still around today.
The longest lasting color process, AFAIK, is pigment-based ink, such as the prints from an Epson 2200P inkjet printer (about $700). Printed onto archival matte paper, it should be good for about 70 years if stored in museum conditions (sealed behind glass, proper temperature, humidity, and not exposed to light).
C-prints (standard photo developing) and color negative film probably won't make it past 40 years without significant fading, and Cibachrome (the best option before pigment ink printers were available) isn't much better.
Why not archive on a high-quality film?
If documents are important to archive, especially for long periods of time, transferring the data to a less technical medium such as film is a much better alternative to the CD.
20 years ago, a great many people owned 8-Track tapes and players, along with record albums on vinyl. Very few of these items are readily available. I certainly know a few people with turn tables, but no one who owns an 8-Track player.
Whos is going to gurantee that the technology in use in 2103 can read a CD created in 2003? By storing data on film, even as a series of light/dark bits, requires very little technology for retrieval. Think about that, a lamp, a lense, and a wall to view an image. Data encoded as a string of bits could easily be read into a recording device.
Many types of film can be stored for much longe periods than CD's, and can be easily copied and in some cases restored.
Why does no one take a Tyrant like approach to this problem?
One common criticism of the use of vellum is that of animal cruelty, although it's worth noting that the goatskin used is a by-product of the leather industry, and comes from goats that had already been slaughtered to make, say, shoes.
I was glad to hear that this latest attempt at pointless `modernisation' (for the sake of appearing `modern' rather than for any deeper and more sensible reasons) was defeated. Not least because it's really cool to be able to say ``This is actually an Act of Parliament from the 16th century, and that's actually Henry VIII's signature - not a photocopy, not a JPEG, but the real thing.''
A related matter concerns the increasing prevalence of digital photography. As this BBC News article explains, digital photography may cause problems for future generations of local or family historians. Proper (printed) photographs tend to get stored away in shoeboxes in attics, and are still more-or-less as legible after a hundred years as they were when they were taken. Whereas an entire collection of digital photographs can be wiped out by one hard-disk failure.
Or maybe in fifty years' time nobody'll know how to display a JPEG. Somewhere I've got a tape with the very first program I wrote, recorded on it. It was a simple bullseye game for the Video Genie II (a TRS80 clone) - it wasn't particularly sophisticated by general standards, but not at all bad for an eight-year-old kid. I have no means of retrieving it on any of the small menagerie of computers currently in my house.
Another, related, example: in the mid-1980s a (for the time, pretty damned impressive) multimedia project was launched - the Domesday Project. This was a laserdisc containing digital reproductions of the original 11th-century Domesday Book itself (a census survey of the entirety of England ordered by William I) together with (I think) the 1981 national census data. All very innovative (albeit rather a costly system) but ironically, 15 years on, the laserdiscs are not readable by current technology - but the original 900-odd-year-old Domesday Book itself still is.
I guess the point is that it's all very well saying ``this media is guaranteed to last for fifty years'' (although personally I'd be happier with something that'll last several hundred) but also you have to guarantee that the data format itself is going to remain readily decodeable. This is not a problem for 1000-year-old documents on vellum (as long as your Latin or Norman French is ok, and you've done a basic course in paleography).
nicholas
The problem :
:
A paper based collection of 100,000+ maps dating back to 1886 that are slowly decaying with use. They require digitisation for long term ( 200 years +, at least as long again as they have survived already ) archiving but need to be available quickly and easily for viewing on demand. Total storage requirement is in excess of 150 Tb.
The Solution
An archiving setup using Magneto Optical technology in managed jukeboxes in a controlled environment. MO has been around for nearly 20 years now and is a highly refined and proven technology. Current capacities are up to 9.1 Gb per media item with 30Gb coming online in the near future. Jukeboxes handling up to 10Tb per unit are readily available now.
MO doesnt use dye at all. The laser melts a magnetic substrate that is then manipulated by the write head to impart the data in a similar way to a conventional disk, the sustrate cools and the data is permanantly stored. There is no degradation of the media by sunlight, heat etc as compared to DVD or CD formats. It's more accessable and requires less management than tape, its cheaper than conventional disk, off site storage of duplicate media is easily achieved, data throughputs are faster then DVD or CD and capacity is as good or better than either.
The idea that name brand does not imply quality in the CD-R market is quite correct. There are only something like 14 companies in the world that make CD-Rs but there are hundreds of brands out there that buy and then resell them under their own name.
So therefore it's important to know which company manufactured the disc as opposed to which is selling it. It's also important to know that some name brands buy from more than one manufacturer. For example, my 'archival' burning is done on FujiFilm discs made by Taiyo-Yuden, one of the best manufacturers. But I have to be careful because Fuji also sells Ricoh media. Typically the "Made in Japan" mark identifies Fujis that are from Taiyo-Yuden. This is how I find the quality.
So how do you tell if some disc on the shelf is good or not? Buy a single disc, bring it home and use an ATIP-reader utility to find the manufacturer name. If it's a good manufacturer (i.e. a manufacturer you have carefully researched and is known to make very good CD-Rs) then go back and buy a truckload. Otherwise keep searching.