Slashdot Mirror
How To Properly Archive Data On Disc Media
An anonymous reader writes "Patrick McFarland, the well-known Free Software Magazine author, goes into great detail on CD/DVD media over at the Ad Terras Per Aspera site. McFarland covers the history of the media, from CDs through recordable DVDs, explaining the various formats and their strengths and drawbacks. The heart of the article is an essay on the DVD-R vs. DVD+R recording standards, leading to McFarland's recommendation for which media he buys for archival storage. Spoiler: it's Taiyo Yuden DVD+R all the way. From the article: 'Unlike pressed CDs/DVDs, burnt CDs/DVDs can eventually fade, due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media).'"
This is practically a word-for-word dupe of a /. posting from December 11th 2006
Spelling mistakes, grammatical errors, and stupid comments are intentional.
"From the article: 'Unlike pressed CDs/DVDs, burnt CDs/DVDs can eventually fade, due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media).'"
Great! Your porn will outlast you.
I'm sure it's just a coincidence that we have to articles posted by anonymous readers linking to "famous author" M*Farland, but it struck me as odd. Especially since he commented on the USB story. Could there little astroturfing going on?
It's not like we haven't seen that before ( Roland P* )
I know who he is. I would've never called him a "famous free software writer" as he was labeled earlier today, and if he's "well-known", it's not for being a writer. The way these summaries are worded, as well as the fact that both stories today were submitted by "an anonymous user", just makes me think that somebody is looking to boost their site's traffic today. Nevermind the fact that the article is old and has already been linked to on slashdot before.
Anyway, just seemed fishy to me. That's my $0.02.
It's just an archive hosted from a long-lasting Taiyo Yuden DVD+R.
You can hold down the "B" button for continuous firing.
Don't archive with anything made from protons, as it is theorized that they decay even when stored in a cool, dry vacuum.
A 500GB hard drive is like $150 these days and 1TB drives are just around the corner. Drop one into a $10 USB enclosure and backup your stuff using rsync. To do it right, do this twice with 2 different drives and store them in 2 different physical locations. I don't care what fancy pants brand of DVD-R you use, a magnetic solution is still superior in both durability and simplicity (try backing up 100GB of data using a DVD writer and a hard drive, then tell me which one took longer).
This just brings up the question:
Is there a point to digitizing human cultural pieces that has survived for 10,000 years onto media that will fail in 10?
Heh...Heh...That was cool
So... you just look at the pictures on slashdot?
Well.. maybe. Or Maybe not. But Definitely not sort of.
For achival DVD-RAMs are best suited, because they are purely chemical and supposed to last up to 30 years. It's a little slower to write than DVD+RW, but data correction and verification is way better. The UFS filesystem also retains most Unix/Linux file attributes. And while a DVD-RAM is a little more expensive, you generally only need a few of them for home backups. (Besides the obvious reuse advantage.)
I prefer cylinders. If they were good enough for grandpa, they're good enough for me.
Note: With I can gather from the name of the site, it appears they might also sell hats, but I couldn't find a link.
What?
And since the optical media is so cheap, make multiple copies of each disc, with data shuffled around on the surface on each copy.
That way, any "spot defects" will be very unlikely to hit the same data on every copy. Making the whole redundant backup set last many times longer.
--
make install -not war
A quick primer: you can "error scan" DVD+/-R media with a drive that supports it. CDSpeed, a [free, IIRC] utility distributed by/with Nero, can easily save these scans. DVD enthusiasts often compare their scans... cdfreaks.com is a great discussion site.
o stcount=294
Some media have been observed to degrade fairly rapidly, others are quite stable. About a year ago, and again recently, I scanned a number of relatively old DVD-R discs [backups, uh, owned by a friend] burned from fall of 2002 on. You can see my post here:
http://club.cdfreaks.com/showpost.php?p=1733269&p
Funny thing is that most of the discs I used were of a brand widely lambasted as "cheap ____" and I was told that they wouldn't last six months. Curiously enough you can see that the cheaper "Princo" media has held up better than the "gold standard of the day" Riteks [although both are much better than some]. You can also see that one of older discs was scanned recently, and more than a year ago. It shows almost no degradation during that time [and what it does could easily be attributed to the aging scanning drive].
The CDFreaks forum has a lot more scans, including of older media. If you've got some discs and are worried about their aging stability, here's a good place to start:
http://club.cdfreaks.com/forumdisplay.php?f=33
april fools. i wonder how many people didnt pick up on this. you do know it is april first somewhere right ;)
The war with islam is a war on the beast
The war on terror is a war for peace
My theory is that it's an April Fools joke one day early. I mean, calling him "famous" and "well-known" has got to be a joke, right?
Mods, can I get an Insightful, Interesting, or Informative here? :)
I've had hard drives fried by failing power supplies. Sometimes you get lucky and replacing the electronics from an identical drive works, sometimes it doesn't. I've never heard of a CD or DVD drive's laser suddenly burning holes in the disc.
Ditto with mechanical shock -- a DVD will survive a lot rougher handling than a harddrive will, even if the latter's heads are parked.
There are always trade-offs.
-- Alastair
I started burning CDs when a burner was $500.00 and only available as a SCSI drive. This is one rare times I'm glad I was on the bleeding edge. Back then you had to know what gold/gold , blue/gold , blue/blue. Was it made in Japan or Taiwan. Turn your screen saver off. Buffer under run protection did not exist. A 4x burner was fast.
Now people complain if you can't burn faster than 24x. Ask them what color their media was and you'll get something like, "Its magenta with Sony written across it." People just don't learn the basics anymore.
If you burn and it fails in a few years, then you need to: gold/gold has the longest lifetime, silver/silver is the most compatible, made in Japan, keep it out of sunlight, burn at 8x or less if you really want it to last.
A friend and a very competent sysadmin said to me (maybe 5 years ago) "tape sucks".
He told me in many ways, why any kind of disk archive was better than tape. I've thought about that conversation many times.
My conclusion FWIW, is the same as everyone knew half a century ago. Accessibility-density-longevity... pick = two.
Are there any good tools for testing DVD and CD discs in Linux?
Magneto-Optical media is used by medical facilities where archival time length is paramount. Such as:
- http://md5.ca/~pavel/md.jpg
- http://tinyurl.com/2cu7zv
MO drives are a bit costly, but if you have important media its worth it. Besides cool look for neo's warez stash in Simulcara book. Quoted guaranteed archival time is over 40 years in most cases, and they continually improving the technology, compared to driving the costs down of the generic blank media market of CDs/DVDs.
Other tips for longativity:
- Use acid free CD-markers when marking
- Do not use as a coaster like your America Online CDs
- Do not play Frisbee outside with your CDs (Exposes to direct sunlight)
- Chewing reduces durability
Once you start despising the jerks, you become one.
... Yuden raus!
The article admits that Taiyo Yuden's formulation used to be the best, until they improved all of the others!
We're to use Taiyo Yuden's "Super Cyanine" "stabilized" Cyanine dye, because for a short time, it was the best dye. But then the article goes on to admit that TDK' "metal-stabilized Cyanine" is rated for the same shelf life. Then it goes on to admit that the "Metal Azo dye" I see listed on the brands I've been buying is rated for a longer shelf life than either of the previously mentioned ones, but then claims it's not better. OK, but you didn't claim it was worse either!
So basically, this guy took an obselete article that said that some obscure brand makes longer lasting DVDs, and as the other major companies have caught up or even exceeded Taiyo Yuden for longevity, he did not rewrite the conclusion.
I challenge anyone to find any support in that article for it's conclusion. What's the reason for prefering Taiyo Yuden over Mitsubishi? I don't see one! Over TDK? I don't see one.
And, I believe that I'm seeing the Mitsubishi formulation sold by Sony.
I wonder if choosing longer lasting dyes is the reason that I'm seeing 2x speed Verbatim double layer disks instead of the 8x double layer disks I used to buy?
Check this out for all the tactile comfort of cylinders, with the convenience of digital - paper tape is where it is at!
This issue is a bit more complicated than you think.
... was available other than in Japan. I think there is an importer of 12X media now though. Though DVD-RAM can't be used in regulated environments that specify true WORM media, for other (small, SOHO) data backup chores, it would be the thing to have (with 12X media). NewEgg has 12X DVD-RAM drives for under $40. Personally, I think they are purposely keeping DVD-RAM media away so as not to suppress Blu Ray adoption. Wikipedia has a pretty good writeup on DVD-RAM. The history of optical media for computer data storage is indeed a fiasco. A severe lack of leadership by the industry IMO.
External HDD: 1000GB for $298 = $0.298/GB
Internal HDD: 320GB for $80 = $0.25/GB
DVD+R: 470GB for $26 = $0.055/GB
It's also a lot easier to lend someone else a DVD than a hard drive, even external ones. Especially if your data really is naturally divisible into smallish chunks.
No matter what format you choose create error correction data.
Analogies don't equal equalities, they are merely somewhat analogous.
Sorry, but if you have to select media and burner based on some black magic, then the technology is entirely unfit to be used for any archiving. Archival media give you ensurances like 30 years data lifetime in any combination. You can already get that with MOD. MODs have >50 years data lifetime and drive makers will allways support the lasdt three media generations (currently they are supporting all, i.e. the last fife generations and that wor reading and writing). The only DVD technology that is somewhat comparable is DVD-RAM with cartridge. But that seems to have zero market share in the computer business.
My conclusion is that either people do not care about their archived data or that the number of, e.g., lost baby- and wedding-photographs is not high enough yet.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Here is an excellent article on the true cost of archiving to CD/DVD. I have grown to distrust optical media more and more. The only CDs I burn now are KNOPPIX discs, OS install discs, etc. Anything important I archive to a hard drive. I have an external SATA enclosure that has a removable tray.
Asfar as i see dvd ram is not discussed
Doesn't the ISO9660 format already have a certain amount of redundancy in it?
Not saying that using PAR2 to create some additional redundancy is a bad idea, but I don't think that one-bit errors are totally disastrous automatically, if you've burned your data in a normal format.
Not sure what UDF does, though.
Although it's not as if PAR2 is a particularly exotic format, just in case it drops out of common use in the future, I think I'd still put a plaintext copy of the source code for the reassembler on every backup volume, or at least on the first volume in each set, just to make life potentially easier down the road. (And even if the format doesn't drop out of use, having the backups be self-contained in terms of containing the software necessary to decompress/assemble themselves would let someone use them even if they didn't have network access, which might be handy.)
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
What a MOD?..... Link Please!
Magneto Optical Disc Probably the cheapest comsumer priced devices are Sonys HIMD which are pc compatible, although even these are getting hard to find too. People dont like good technology it seems.
Escher was the first MC and Giger invented the HR department.
I ask myself this a lot: how reliable are HDDs in the long term?
Say you use them for archiving. It is better to go the NAS route and have them always on, or powered down when not in use? How about a USB enclosure only connected when in use? How reliable is an almost-new used-it-once HDD that has been sitting on a shelf for five years?
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
whilst the data density of stone tablets is quite low, if carefully stored they're pretty much readable indefinitely!
develop the disk platters with MagnaSee (Reeves, 1956 product) and impress the platters in potter's clay or slip clay. then fire the clay, and you have a permanent archival copy.
I can't reply to this april fools stuff any more, my head hurts.
if this is supposed to be a new economy, how come they still want my old fashioned money?
you see, SOMEthing corrupted some jpgs at some point between 5 years ago and now. the fact that i was backing up my entire digital camera photo repository to another harddrive didn't help. the corrupted jpgs were backed up as well.
so you see, having an optical burn does indeed help. his point about the data not being overwriteable is completely valid. another harddrive isn't good enough.
-Clio
Karma: Bad (mostly from not giving a fuck)
Blog: http://clintjcl.wordpress.com
I still prefer SCSI LTO tape for archival data. Back up often to HD, then dump to tape every once in a while and bury the tapes in a safe place.
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
they just happen to sell a superior solution. For offsite archiving, I prefer something I can slip into an envelope and mail cross-country without being concerned about it being dropped or subject to magnetic fields.
For workstation-size data sets, I use a drive mirror for short-term and DVD+R for long term... and looking forward to terabyte recordable in 5.25" form factor which should be available in a few years.
Tech Public Policy stuff
Also, the error correction/detection is not dependent on having any particular file system on the medium. Any data CD (regardless of file system) has the data correction built-in.
You can do without it in the case of CDs and DVDs. It's just that the media and burning equipment would generally have to be of such high quality that nobody would be able to afford it. Think of it at as a cheper alternative to a perfect manufacturing process.
HAND.
MOD = Magneto Optical Disk.
s torage/mo/
Link e.g. here: http://www.fujitsu.com/global/services/computing/
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.