How To Properly Archive Data On Disc Media

An anonymous reader writes "Patrick McFarland, the well-known Free Software Magazine author, goes into great detail on CD/DVD media over at the Ad Terras Per Aspera site. McFarland covers the history of the media, from CDs through recordable DVDs, explaining the various formats and their strengths and drawbacks. The heart of the article is an essay on the DVD-R vs. DVD+R recording standards, leading to McFarland's recommendation for which media he buys for archival storage. Spoiler: it's Taiyo Yuden DVD+R all the way. From the article: 'Unlike pressed CDs/DVDs, burnt CDs/DVDs can eventually fade, due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media).'"

Comments

Dupe...

[LighterShadeOfBlack]

This is practically a word-for-word dupe of a /. posting from December 11th 2006

Re:Dupe...

[LighterShadeOfBlack]

It is a good article on the pros and cons of DVD-R vs DVD+R in general, however I'm not too happy with the way the author recommends one particular brand over others like that without any hard data to back it up. It's quite possible it's absolutely true and they really are a superior brand of discs, but without presenting any numbers to support his assertion it will remain nohing more than one person's opinion.

Re:Dupe...

[Dogtanian]

I'm not too happy with the way the author recommends one particular brand over others like that without any hard data to back it up. To be fair, whilst I'm not claiming that I've never come across criticism of Taiyo Yuden, they seem to be consistently ranked #1 in reliability and quality. In fact, come to think of it I can't recall seeing any reviews where their media (overall) weren't the top rated.

I believe that Verbatim (owned by Mitsubishi) are also very highly rated (except for a brief period in 2002 when they switched to a far less reputable media supplier). More info in this article.

Bear in mind that a *large* number of major brands don't make their own media; these include companies such as Memorex, Fuji and Emtec (formerly BASF). Apparently you can look at the cakebox/packaging style and media type and figure out if it's good-quality rebranded media inside, but I don't buy enough discs for this to be worth my time; more sensible just to buy from companies that do their own.

M*Farland?

[MDMurphy]

I'm sure it's just a coincidence that we have to articles posted by anonymous readers linking to "famous author" M*Farland, but it struck me as odd. Especially since he commented on the USB story. Could there little astroturfing going on?

It's not like we haven't seen that before ( Roland P* )

Patrick McFarland?

[MysticOne]

I know who he is. I would've never called him a "famous free software writer" as he was labeled earlier today, and if he's "well-known", it's not for being a writer. The way these summaries are worded, as well as the fact that both stories today were submitted by "an anonymous user", just makes me think that somebody is looking to boost their site's traffic today. Nevermind the fact that the article is old and has already been linked to on slashdot before.

Anyway, just seemed fishy to me. That's my $0.02.

This isn't a dupe.

[game kid]

It's just an archive hosted from a long-lasting Taiyo Yuden DVD+R.

Why bother with optical?

[Anonymous Coward]

A 500GB hard drive is like $150 these days and 1TB drives are just around the corner. Drop one into a $10 USB enclosure and backup your stuff using rsync. To do it right, do this twice with 2 different drives and store them in 2 different physical locations. I don't care what fancy pants brand of DVD-R you use, a magnetic solution is still superior in both durability and simplicity (try backing up 100GB of data using a DVD writer and a hard drive, then tell me which one took longer).

Re:Why bother with optical?

[Hatta]

At $150 a 500gb drive is $.30/GB. At american-digital.com (my personal favorite place for bulk media), I can get 16x Taiyo Yuden dvd+r media for about $.60/disc; which, at 4.7GB/disc, works out to $.128/GB. So hard drives are more than twice as expensive per gigabyte as DVD+R.

Re:Why bother with optical?

[Anonymous Coward]

I think you're really making their point. Optical media just isn't cheap enough to bother with it anymore, and with the prices of magnetic storage always decreasing, there's even less of a reason to use optical media.

Yes, you'd save a few bucks initially by using optical media. But then you have to split all your data in evenly sized 4.7GB archives or such. Label them all manually. Waste hours swapping hundreds of discs by hand. Catalog all these discs (number them, keep a database or something0. And they take LOTS of physical place to store.

I got tired of looking for a specific DVD. After an hour of flipping thru pages of those (expensive and large) CD wallets and not finding it, I gave in, and bought several TB worth of HD space. Now if I want a movie, it's there, listed alphabetically and all. Jewel cases suck too -- too brittle, wastes space too, and a waste of money.

HDs have a very high density (the new 1TB drives will store more than a spindle of 200 DVDs), requires no storage cases, no constant media swapping, no splitting to fit the size of media, etc. It's all-around better! Restoring stuff is almost instantaneous. You never have to look for a specific disc. Transfer speeds are great. Everything is sorted alphabetically inside folders/directories... What more could you ask for?

Want to make a new copy - to another format, or just onto newer media? It takes what, 15 seconds to start a copy job using HDs? With optical media, you'll be swapping discs by hand for months.

Plus, HDs are not read only -- you can make monthly backups on them no problem (full or differential). Using optical media this is a pain. The cheap discs are write-once, and even if the rewritable discs were cheap, it still takes a while to erase them all manually.

Backup/sync jobs can be totally automated using HDs. Want to do daily differential backups? Schedule it once, never have to bother ever again. With optical discs you'd be swapping, labeling and cataloging media 365 days a year... What a pain.

Long story short, I might as well backup all my storage on 1.44MB floppies instead of optical discs. It'll take hundreds or thousands of them, and you'll spend countless hours swapping media and splitting stuff to fit the media size. NO THANKS!

Re:Why bother with optical?

[Master of Transhuman]

What's your time worth? As someone said, backing up 100GB of data - something I do periodically and I do it on DVD - takes time. I back up over 200GB of data off my machine to DVD - it takes me most of a day of personal attention. Backing that up to an external hard drive would take much less time and require nothing more than starting the transfer (depending on how the files are being selected of course).

That said, DVD (or tape, of course) is better for offsite archival storage than disk in many, if not all, cases. But nothing can beat disk for fast, local, immediately available backup.

And disks are getting cheaper and with more capacity all the time. While tape is approximately matching disk capacity - and even speed if you buy high-end units - tape drives that can handle that capacity and speed are way more expensive than disks.

It's a no-brainer. The only reason I haven't switched yet to disk backup for everything is that I can't right now afford to buy the drives vs buying 50 DVDs at a time. Plus I'm waiting to upgrade to a whole new machine, at which point I'll move the current machine to act as a file server and attach an external USB or NAS drive system as a backup. This will give me three tiers of backup - the workstation, the file server, and an external backup. Then I can still periodically burn to DVD for offsite storage if I want to.

Re:Why bother with optical?

[KokorHekkus]

And add to this that failure of one optical disc is less catastrophic. If one optical disc fails you have a lot less loss than if one drive does... of course that means more work but there is always a trade-off, isn't there. You could argue that you should have more than one drive but the same goes for optical disc.

From personal experience optical discs might need one more step though: to verify that the write was ok. I don't have any optical discs that have failed me yet after I started to check writes... turns out the only problem was that I had a writer that was about to give up. Hence not a read error but a write one... after that I always run a check.

And a anecodotal story against those who says that it will "allways fail in N months". A swedish tv program took some burned DVDs, put them in a plastic bag and chucked them outside under a bush for about 3 months no problem reading them. So with optical media: check your write + store them as proper archieval media should be.

Re:Why bother with optical?

[pogopogo]

There is a difference between an archive and a backup.

An archive is something that is stored in a safe location for possible use later, but also to save for posterity.

A backup is used to keep current data in two locations in case one set of data is lost.

Hard drives are fine for backups. For archives you want write-once media that can't be easily (or even possibly) erased.

Two different solutions to two different problems.

Re:Why bother with optical?

[MMC Monster]

This article is not a dupe. It is simply a backup of the /. article of December 11, 2006.

Re:Why bother with optical?

[MMC Monster]

On topic: People should use something like par2 to create some fault tolerance so that when a few bytes go bad (or a scratch develops) the entire DVD+/-R is not toast.

I typically create data DVDs with 30% redundancy information (though smaller percentages are probably more than adequate) with par2create, and store those par2 files on the same DVD. That way I survive the little scratches and can recreate the data.

Which leads to...

[BlurredOne]

This just brings up the question:

Is there a point to digitizing human cultural pieces that has survived for 10,000 years onto media that will fail in 10?

Re:Which leads to...

[the eric conspiracy]

That's the benefit of digital, it doesn't degrade over copies, so each generation of storage media gets a perfect copy from the last.

That is certainly not true, and even worse digital is less resiliant to errors than analog formats.

Disks BAH!

[iminplaya]

I prefer cylinders. If they were good enough for grandpa, they're good enough for me.

Note: With I can gather from the name of the site, it appears they might also sell hats, but I couldn't find a link.

Good experience with DVD-Rs myself

[bouis]

A quick primer: you can "error scan" DVD+/-R media with a drive that supports it. CDSpeed, a [free, IIRC] utility distributed by/with Nero, can easily save these scans. DVD enthusiasts often compare their scans... cdfreaks.com is a great discussion site.

Some media have been observed to degrade fairly rapidly, others are quite stable. About a year ago, and again recently, I scanned a number of relatively old DVD-R discs [backups, uh, owned by a friend] burned from fall of 2002 on. You can see my post here:

http://club.cdfreaks.com/showpost.php?p=1733269&po stcount=294

Funny thing is that most of the discs I used were of a brand widely lambasted as "cheap ____" and I was told that they wouldn't last six months. Curiously enough you can see that the cheaper "Princo" media has held up better than the "gold standard of the day" Riteks [although both are much better than some]. You can also see that one of older discs was scanned recently, and more than a year ago. It shows almost no degradation during that time [and what it does could easily be attributed to the aging scanning drive].

The CDFreaks forum has a lot more scans, including of older media. If you've got some discs and are worried about their aging stability, here's a good place to start:

http://club.cdfreaks.com/forumdisplay.php?f=33

Fails to mention MO

[zoftie]

Magneto-Optical media is used by medical facilities where archival time length is paramount. Such as:
  - http://md5.ca/~pavel/md.jpg
  - http://tinyurl.com/2cu7zv
MO drives are a bit costly, but if you have important media its worth it. Besides cool look for neo's warez stash in Simulcara book. Quoted guaranteed archival time is over 40 years in most cases, and they continually improving the technology, compared to driving the costs down of the generic blank media market of CDs/DVDs.

Other tips:

[pizzach]

Other tips for longativity:

- Use acid free CD-markers when marking
- Do not use as a coaster like your America Online CDs
- Do not play Frisbee outside with your CDs (Exposes to direct sunlight)
- Chewing reduces durability

Use dvdistaster

[arose]

No matter what format you choose create error correction data.

DVD Writable sucks badly as archival technology.

[gweihir]

Sorry, but if you have to select media and burner based on some black magic, then the technology is entirely unfit to be used for any archiving. Archival media give you ensurances like 30 years data lifetime in any combination. You can already get that with MOD. MODs have >50 years data lifetime and drive makers will allways support the lasdt three media generations (currently they are supporting all, i.e. the last fife generations and that wor reading and writing). The only DVD technology that is somewhat comparable is DVD-RAM with cartridge. But that seems to have zero market share in the computer business.

My conclusion is that either people do not care about their archived data or that the number of, e.g., lost baby- and wedding-photographs is not high enough yet.

Re:DVD Writable sucks badly as archival technology

[FateStayNight]

Magneto Optical Disc Probably the cheapest comsumer priced devices are Sonys HIMD which are pc compatible, although even these are getting hard to find too. People dont like good technology it seems.

But how reliable are HDDs?

[AmiMoJo]

I ask myself this a lot: how reliable are HDDs in the long term?

Say you use them for archiving. It is better to go the NAS route and have them always on, or powered down when not in use? How about a USB enclosure only connected when in use? How reliable is an almost-new used-it-once HDD that has been sitting on a shelf for five years?