How To Choose Archival CD/DVD Media

An anonymous reader tips us to an article by Patrick McFarland, the well-known Free Software Magazine author, going into great detail on CD/DVD media. McFarland covers the history of these media from CDs through recordable DVDs, explaining the various formats and their strengths and drawbacks. The heart of the article is an essay on the DVD-R vs. DVD+R recording standards, leading to McFarland's recommendation for which media he buys for archival storage. Spoiler: it's Taiyo Yuden DVD+R all the way. From the article: "Unlike pressed CDs/DVDs, 'burnt' CDs/DVDs can eventually 'fade,' due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media)."

Comments

Moo

[Chacham]

"Unlike pressed CDs/DVDs, 'burnt' CDs/DVDs can eventually 'fade,' due to five things that affect the quality of CD media: sealing method, reflective layer, organic dye makeup, where it was manufactured, and your storage practices (please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media)."

How apropos.

'slashdotters' can eventually fade due to five things that affect the quality of slashdot comments:

• sealing method - The Sealing, in reality is the ceiling, and refers to the need to ceil() slashdot user's age to hit the double digits.
  
• reflective layer - The Reflective layer is the use of low UIDs to represent importance of comments, rather than something actually informative.
  
• organic dye makeup - Most comment's make up are so bad, they're DOA, and one can hear the organ playing.
  
• where it was manufactured - There are no new comments on slashdot, everything is either culled from its dupe, or copied from the Microsoft Hater's handbook.
  
• and your storage practices - Some comments are posted before they are fini
  

Re:Moo

[Anonymous Coward]

And that was modded troll why exactly?

I'm Surprised

[aplusjimages]

I'm surprised to hear that consumer media can last so long. I was under the impression that consumer media would only last at most 20 years. Good to know it is longer.

Re:I'm Surprised

[FunkeyMonk]

From TFA:

Short history of the Compact Disc The invention of the Compact Disc has had a large impact on both music and computing in the last 20 years. Invented in 1979 as a joint project between Sony and Phillips to counter the self-destructive nature of consumer audio playback (such as tapes and records that could only be played so many times before the recording degraded significantly) by switching to a resilient digital format.

Imagine that! Two companies working together to create a media that sounded better and lasted longer... and it succeeded!

Re:I'm Surprised

[aplusjimages]

I like how it was Sony who did the cooperating back then, but not so much today.

Re:I'm Surprised

[Anonymous Coward]

I'm surprised to hear that consumer media can last so long. I was under the impression that consumer media would only last at most 20 years. Good to know it is longer.

All current forms of optical storage share the same problem that will limit their reliable life span. This problem will effect different media design types over different periods of time, but 20 years is a good average before you will start to see bit rot. How ever ALL optical media, regardless of being commercialy pressed or being consumer grade CD/DVD-R, will eventualy suffer bit rot for the exact same reasons. The problem is that the rate of expansion and contraction of the metal substrates that make up the innner layers and the plastics that make up the outter layers are very different. All optical media will suffer bit rot over time because of this, as what eventualy happens is the metal substrate inside the plastic protective layers gets warped and ripples start to form in the surface. This of course starts to alter the smooth/pit reflection encoding of data on the disc and ruins the data.

Again, different types of optical media design will last longer than others. Yes, commercialy pressed CD/DVDs will last longer than consumer grade CD/DVD-R media on average. How ever none of these formats have the reliability and shelf life of magnetic backup tapes! Espeically newer formats like AIT, SAIT, and LTO (and VXA isn't too bad either, awesome pricing on smaller VXA auto loaders). So if you are looking for reliable long term archiving CD and DVD are NOT what you want to use! You want to use tapes. How ever, you can use CD or DVD if you keep in mind that this format has a shorter shelf life and you plan a migration of that data to a new removable media format say 10 years out from now. You have to do the same thing with tapes too, as eventualy formats become old enough that it gets hard to find tape drives that will read your backups. And yes there is still a limit to the shelf life of backup tapes, as over time the magnetic signals encoded in the tape start to transfer between layers due to the tape layers being tightly wound around each other on the take up spool. How ever, even with that in mind, tapes provide a much longer useful life span for archiving data. And beyond having a longer shelf life tapes have several other advantages. They have much larger capacities than CD/DVD media, so you don't have to sit there rotating tons of discs to restore a large archive. And in many cases newer tape formats have much faster transfer rates than CD/DVD (granted they cannot do random I/O, but burning CD/DVD isn't a random I/O process either). Any one who has serious data to protect shouldn't be using CD/DVD for backup. This is more of a cheap low end consumer approach for those who cannot dish out $1000+ for a good tape drive.

Those who still have the wool pulled over theirs eyes and still think CD/DVDs are a long term storage platform need only look to those of us who have very old CDs and laser discs for proof to the contrary. I know plenty of people who bought some of the first pressings of CD albums back in the early 80s, and many of their earlier CDs are now suffering bit rot. This is even more prevelant on laser disc, probably because of the much larger surface area being affected. Many laser disc owners are fully aware of the problem of optical media bit rot. In many cases the bit rot gets bad enough that you can visualy see the distortions on the reflective surface.

Bottom line, if you value your data then use backup tapes! That's what that technology was invented for! CD/DVD-R is more of a short term backup option, best for cheap short term archiving or transporting of data. I use CD/DVD-R for regular backups of the documents on some of my workstations. But when it comes to our servers and our customer's servers it's AIT or LTO tape drives all the way!! Use the right tool for the job!!

Re:I'm Surprised

[Anonymous Coward]

Very informative post, thanks.
But:
"Those who still have the wool pulled over theirs eyes and still think CD/DVDs are a long term storage platform need only look to those of us who have very old CDs and laser discs for proof to the contrary.
Wouldn't it be fairly trivial to re-copy a backup every 10 years or so? I'm assuming that one is doing this for smaller scale personal data (i.e. my photos), not industrial amounts of data (i.e. Google's backups).

Re:I'm Surprised

[asuffield]

Bottom line, if you value your data then use backup tapes! That's what that technology was invented for!

Tapes were invented as a short-term mostly-portable bulk storage medium - they were the first such medium invented.

Nowadays, you can actually buy archival backup tapes - but the tapes that you bought for your general purpose backups today are not archival-quality. They will not last 20 years, even on a dark dry shelf. Do not expect them to. If you want to keep them, they need to be regularly rewritten every few years. Even archival tapes need to be rewritten every few decades (all magnetic storage media fades eventually).

If you want your data to survive, don't use magnetic media. The most reliable long-term bulk storage we currently have is actually a chemical/optical technology - microfiche (estimated lifespan ~500 years under the right conditions). That's what people use for things that really, really need to survive (ask your librarian).

Re:I'm Surprised

[Bigman]

There is no "Golden Bullet" for archival. Any technology can fail. The only answer is good archival management - that is, at the very least, you should keep copies, and regularly (e.g every few years) copy onto fresh media, and compare with the "old" media. Keep a few generations of copies.

Most people do this "accidently" over time by moving their data to newer technologies (e.g. floppy->cd->DVD) and no storage technology remains king forever. about 20 years seems the average, so if your cheapo CD's only manage 30 years, and you're not stupid enough to only keep 1 copy, then you don't have any worries.

Storing archival data on powered up RAID storage seems to me to be a sledgehammer to crack a nut. Why not just make 3 copies, store them in different places, and periodically copy/compare them to make sure they don't deterioate? Not so many geek-points I guess.

Re:I'm Surprised

[tbuskey]

Magnetic media eventually gets randomized by the earth's magnetic field.

Also, given tape's history of obsoleting formats, I'm not sure how long and format will stay around.
9 Track, QIC, Travan, Exabyte.....

He forgot the most important one...

[hal2814]

"(please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media)." And NEVER ever feed them after midnight. On a more serious note, I used to worry about eventual degradation but it's coming up on 10 years that I've owned a CD-R drive and I have yet to run across a burned CD I cannot read due to this sort of degradation. Maybe at the decade mark, some of my discs will fail me and I'll change my mind but right now I'm not too concerned.

Re:He forgot the most important one...

[Super Dave Osbourne]

Used to run an archiving business of sorts (trade stuff in exchange for space) on many towers of CDs. I burned in the mid to late 90s many CDs and have them all archived still. Went back and uploaded them to a terabyte raid I built and without one failure of the CDRs that I archived, not one degraded to not mount and copy. I had a few that had scratches, but that is a different story and not related. Bottomline, buy what works, cheap and don't move them once archived, keep in cool dark place, and you are good to go. Oxidation is a CD-ophile's issue, not much in the reality zone. Heat is an issue, and if you store them on your dashboard, you deserve what results.

Re:He forgot the most important one...

[Rob T Firefly]

"(please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media)." And NEVER ever feed them after midnight.

Do not taunt Happy Fun CD.

Re:He forgot the most important one...

[r3m0t]

Maybe you should download a program which checks the error correction codes on the CDs. You might find that some of them are almost unreadable but the error correction means that they work as normal.

Your CDs are likely to fail all at (almost) the same time.

Re:He forgot the most important one...

[Kenshin]

Do not taunt Happy Fun CD.

Wanring: Happy Fun CD may install a rootkit on your system if it feels threatened.

Re:He forgot the most important one...

[Anonymous Coward]

I've had cheap CDs that died in 2 years. I've had expensive CDs that still work after a decade. You Get What You Pay For.

Heck, one of my earliest cheap CD purchases I could hold in my hand and clearly see my hand through the disc, I think the data lasted on that CD about 6 months.

Re:He forgot the most important one...

[greg1104]

I got my first burner in mid-1996, so my early discs are over a decade old. I've seen a small but non-zero number of failures among the CD-R burns over time, maybe 10 discs out of over a thousand discs. Haven't seen any from the better quality gold media (Mitsui, Kodak), but a few from CD-R and DVD-R discs made with other formulations. The failures I remember the details of were from Sony (x2, but I used a lot more of these than the others here), 3M, Memorex, and Mitsui Silver (x3). At any time I normally use a 50/50 mix of expensive gold media for important files and whatever's cheap for everything else. Normally the failure is that the disc will still mount but many sectors have unrecoverable errors.

I've lost multiple discs burned onto completely worthless KHypermedia DVD-R media after less than 3 years--two boxes of media that were free after some promotion, yet I was still ripped off.

Re:He forgot the most important one...

[freedumb2000]

Just to give an example how it can go not so well: A few years ago I bought and burned over the course of a few months exactly two brands of CD-Rs. Only 2 years later I had to realize that all disks of one of the brands had become strongly discolored and i was only able to rescure about 50% of the data of each disk on average. The other brand was completly fine. And BTW, i stored brands cool, dry, in plastic casing and at the same place. This experince has really scared me off of using DVD+/-Rs for backup purposes. I use mirrored external HDDs now.

Re:He forgot the most important one...

[WuphonsReach]

Heat is an issue, and if you store them on your dashboard, you deserve what results.

I have MP3 CD-Rs in my car from 2001 that have either been up in a visor holder or stored in a door compartment. I live in Pennsylvania (to give you an idea of the weather extremes). Now, they weren't in direct sunlight, which probably helped. But you figure they still ran the temperature range of 0F up to 120F.

I think I might've had trouble with one or two disks over the years. But since they're MP3 CDs, I just re-copy them from my central storage (or else I backed the disks up to DVD-R which I keep in my house).

Re:Price is all that matters in the short run

[Anonymous Coward]

Brilliant. Now try reading them on another drive. See ya! /idiot

Bummer

[Deagol]

I always thought Matsui "Gold" and "Silver" were the top-rated media. At least for CD-Rs (though I thought they were held in high regard for DVD blank media, too). I used to mail-order un-branded blanks them by the spool.

Re:Bummer

[greg1104]

Mitsui's gold media has generally been considered the best available for CD-R work, particularly from an archival perspective. The company has reorganized and now goes by the name MAM. If you look through the comments after the article, the author suggests that the currently available MAM media isn't as high of a quality as the older Mitsui discs. I would like to see some citation for that fact, as I wasn't aware the formulation was changed at all from that reorg, but I haven't researched this subject recently enough to be able to dismiss his suggestion outright.

Re:Bummer

[Anonymous Coward]

I'd like to see some citation that any media is better then any other media in terms of longevity. I can find 100's of sites that claim Mitsui's are the best but I've yet to come across a single one that is not based on a very thin theory or just a "feeling" of why it is better. Yes, there are some chemical names thrown around but simply stating a specific chemical name is not proof it is better or worse. Everyone that jumps on the bandwagon assumes the others know. A truck load of people preaching the same thing can't possible be wrong, right?

Re:Bummer

[Firehed]

Modded funny? I know that the moderation system on Slashdot is totally out of whack, but I fail to see any possible opportunity for humor in this post. Any of the In_ mods, certainly, but funny?

Of course, I expect the requisite *woosh* response to my own post, but at least be kind and explain what I've missed when you do.

Re:Bummer

[bazorg]

oops, clicked 3 pixels above the intended spot. no need for conspiracy theories, ok?

Re:Bummer

[lostboy2]

FWIW, it looks like some comments to the original article suggest that MAM-A media is pretty good too, although the article's author seemed dubious. From one comment:

I have two production shops that, combined, produce over 60,000 pieces of media per year on six Rimage Protege II units. A combination of CD's (~90%) and DVD's (remainder). All of our media is MAM-A (Formerly Mitsui Advanced Media) and uses Mitsui's Phthalocyanine dye (patented by Mitsui Chemicals). That media is also produced here in the U.S.A. - at their plant in Colorado.

While I agree that TY media is extremely good, I do not feel that it is light years ahead of all other media produced. Not to the point you have made it out to be. It's good media, and I would recomend it. Their surface printing characteristics are not to our standards. That's all.

And another:

I have to agree with [the previous comment]. MAM-A media is solid stuff (I work with a large media archival group where this stuff has to last 100's of years).

And to be honest, as much as I respect the folks over at CDFreaks, it seems to have taken on a kind of Taiyo Yuden fanboy mentality. TY undoubtedly makes some of the best quality media around, but the CD community seems to have made their media into some kind sacred cow.

I tend to agree with the second comment's statement about the folks at CDFreaks, especially since the original article suggested that there was no such thing as a gold DVD (the article has since been edited). We've been using gold DVDs for awhile. So, this suggests that the article's author had not done his homework and/or was operating under old assumptions or data.

Anyway, I wouldn't toss your MAM-A media just yet. :-)

Safety in Numbers

[Doc Ruby]

Cheap but adequate DVD-R media costs $200 for 1000 discs, about 4TB capacity. And a cheap DVD-R changer jukebox costs under $500, about 800GB per load.

Why not just burn a few copies of the archive to a bunch of DVD sets? The DVDs will get defects, but shuffling the chunks across the discs just a little will probably ensure that the random distribution of specific defects will not hit every copy of a given bit, against the odds a low defect rate will produce.

How about a pair of those archivers, which fire up every few years just to transfer the aging DVDs to fresh new ones? For another $1000, that's another 5 cycles of DVDs, 800GB per cycle. Another $1000 gets a pair of backup jukeboxes.

For higher capacities than 800GB, there are pricier pro jukeboxes, but with dual drives for the retranscription cycle (and faster restores). But the architecture is the same. Why try to make the media more reliable, when there's cheaper/easier solutions that just accept unreliable media, and move on?

Re:Safety in Numbers

[ericdano]

Why not just get a NAS that has RAID? That would make more sense. When a disc dies, you can replace it, rebuild your array, and everything is fine. PLUS, you could expand your archive over time.

I think it's absolutely stupid to use a DVD jukebox. Really. Look into a NAS box with RAID.

Re:Safety in Numbers

[God'sDuck]

I think it's absolutely stupid to use a DVD jukebox.

Judged by convenience/ease/performance, I agree. But no level of RAID tiering is going to help you with the one thing off-line (tape/removable disk) solves quite nicely: that is to say, our good friend, more-than-a-surge-protector-can-stop, lightning.

Re:Safety in Numbers

[failedlogic]

Yeah, I'm starting to realize this is really the more practical alternative too. I think with the higher density of storage, more easily organize files and faster retrival speeds its really ideal. Theres some nice Linux distros pre-configured for NAS/SAN on different protocols which are super easy to setup. I've tried a few of them. It saves the price and hassle of proprietary units.

I would think - I don't know that much about backup media - that something like tapes would be better for archnival than DVDs. Even with higher density Blu-Ray or HD-DVD would be prone to the same fail over as CD-type media. I've been looking breifly at prices on Tape drives. Some seem quite high. Is there a make/model ideal for personal and SOHO users now? There seem to be a lot less on the market than I remember 10 years ago targeted at this price range. Looking to backup about 1 TB max (for future). Probably closer to 500 Gig for now though.

Re:Safety in Numbers

[ericdano]

How often does that really happen? Really? Got Stats for it? Are DVD's capable of surviving a bolt of lightning? I think Fire would be more probable. Or Flood. But if that was happening, it would be rather easy to disconnect my ReadyNAS NV and take it with me in the car than a huge jukebox DVD thing.

If you really want to be safe, have two NAS devices. One as your main one, for putting stuff on, and the other to make a backup of the NAS device. Or have the NAS backup itself to another drive. There are a number of ways to do it.

Re:Safety in Numbers

[ericdano]

I think the little box NAS options, by such compies as Infrant (ReadyNAS NV, which I own), are great. They are small (like shoe box), quite, and run linux. And they are expandable. That is important. You can swap out drives, and expand your RAID. So, the 1 Terrabyte array I have now, when I decide I need more room, I can swap out the disks (one at a time, letting it rebuild), and when I'm done, I can have 2 Terrabytes, perhaps more (Infrant is supposedly going to offer over 2 Terrabyte filesystem next year).

Doesn't really matter what you do, but a NAS or a build your own NAS (PC with Linux) that is RAIDed is a great backup solution. Two would be ideal (one for use, and one for backing the for use one).

Re:Safety in Numbers

[Hijacked Public]

And neither will help with fire, unless you ship your offline stuff offsite.

I use a RAID both at home and at my studio. At the studio, mostly empty warehouse but good fire supression, I also back up to tape. At home I have a wireless NAS inside a safe, but your comment just made me realize that lightining could actually set a fire inside the safe. Unlikely but possible I guess.

Re:Safety in Numbers

[devilspgd]

Are DVD's capable of surviving a bolt of lightning?. Yes -- They don't get hit, whereas all of the NAS products I'm aware of need power, and as a result are wired.

With DVDs, you can have an off-line copy, or even store copies at alternate physical locations.

Re:Safety in Numbers

[Doc Ruby]

With the single DVD jukebox, the first 800GB is online at one time, for $450. A 750GB HD costs $350. But the next 800GB in DVD costs only $40 - each 4TB costs $200. And there's no limit to how many $50 TBs you can archive, with a sizeable enough closet. The downside is un/loading the jukebox, 200 at a time. But that's archive, "nearline" storage.

Plus, you get a DVD reader and writer. For dealing with the DVDs (and CDs) that still distribute lots of content as a transfer medium. And for those without distributed endpoints to where they can archive data, or insufficient network bandwidth to archive all their data across the WAN frequently enough, DVDs are good and cheap offsite archive repositories. Plus you can burn DVDs that will play in every consumer player, which can connect your data to lots of people without data processing HW. HDs are a cul de sac for data, trapped within the infosystem.

DVD archiving isn't really competition to online HD storage. It's complementary, in different use cases, different user environments. There's considerable overlap in their related extremes, but there's a lot of difference that makes leaves the DVD solution worthwhile for many scenarios.

BTW, while I'm offering detailed factual analysis of HD vs DVD mass storage, don't throw in your "opinion" that "it's absolutely stupid...". Especially if you're going to offer a disagreement worth considering. Do you want to work together to figure out the real merits in a debate, or do you want to get into an obnoxious pissing contest that few other people will want to wade through? Few people worth teaching will learn anything from such unnecessary conflict. Including ourselves.

Re:Safety in Numbers

[kimgkimg]

I'm interested in finding a good DVD-R jukebox. Could somebody point out some good ones to take a look at?

Re:Safety in Numbers

[novus ordo]

Tape systems are quite expensive. Even more expensive than HD solutions. I didn't think that was possible.

Re:Safety in Numbers

[ericdano]

"With the single DVD jukebox, the first 800GB is online at one time, for $450. A 750GB HD costs $350. But the next 800GB in DVD costs only $40 - each 4TB costs $200. And there's no limit to how many $50 TBs you can archive, with a sizeable enough closet. The downside is un/loading the jukebox, 200 at a time. But that's archive, "nearline" storage."

But what happens if a DVD gets corrupt? Or scratched? Or, lost?

"Plus, you get a DVD reader and writer. For dealing with the DVDs (and CDs) that still distribute lots of content as a transfer medium. And for those without distributed endpoints to where they can archive data, or insufficient network bandwidth to archive all their data across the WAN frequently enough, DVDs are good and cheap offsite archive repositories. Plus you can burn DVDs that will play in every consumer player, which can connect your data to lots of people without data processing HW. HDs are a cul de sac for data, trapped within the infosystem."

Ok. A DVD Reader and writer. Woohoo. For little bits of data, yeah, ok, DVDs are good. But if you are archiving data, say, lots of data, do they work? What about getting data off bad ones? I've tried getting data off a damaged DVD once. It wasn't pretty. At all.

"DVD archiving isn't really competition to online HD storage. It's complementary, in different use cases, different user environments. There's considerable overlap in their related extremes, but there's a lot of difference that makes leaves the DVD solution worthwhile for many scenarios."

For some, yeah. I use DVD to archive a lot of projects, but at some point, it seems to make sense to consolidate all the DVDs. I think maybe when we can burn 20 gigs a DVD or more, it would make more sense.

BTW, while I'm offering detailed factual analysis of HD vs DVD mass storage, don't throw in your "opinion" that "it's absolutely stupid...". Especially if you're going to offer a disagreement worth considering. Do you want to work together to figure out the real merits in a debate, or do you want to get into an obnoxious pissing contest that few other people will want to wade through? Few people worth teaching will learn anything from such unnecessary conflict. Including ourselves."

Your first "factual analysis" included "Cheap but adequate DVD-R media costs $200 for 1000 discs, about 4TB capacity. And a cheap DVD-R changer jukebox costs under $500, about 800GB per load." which is more expensive than a HD. Then you say it $450? Which is it? I think it's more like $450 (jukebox) + $40 (200 DVDs). Then "How about a pair of those archivers, which fire up every few years just to transfer the aging DVDs to fresh new ones? For another $1000, that's another 5 cycles of DVDs, 800GB per cycle. Another $1000 gets a pair of backup jukeboxes." which, doesn't make sense. If you are just going to "fire then up every few years" why not just use a HD? Or two HDs? Just seems a huge waste of time (burning all the DVDs), and effort (loading/unloading) to get what benefit? To say you have all that neat stuff? How many hours would it take to archive all that stuff? I'm thinking days......week? Two weeks with verification after burn?

I'd just stick to hard drives, and have intervals where you'd check out the data, and every 2 years or so, consolidate data on a newer bigger drive.

Re:Safety in Numbers

[caseih]

Don't mistake NAS for backup. We plan to implement a NAS and a DVD archive. Here's how it works. Every night the main array is snapshotted. We figure we'll easily have enough storage to maintain about 6 snapshots, given that we change an average of 6 GB of data a day. We'll copy the full snapshot off to backup disk (IE removable disks), about 2 TB worth of data. Then we'll take the snapshot and make ISO images of just the changed data which we'll write to a dvd. If we plan on keeping a year's worth of DVD's, we can track a file back an entire year. This way we back both redundant storage (the RAID), full backup (going back 6 days) and an archive (going back an entire year. This may seem like overkill, but in a scientific institution this is what the doctor ordered.

Re:Safety in Numbers

[megaditto]

So, the 1 Terrabyte array I have now, when I decide I need more room, I can swap out the disks (one at a time, letting it rebuild), and when I'm done, I can have 2 Terrabytes, perhaps more

[emph. added]

No, you will not get "more room". You will still have 1 Terrabyte of the same data copied out over more disks, but still only 1 Terrabyte.

Re:Safety in Numbers

[OldeTimeGeek]

NAS is a nice solution, but that doesn't solve the problem of off-site storage.

I use NAS for my home business but still archive my work to DVD and send it to a relative to store. If my house burns to the ground or if someone breaks in and steals my equipment, I can still restore. If I had everything on NAS, it would be gone.

Re:Safety in Numbers

[Jeff DeMaagd]

Are there consumer systems for doing this, and are they reliable? Are the DVDs easily recyclable? I'd hate to be buying more and more stacks of plastic laminated metal that can't be recycled when they are no longer needed.

I certainly don't want to be dealing with jukeboxes that fail. I knew some people that dealt with a commercial grade jukebox that was simply unusable, it would routinely drop media and otherwise misbehave.

I'm also not interested in manually changing out discs.

For me, it's all adding up to just using a remote second computer to store a backup on hard drives.

Re:Safety in Numbers

[Anonymous Coward]

Do you want to work together to figure out the real merits in a debate, or do you want to get into an obnoxious pissing contest that few other people will want to wade through?

Do you think I'm reading /. to get real-merits-in-a-debate crap?!

Now if you don't mind, I would appreciate it if you would please stop wasting my time and get on with the DVD-jukebox -vs- RAID "Archive Storage: disks in the mists!!!" pissing contest.

Thank you, harumph, and ding-ding.

Re:Safety in Numbers

[Doc Ruby]

The $400 I cited is for consumer models like the Sony Vaio version for their home theater PC systems. I don't know how reliable, but I expect they probably are, and under warranty. But I still mentioned buying extras for redundancy.

I don't know whether DVDs are easily recyclable, but I believe that they are more recyclable than are hard drives. Maybe not.

Everything fails. HDs have MTBF, as do jukeboxes, though I don't know whether the consumer ones are either really tested and that spec published, or how high it is.

As far as manually changing loaded sets, I haven't seen a consumer model (<$5000) that offers bulk loads of sets in a holder. Though even the holder would get loaded from the packs of DVDs, unless the "cake boxes" could be loaded directly - never seen that. 200 discs takes about a half-hour mind-numbing un/loading to cycle a full load. A half hour every few years doesn't seem like a big deal, though HDs are better. And it does seem like it could be automated, a robot arm to fill other carriers (like a 200-disc book). I'd love to see a Lego Mindstorms robot combined with one of these cheap jukeboxes, with 1000 DVDs in books stacked on top, handling 5TB for under $1000.

This system isn't for everyone. In fact, my post is in response to the strategy in the story to try to get DVDs to last longer. I say just use more DVDs, with an automated redundancy system.

Re:Safety in Numbers

[Doc Ruby]

In the original grandparent post to which you replied, I started out by saying that the point of the DVD changer is to make several copies of the same data, on different DVD sets. So if a DVD gets scratched, or otherwise ages (as detailed in the story which we're tangentially discussing), there's another copy. More expendable DVDs, instead of tougher ones.

I'd say that the 4.7TB that $1K will buy, along with the DVD-R burner jukebox, is more than "little bits of data". I already pointed out how to deal with inevitably damaged DVDs.

A 200 disc book can hold a lot of DVD data. For most people, 4.7TB is more data than they own, including all their movies. Since DVDs can actually hold 17GB when double sided/layered, that's 17TB. If you've got more data than that to store, you can afford $10-20K+ for substantial robotic archival systems.

Your attempt to argue with my economics is contradictory only because you didn't understand its plain point. The jukebox costs about $400 (as little as $300). 1000 DVDs cost $200, but only 200 will fit into the jukebox at once. So that first 800GB on 200 discs costs $400 + 40, or about $450 - the rounding error for shipping, and actual price of the jukebox, makes $450 a charitable statement, when it will probably be less.

I'll even parse the next statement you didn't understand. The point is that larger storage than just the first jukebox load quickly becomes much cheaper than HDs. And more incremental, without having to buy an entire 750GB HD to store the next 500GB. Or by spending that money on much more DVDs. Which also bring the the other benefits, including flexibility, that HDs don't offer. And though you don't get it, the point of a pair of jukeboxes is to transfer an entire 200 discs to "fresh" DVDs that haven't aged like the story we're discussing describes. Again, solution to the "expiring DVDs" problem that just uses more DVDs.

I don't know how long the unspecified DVD jukebox would take to burn a full load. So what? The archive is supposed to last for months, years. If you have more frequent archiving demand, use a different tool.

The point isn't "more neat stuff", but the right tool for the right job. Since the job we're discssing is "coping with expiring DVDs", there is a way to just use more DVDs, as I've detailed. There are arguments for using HDs instead, but not necessarily in all situations, or even in most. But then, since you're not discussing the same thing as am I, we haven't gotten anywhere in the point of this thread.

Re:Safety in Numbers

[maxume]

Or choose your acceptable failure rate with par or par2 and the like:

http://www.quickpar.org.uk/
http://www.binaries4all.com/quickpar/
http://en.wikipedia.org/wiki/QuickPar

Rather than hoping that your data exists on 1 of your four 5GB dvds, you hope that one of your three 4GB dvds is mostly ok and everything is dandy. It's a choose your own level of error correction world, rather than a minimal error correction world. Popular among the porn/rom/binary set on usenet.

Re:Safety in Numbers

[profplump]

Only until you start making archives. Tape media is easily 50% cheaper on a per-GB basis once you get past the hardware purchase. With hard drives you never get past the hardware purchase.

If all you want it a copy of the current (or the last few) day's disk state, then another disk makes good sense. If you want to be able to restore last December's tape because someone trashed your 1099-processing software in January and no one noticed until today, tapes are definately cheaper.

Re:Safety in Numbers

[Kent Recal]

With the single DVD jukebox, the first 800GB is online at one time, for $450. A 750GB HD costs $350. But the next 800GB in DVD costs only $40 - each 4TB costs $200. And there's no limit to how many $50 TBs you can archive, with a sizeable enough closet. The downside is un/loading the jukebox, 200 at a time. But that's archive, "nearline" storage.

Hmm, where can I buy that $400 jukebox that will also write my DVDs?
Because with a single writer (or even a bunch of them) writing 200 discs is quite a painful process.
You're looking at 13hrs writing time and you can't go away because you have to babysit (swap disc, write label)
every two minutes.

Unless you need fast random access to your archive then tape is still the way to go.
A single AIT4-drive sets you back about $1500. Each tape holds 200G so you get
4 tapes instead of 200 discs. Quite a difference..

Re:Safety in Numbers

[blankoboy]

Can you easily replace the pron your wife deletes from your NAS? Moral of the story, RAID != backup. It is a good supplement bot not a replacement.

Re:Safety in Numbers

[WhoBeDaPlaya]

A NAS ain't gonna save you from the software risk factors (virii, malware, PEBCAKs, OOPses, etc.) With DVD+/-Rs being so cheap, just reimage them every once in awhile.

Re:Safety in Numbers

[Anonymous Coward]

Remote NAS?

Re:Safety in Numbers

[Doc Ruby]

People still trade porn on Usenet? There's still a Usenet?

Maybe we should chuck all these archival media strategies, and just hide our data in steganographic low bits of porn, uploaded for eternal trading on whatever media Usenet is still "temporarily" stored.

Re:Safety in Numbers

[Doc Ruby]

Sony's Vaio DGP-XL1B.

You're right about tape. But we're talking about "How to Choose Archival CD/DVD Media". Can't play your copy of _The Life Aquatic_ in a tape drive.

Re:Safety in Numbers

[Anonymous Coward]

We use Ultrium LTO3s. They average about $70 each for 400GB uncompressed. 400GB HDs can be had for $110.

Do not know if that price difference proves or disproves your theory.
I do know we paid a LOT for our StorageTek fiber connected library and our Commvault backup software, a lot like over $300K USD. Of course, I have no idea how much a removable HD solution would be. We paid well more then 300K for our SAN but I guess a HD backup would not need the redundancy and speed of a SAN.

Re:Safety in Numbers

[trentblase]

How do I expand an existing X-RAID volume with larger disks?

Replace one disk at a time with a larger disk, letting it finish initializing and syncing after each replacement (this process can take several hours depending on disk capacity, but you can continue access to the ReadyNAS), and after the last disk has been replaced, reboot the ReadyNAS. The expansion will occur at boot time. The expansion time will depend on your existing volume size, the ending volume size, and the number of files in your volume. Typically, it'll take anywhere from an hour to several hours. You will be notified by email at each step of the process.

Keep in mind that when your replace your disks, with ReadyNAS that supports drive hot-swapping (NV, 1000S), you do not need to shutdown the box before replacing the disks. Simply pull out the disk, wait at least 10 secs, and then add the replacement disk.

Re:Safety in Numbers

[trentblase]

HDs are a cul de sac for data, trapped within the infosystem.

Yeah... I've yet to see a computer that knows how to read a HD.

Re:Safety in Numbers

[ericdano]

Not true. If you strictly log on it the NAS as a read only user, and only add stuff to it using a virus/spyware free computer, say, oh I dunno, a MAC, then there should be no problems. Right?

So far, I've had no problems with it.

Re:Safety in Numbers

[Anonymous Coward]

It's still dumb as the previous posted noted. Geeze dood, how many hours are you going to spend burning DVDs?

Get a HD. Then go take your fat ass out and exercise. See the sun light, etc.

Re:Safety in Numbers

[ericdano]

This is true. A solution would be to have a NAS hooked up at your relatives house, and have a high speed internet connection between the two, and have your NAS backup to the remote one.

Re:Safety in Numbers

[ericdano]

Wise words indeed. But, you don't use DVDs to archive all that stuff. You use REMOVABLE DRIVES. There is a HUGE difference.

For the record, I use a NAS and back it up every time I change a bunch of stuff on it. It basically is holding music and various projects I have done.

The backup drive goes home with me.

And, regardless of what I have done, I use another drive, on Saturdays, to backup the whole NAS.

Re:Safety in Numbers

[Doc Ruby]

The computer is in the networked infosystem. Most humans aren't. Removable discs can connect more people to info than can HDs.

Sarcasm has become an end in itself on Slashdot. Why bother thinking about the post, when you can just shuffle the words around and insert some ellipses?

Re:Safety in Numbers

[Doc Ruby]

I have machines that burn DVDs, and do other tedious automated work, for me. Like a 200 disc changer, so I can do other stuff then pick up the finished product.

Get a brain. How many hours are you going to waste with computers you think you have to wind up or something?

Re:Safety in Numbers

[trentblase]

Sorry, I really did misunderstand your post. Apparently you were not saying that computers can't read HDs. You were saying that humans can't. Which leads me to revise my sarcastic comment to: Yeah, cause humans can read DVD data just fine without a computer.

Re:Safety in Numbers

[Doc Ruby]

Yeah, because there are so many more people with computers than with DVD drives.

That was so easy, that it isn't worth doing again.

You are a jerk. Goodbye.

Re:Safety in Numbers

[JazzLad]

But at ~$700 (cited above) for ~800GB ... I can get 3 320GB HDDs for $300, a second set & we're talking two copies of everything for still $100 less. Add 3 external cases & toss 3 HDDs in a safe deposit box for alternate physical location and still spend less. HDDs are a lot cheaper & a heck of a lot faster than carousels of DVDs.

That said, I personally use Verbatim DVD+R for backup, about 1-1.5TB currently (not including movies, etc) prolly cost me about $125. Not "always on," but like it's a hassle to grab a binder off a shelf when I need it.

YMMV

Re:Safety in Numbers

[jabuzz]

A couple broadband connections later and a bit of fiddling and you can locate those two NAS drives anywhere you can get a broadband connection. I am looking to put my second one two hundred miles away at my brothers.

Re:Safety in Numbers

[L33tGreg]

RAID is not a backup. It is redundancy. A real backup will allow you to restore files you accidentally deleted or a database that corrupted 3 weeks ago. RAID mirroring and will mirror defects as well!

Re:Safety in Numbers

[ericdano]

True, which is why you should back up a RAID as well to hard drives or another RAID.

Re:Safety in Numbers

[trentblase]

You're saying I should back up my database to DVD so that people can watch it on their home entertainment systems?

Re:Safety in Numbers

[profplump]

You're paying too much for tapes. You can have LTO-3 for $55 in 5-packs, and even a little less if you buy in bulk or refurbished (which may or may not be approrpiate depending on your archive needs). That's where I got my 50% number. A quick google should for something like "LTO-3 tapes" should show you what I mean.

And if you honestly know where I can get a 400+ GB hot-swappable (i.e. SATA or SCSI) drive for $110 I'd be interested. I'm not doubting your numbers, just looking for deals -- I thought the 320 GB for $95 I just bought were a good price.

Re:Safety in Numbers

[Anonymous Coward]

The tapes. We usually buy 5 cases at a time, each case is 4 boxes of 5 for 100 total. I assume we probably get a discount.

Here is a link to referenced HD for $110
http://www.newegg.com/Product/Product.asp?Item=N82 E16822152046

YMMV with the quality.

I agree... But where can I find some?

[madhatter256]

Taiyo Yuden All the way. They are great for copying PSX/PS2 games (seriously) where media quality makes a difference between burning out a laser or playing your back-up game, as well as DVD Movies.

The only drawback is that you can only order them from the Internet. I do not know of any retail store who actually sells the brand outright nor do I know of any brand (like Sony, Memorex, Fujifilm) who sells rebranded Taiyo Yuden discs.

Also, the Taiyo brand is more expensive than any other brand.

Re:I agree... But where can I find some?

[Tiro]

Sony sells them, I think you just have to look for the ones that say "Made in Japan" I saw them in Circuit City, but the floor staff pushed me to buy the product with the big rebate instead.

Re:I agree... But where can I find some?

[Anonymous Coward]

www.rima.com, cheap unbranded TY Media. Get the 8x its cheap these days and usually will burn at 12x.

Re:I agree... But where can I find some?

[bill_mcgonigle]

The only drawback is that you can only order them from the Internet. I do not know of any retail store who actually sells the brand outright nor do I know of any brand (like Sony, Memorex, Fujifilm) who sells rebranded Taiyo Yuden discs.

I'm told the Fuji's that are "Made in Japan" are usually Taiyo Yudens. I've gotten a pack of them at Best Buy (the country of origin isn't always the same) and they've worked well. I haven't gone the additional step of running a disc ID program on them.

Usually I just order a spindle of a hundred injket-printable discs from Meritline for $30 or so when I'm running low. Watch DealMac for coupons every once in a while. Thirty cents for 4GB of archival data is worth it for me. I burn two if I really care about the contents. 60 cents is also worth it.

Also, the Taiyo brand is more expensive than any other brand.

Ah, that old "you get what you pay for" canard again...

Re:I agree... But where can I find some?

[mobby_6kl]

Some Verbatim DVD+Rs are manufactured by Taiyo Yuden. Strangely, the ones I used were the cheap-looking colored ones, but I've used several batches of those with excellent results. Speaking of Verbatim, some of their other discs are made by Mitsubishi, and those also very good, although I'm not sure how they'd work for PS2 piracy ;)

Re:I agree... But where can I find some?

[Anonymous Coward]

If you happen to live in Edmonton or Calgary, Alberta, you can get them from Memory Express. Pretty cheap too.

Re:I agree... But where can I find some?

[settrans]

Sony media imported from Japan is Taiyo Yuden as far as I can tell.

Re:I agree... But where can I find some?

[cockroach2]

As far as I know, *ALL* Plextor CD-Rs are made by Taiyo Yuden - so I usually just buy those.

Wikipedia can also help you identify them.

Re:I agree... But where can I find some?

[Anonymous Coward]

The last few batches of Verbatim DVD-R and DVD+R i've bought over here in germany have all been made by Moser Baer India, were of mediocre to bad quality, and impossible to tell apart from the previous (ok to good quality) Mitsubishi Chemical blanks. As I have no interest in playing the media quality lottery when buying blanks, I have since switched completely to Taiyo Yuden media, usually from Plextor.

Re:I agree... But where can I find some?

[Anonymous Coward]

Fuji brand CDs that are "Made in Japan" are pretty much guaranteed to be manufactured by Taiyo Yuden. The cases they are stored in usually have a somewhat rounded bevel on top about 1 cm thick. Some Fuji CDs are made in Taiwan, so be sure to check. I don't know any DVD brands in the US that distribute Taiyo Yuden.

The ever-loving NewEgg.com supplies media direct from Taiyo Yuden as well.

Re:I agree... But where can I find some?

[mrchaotica]

although I'm not sure how they'd work for PS2 piracy ;)

He didn't say "piracy," he said "backup!"

Re:I agree... But where can I find some?

[Anonymous Coward]

I work for a retail store that sells Taiyo Yuden. We sell them for $30 for a spindle of 100. We're only really located on the west coast, with many stores in California. I'll refrain from posting the name unless specifically requested just so I won't be accused of trying to drum up business using Slashdot. They are not the most expensive media. MAM-A (formerly Mitsui), I believe, has the highest prices. They are certainly the highest priced media I've seen recently. That said, I find them to be the highest quality media I can find. Taiyo Yuden is not bad by any means and is certainly the best quality I've ever seen in a store but still not the same as MAM-A.

Re:I agree... But where can I find some?

[Anonymous Coward]

"cheap-looking colored ones"

Racist bastard. ;)

Re:I agree... But where can I find some?

[nogginthenog]

Actually I've found that for the PSX media quality isn't the main problem. It's the weight/thickness of the disc. My PSX reads 'backup' CDs much better if I pop one of those transparent things you get on the top of a spindle on top of the CD. Not that I use the PSX anymore...

Re:I agree... But where can I find some?

[runderwo]

Speaking of Verbatim, some of their other discs are made by Mitsubishi

Considering that Verbatim is a division of Mitsubishi Chemical, that would seem to make sense, no?

Re:I agree... But where can I find some?

[mobby_6kl]

>Racist bastard. ;)

Yeah, and I'm not ashamed of it! I like my discs like my women: white and... printable.

Re:I agree... But where can I find some?

[alexo]

Taiyo Yuden All the way. They are great for copying PSX/PS2 games (seriously) where media quality makes a difference between burning out a laser or playing your back-up game, as well as DVD Movies.

The only drawback is that you can only order them from the Internet. I do not know of any retail store who actually sells the brand outright nor do I know of any brand (like Sony, Memorex, Fujifilm) who sells rebranded Taiyo Yuden discs.

Also, the Taiyo brand is more expensive than any other brand.

Recently, Taiyo Yuden disks have serious bonding problems.

Here's what the "Digital Dolphin" recommends:

Maxell BQ Series 8x DVD-R
Emtec Gold 4x DVD+R
(these 2 may change places, but are close)
(big quality gap)
Verbatim Ultralife Gold 8x DVD-R (maybe? jury is still out on this one)
Maxell Plus Series (made by Maxell)
Verbatim (Made by CMC)
Verbatim (Made by MBIL)
Taiyo Yuden
Verbatim (Made by Prodisc)

Not likely to ever go on the quality list:
Ritek/RiData DVD-R
Prodisc DVD±R
Princo
pretty much anything made in China or Hongkong

Re:I agree... But where can I find some?

[interzone]

www.mediasupply.com has Mam-A (Mitsui) and Taiyo Yuden. I've used them for two years and have been pleased. It's been my experience, though that Mam-A knocks Taiyo Yuden out of the "most expensive" slot.

Not a concern with MY optical media

[loimprevisto]

I have some movies on laserdisc that're pushing 20 years, and I haven't had a problem with them yet!

Re:Not a concern with MY optical media

[ericdano]

Yeah. Me too! Though a couple have laser rot on them. :-(

But, I really wish I still had the double sided player. It died. I still have a single one though.....

I have some disc in my studio, on the wall, and people always ask about them......

Re:Not a concern with MY optical media

[Anonymous Coward]

How'd that get modded +2 when the article is about media you burn, not factory-pressed media?
That guy actually burned his own laserdiscs?

Re:Not a concern with MY optical media

[Anonymous Coward]

The nice thing about Laserdisc is that the company that made them actually had pride in their product. The technology isn't perfect, so the medium had to be perfectly created. They were made for home theater enthusiasts in mind, not mass distribution as DVD was.

I still prefer Laserdisc movies to DVD, particularly for action scenes. DVD seems to get choppy at times, something I have yet to see on LD.

Re:Not a concern with MY optical media

[loimprevisto]

Well, I guess it would be possible... althought I confess I didn't actually make any of my own.

Check out http://www.laserdiscarchive.co.uk/laserdisc_archiv e/pioneer/pioneer_vdr-v1000/pioneer_vdr-v1000.htm

Re:Not a concern with MY optical media

[greed]

And I have a commercially-pressed DVD that failed 2 years after I bought it. So did a friend's. And so did the rental store down the street....

Now, that's only 1 out of 1000....

DVD-R media, though; I've got a bunch of early 2X and 4X discs from back when that was pretty new that were unuseable after 6 months. No drive I could get near (and that's at least a dozen) would register that media as being loaded.

I'm not sure about the current state-of-the-art for no-name media; I've been using Verbatim a lot recently. (Mainly because of some crazy-good sales Staples had a year ago; I've got _lots_ of Verbatim blanks now.)

For el-cheapo CDs, one thing to watch is lack of protection on the recording layer: I've got some from Rat Shack that you can peel the recording surface off with a Post-It note.

Re:Not a concern with MY optical media

[mrchaotica]

Yeah, but they're analog. When they do have a problem, how will you know? It'll just look like a slightly degraded picture or sound, which you might either not notice or might attribute to the player.

Re:Not a concern with MY optical media

[maxume]

Did you make them yourself, with $500 worth of hardware? Pressed cds tend to age quite a lot better than anything that is burned.

Re:Not a concern with MY optical media

[Detritus]

I recently ran across some commercially pressed CDs that had been ruined by water damage. Both the protective coating and metal layer were missing on large sections of the discs. The pits may still be there, but without the metal layer, they're unreadable. Many similar pressed discs were 100% readable after being cleaned.

Re:Not a concern with MY optical media

[maxume]

Typo? Similar burned cds?

Re:Not a concern with MY optical media

[Detritus]

They were all commercially pressed CDs from major labels. It was one boxed set of three CDs that suffered major damage to the protective layer and metal layer.

Re:Not a concern with MY optical media

[maxume]

Ah. I thought maybe you were contrasting pressed/burned, but it appears you were just pointing out that pressed have plenty of issues. I've had the reflective layer of a cdr that was simply stacked with other discs peel off after ~18 months, so I have some pretty serious trust issues there; I have yet to have a commercial disc fail, but I am pretty careful, and don't understand how discs that people occasionally hand to me get the way they are.

You don't get

[sa1lnr]

direct sunlight in your parents basement. ;)

Re:You don't get

[NewWorldDan]

Or, to summarize the article, treat them the same as you would your comic books!

This bears repeating

[Daishiman]

Repeating again and again and again:

For backups and archival you need tape backups, stored offsite. If you want something with more capacity and faster recovery, a backup server with rsync and beefy hard drives. Nothing else will do. With the time and effort you'll spend searching and writing DVD media you could have already bought and set up a file server or bought that tape drive.

Unless you're going to be taking those backups with you and using them in high volume, backing up to DVDs is simply a waste of time and space, and when you get some dreaded CRC errors you'll be crying for not having done otherwise.

sig: Cosas varias de un sysadmin argentino: http://aosinski.phpnet.us/

Re:This bears repeating

[nine-times]

What you're saying is probably true enough if you're in a business, in a position to have a file server and tape drive. But now let's imagine you're a home user with a laptop. You already have a DVD+-RW drive built in to your laptop and a 2 GB of personal data you want to archive. What are you going to do then?

Re:This bears repeating

[TheoMurpse]

If you want something with more capacity and faster recovery, a backup server with rsync and beefy hard drives.

Correct me if I'm wrong, but doesn't rsync pretty much fail for binary transfers? I was under the impression that rsync is really only useful when you have version X of something on your offsite server, and want to update it to version Y. Thus, rsync is great for logs, source code, or other text files, but is not really any more useful than

if FILE not in OFFSITE_DIR:
      cp FILE OFFSITE_DIR

Re:This bears repeating

[mrchaotica]

What are your thoughts on removable hard drives? Some of us [home users] don't want to spend the $2000 entry fee to get into tapes...

Re:This bears repeating

[Ucklak]

That realy depends today.

If you have a large organization and you're backing up terabytes+ daily, then yes with incremental. You can probably afford the $37,000+ for a TB storage solution.
A good bit of small businesses really don't have more than a couple hundred gigs that need to be backed up and the nightly stuff is probably under a gig unless you're in the media business.

Tape backup for archival is a horrible solution. You're dependant upon the media and the media player and in the case of Microsoft, the OS as well. BackupExec issues anyone?
How many of you had that backup you made to tape take forever to retrieve where that happenstance copy you made to another server work just fine?

Tape backup for `backup` is a fine solution but not for long term storage.
When marketing companies are finished with projects and they need to be removed from the server, they are archived on CD/DVD media.

If is probably cheaper to buy about 20 500 Gig Hard Drives and run a backup scheme using that media than to spend $2,000 on a 120Gig DLT tape drive that will take 6 hours to make a backup.

I'm not really discounting the usefulness of tape backup, I just question if it is relevant today when disk storage is cheaper.

My .002 cents.

Re:This bears repeating

[TheoMurpse]

Sorry, I meant to finish that last sentence with:
"...for binary files."

Re:This bears repeating

[XSforMe]

"For backups and archival you need tape backups,"

I agree when it comes to backing up, but not necessarily on the archival issue. Whenever you are planning long term archival, your best shot is a nice laser print out on acid free paper. If this is simply not possible then your second best choice will be a media which you suspect you will be able to read in the next 25 years, and in this case my money is on CD/DVD rather than on a particular type of tape.

Try getting now a days a QIC tape drive which will read a 20 year old tape (QIC-40). You might be able to find a used tape drive in ebay, but I can assure you computer tapes do get eaten up, specially when the tape drive is not under pristine conditions. Sending the tape to an off site recovery center might not always be feasible (either due to the urgency of the request, data sensitivity or simply because there might be no service which can handle your format). Even now, you are better off with 3.5 inch floppies rather than QIC-40 tapes. Further more, don't even get me started on the issue that will be getting the device drivers to work on modern day hardware.

To cover your rear against data loss on unreliable media such as CDs / DVDs do make a second copy of the volume. Seriously, it takes 5 minutes and almost guarantees you that the data will be available no matter what. Store them separately, and you will be set.

Re:This bears repeating

[CCW]

You are wrong. rsync works fine for binary files. Where it falls down (a bit) is on compressed files, as it has to transmit the entire file for minor changes. This can be mitigated on the upstream side by using the rsync-friendly gzip patch, which resets the compression periodically midstream at a small cost to total compression size.

Re:This bears repeating

[TheoMurpse]

Ah. You are correct, sir. One thing I erred upon is that I was thinking about rsync as a way of updating text files and as a way of updating completely changed multimedia files. I wasn't even thinking about the fact that if you add a scene to the end of an AVI file without changing its framerate, etc., merely a little metadata at the beginning of the file and then the added scene streams at the end (if there is audio-video interleave). This would have been the correct analog(ue? I'm from the US -- which is correct?) to updated log files.

Re:This bears repeating

[WuphonsReach]

Also, some version control systems only store the diffs of binary files (such as Subversion).

Which makes it efficient enough that we don't worry much about what we store in our Subversion repository (as opposed to older, dumber systems that stored a fresh copy of binary files for each commit).

Re:This bears repeating

[Anonymous Coward]

For backups and archival you need tape backups,

No.

For reliable 50+ year archival, you should consider UDO. 30GB per disc (15 per side) with a garanteed life of 50 years.

I have CDs good since 1998

[scottsk]

I now have some no-name-brand CDs burned in 1998 that are still good. I have never had a good, name-brand CD fail for any reason. The only failure I have ever had was the top layer peeling off some el-cheapo CDs which were stored in plastic sleeves, not jewel cases. One BIG key the article does not mention is to store the disc where the burned surface is not touching anything, such as in a jewel case -- the article should have mentioned that. Do not put in plastic sleeves or cases with slide-in sleeves. Odd that the article is a sales pitch for that T-Y brand -- what about RiData? That's what I use for DVD archival storage. I haven't been using DVD-R long enough to comment on how long they'll last. I have always found the alarmist idea that CDs will spontaneously self-destruct to be sort of over-the-top. CDs seem much more reliable for archiving than any other medium like diskette, hard disk, USB flash, or tape. Flash is more reliable, but has to be refreshed or it will disappear.

Re:I have CDs good since 1998

[slamb]

I now have some no-name-brand CDs burned in 1998 that are still good.

Probably using the original dyes, then? According to the article, they are most likely to fail in 2008:

The first organic dyes, designed by Taiyo Yuden, were Cyanine-based and, under normal conditions, had a shelf life of around ten years; simply, that was simply unacceptable for archive discs.

These people are talking about serious long-term archiving, not "it worked for this one guy for eight years".

I haven't been using DVD-R long enough to comment on how long they'll last.

No one has successfully used them as long as these people are talking about; they haven't existed that long. The lifespan claims are made from an understanding of chemistry (theory) and accelerated aging techniques (experiment).

Re:I have CDs good since 1998

[Valacosa]

RiData sucks. Seriously, I'm not trolling, I've been backed up on this. Not to mention I've personaly had RiData discs fail on me within a month of burning, their double layer DVDs are good for nothing but coasters, and I have an entire spindle of RiData CDs with visible defects on them. How the hell did that get past quality control?

For me, it's Verbatim all the way.

Simple solution

[NecroBones]

The best solution is not to use burnt discs for archival purposes. Magneto Optical would be ideal, but has never caught on.

Re:Simple solution

[VGPowerlord]

I prefer Professor X Optical myself.

Re:Price is all that matters in the short run

[ericdano]

Hmmm, I've burned many, many DVDs, and I have had problems with about 2 per 100. I always verify after burning.

my experience: some DVD media dies with no reason

[Anonymous Coward]

One Maxell DVD-R I burned in Sept. 2003 went bad within 3 years, despite every detail of the burning, readback, handling, and storage being in accord with the advice I've seen posted. An email to Maxell support on this issue had the reply: "The media if stored properly will have a life of at least 50 years."

Possibly relevant, I noticed an internal pattern of small spots visible with a loupe or macro lens (on order of 10 microns in size; much larger than the data pits). You can read more about it here: http://www.bealecorner.com/trv900/DVD/Maxell-DVDR- spots.html

Maxell America agreed to take back this DVD for analysis. As instructed I sent it to their Fair Lawn, NJ site. It was received Oct. 5 2006 and Maxell acknowledged receipt. They have apparently done nothing with it since, despite several emails to them in the ensuing two months.

Long term storage

[jo42]

The only way to get your stuff to last more than a generation is to chisel out yer 0's and 1's on clay tablets...

Age...

[Anonymous Coward]

There was a different technology ten years ago, and they have gone to less expensive technology today. There are legal standards for banks, etc., retaining info on CD's, and the bargain stuff, from what ever source, won't cut it. Banks have to have something like no loss for 8+ years. I have been told to buy nothing but made in Japan if it was important, and only the gold made in Japan stuff if it was really really important (family pictures, etc., that you may not go to the vault for for a long time, assuming you keep a lot on the hard drive somewhere). Ripping audio CD's, go with the best price from Taiwan manufacture. Gold color may or may not be gold, you have to look at both sides of the CD. Some (not all) of the made in Mexico and otherplaces stuff is rated for data loss in less than 2 years (typically 18 months), although the error correction will probably cover you for twice that long, maybe longer if you take care of the audio CD and/or got lucky with brand if not made in Japan or Taiwan. I found gold made in Japan CD's available at Walmart of all places, (don't waste your time at OfficeXXX), and have a small expensive stack for archiving precious stuff, and I buy decent brand name CD's that are made in Taiwan otherwise. Also, don't count on getting good results if you bottom fish the market for burners. I haven't burnt any DVD's yet, and haven't read anything about them either, so cannot comment.

Sorry about the anon.

I use DVD+/-R over CD-R for archiving.

[Vellmont]

The reason being that DVD+/-R has the recording surface sandwiched between the two layers of plastic. CD-Rs have the recording surface on top, which can flake off unless you handle it very carefully.

Sure, you can handle the CD-Rs carefully and avoid this problem. But wouldn't you rather use a more reliable medium in the first place?

Re:I use DVD+/-R over CD-R for archiving.

[Anonymous Coward]

WRONG! It's the same for both: data sandwiched between plastic and reflective layer on top.

Re:I use DVD+/-R over CD-R for archiving.

[evilviper]

CD-Rs have the recording surface on top, which can flake off unless you buy halfway decent CD-Rs, which have always included a thick coating on top of the discs.

Fixed that for ya.

Patrick got his bits wrong...

[Anonymous Coward]

This method is called 'pits and lands', where pits 'absorb' light (ie, are 'off' bits) and lands 'reflect' light (ie, are 'on' bits). This is incorrect - CDs record data using the transitions between light/dark and not the light/dark itself: see bottom of page http://www.digitalprosound.com/Features/2000/Sept/ RecCD2.htm

Re:Patrick got his bits wrong...

[dotgain]

Well, if you're going to be that pedantic, the data still goes through an encoding layer called "Eight to fourteen modulation", which guarantees that a pit / land will be at least x bits and no more than y bits long, so the laser won't lose tracking due to not finding any transisions in a long time.

Re:Price is all that matters in the short run

[Fallingcow]

Yeah, I get non-error-producing (in the burning process, I mean) misburns often enough that I ALWAYS keep the "verify burned data" option checked.

Takes way, way longer to burn a DVD that way, but it's worth it.

MediaSupply.com

[Artifice_Eternity]

I order Taiyo Yuden DVD+Rs from MediaSupply.com. $25 for a spindle of 50.

Inkjet Printables

[Amazing Quantum Man]

Side comment....

What's a good inkjet printer for printing on printable CD/DVD media?

Re:Inkjet Printables

[YrWrstNtmr]

What's a good inkjet printer for printing on printable CD/DVD media?

Epson R200/220/300/340 series. I have the R300, and can't say anything but good about it. Prints look like it came from the factory.
Epson has a refurb R340 at their outlet center for $70.

Re:Inkjet Printables

[bill_mcgonigle]

What's a good inkjet printer for printing on printable CD/DVD media?

I'm using an Epson Stylus Photo R220. $89 or so and very good quality.

If you're using a mac, e-mail me for an explanation of the right steps to open the printer doors in so the print driver doesn't drive you up a wall.

Re:Inkjet Printables

[sane?]

Canon ip3000

At least if you are outside the US, the Canon's are probably the best at printing on CD/DVDs. Not sure what the current models are, but I've been using mine for ages, no hassles. The major advantage of the Canon models was always that you could use refills, no hassle. My cartridges cost ~£1 a shot.

Unfortunately in the US, they blocked off the feed slot for the CD/DVD tray, because they didn't want to pay someone a fee.

Re:Inkjet Printables

[WuphonsReach]

Aye, I have the Epson Stylus Photo R200 and it's worked well. It uses separate cartridges for each color (6 total, including black).

The only issue that I've had with mine is that the CD tray will not automatically feed into the unit. I have to stand there putting a little inward pressure on it to get it to feed properly. Once it feeds, it's good to go and works just fine.

just upgrade...

[7macaw]

I just "upgrade" important stuff to the new media: take CDs and reburn them on DVDs. When the successor of DVDs appears, I'll move the data there. This way I don't worry about a) degradation of the particular media and b) non-compatibility of the old media and the new hardware.

Re:just upgrade...

[madhatter256]

Thats exactly what I did a couple of months ago. I took about 60+ CDRs filled with data that I've backed up in the past 4 years and put them all on DVD+R. Now I onlly have about 10 DVD+Rs instead of 60CDRs.

But yeh. Not only as long as you 'upgrade' to different, newer media, but also diverse your methods of backing up, like printing out documents, printing photos, etc. As long as you do that, you can expect to keep your data for a full lifetime. NOw if you really want it to last forever, then what you do is, store it all in a lead container filled witn Nitrogen and have 10%humidity and in 40degrees Faren. That will surely last for thousands of years. All that matters is if they have the equipment to read the data hundreds of years from now.

Re:just upgrade...

[biocute]

All that matters is if they have the equipment to read the data hundreds of years from now

That's why I made sure my lead container has a DVD-ROM in it.

How To Choose Archival CD/DVD Media

[rjdegraaf]

... 'burnt' CDs/DVDs can eventually 'fade,' due to five things that affect the quality of CD media.

So the answer is MU.

Verbatim DatalifePlus 8x MCC 003

[blckbllr]

Here's my two cents:

Stay away from Taiyo Yuden 16x media. I'm using a BenQ 1620 for all my DVD burning needs, and the PI/PO tests done on T-Y 16x media using DVDInfo Pro have always resulted in low quality burns. I am currently using 8x Verbatim DatalifePlus DVD+R media, and burning them at 4x. The results are truly unbelievable. The media code on the 8x Verbatims is MCC 003. I've heard through the grapevine that T-Y changed their media somehow from their 8x sets to their 16x sets, which has resulted in that the 16x DVD+Rs aren't as good. If you can get your hands on 8x T-Y DVD+Rs, then go for it; otherwise, stick with the Verbatim DatalifePlus series.

-BB

Re:Verbatim DatalifePlus 8x MCC 003

[Anonymous Coward]

MCC 004 is even better with my burner (NEC 3520). Incredible quality burns. PIE doesn't go over 40-50, PIF gets 1-2 sometimes, and I've had burns with 5-10 PIFs total on a full DVD.

Re:Verbatim DatalifePlus 8x MCC 003

[blckbllr]

Yeah - I worked with MCC 004 beforehand. I believe that's either the 12x or 16x Verbatim DataLifePlus DVD+Rs (I don't remember which). All in all, I've been happy with the DataLifePlus series.

-BB

Still room for DVDs

[Kadin2048]

While I appreciate the sentiment, there are still lots of valid reasons for wanting off-line storage in the form of DVDs.

Consider the cost. Hard drive space is, at best, around $0.25 per GB? Ignoring the cost of the infrastructure you'd need (servers, RAID cards, etc.) to keep them running, that's still about 4x the cost of decent DVD media. (If you follow TFA's recommendations and go with DVD+R, about 2x.)

I've been doing a lot of slide scanning recently, basically producing 3200 dpi x 64 bpp (64 because it's RGBi) TIFFs from slides. Each one is 100MB or so, and there are two per image (one is the raw scanner data, the other is color-corrected). It would be a waste to store all that on drives -- I'd need well over a TB RAID array -- it makes more sense to burn them to DVD and keep downrezzed, compressed versions in on-line storage. Actually, I keep two copies; one gets put with the slides and stored with them when they get reshelved, and the other copy gets put in my CD rack with other backups.

Could you do this with a remote server and rsync? Sure (and in fact, I do have a rsync setup for other documents); but it would be atrociously expensive, not to mention a colossal use of bandwidth. You'd need two very large RAID arrays, and all the interface cards they'd use, plus the server itself, plus electricity...those are all constant expenses. The rack of DVDs doesn't cost me anything (the opportunity cost of the floor space it requires is minimal). 'Scaling' a DVD-based offline storage system is similarly simple; you just add another DVD to the rack.

I would certainly never recommend that someone use a DVD-based system for storage of frequently-changing documents, but for large quantities of data there are still lots of applications where they are economically and logistically the best option.

Re:my experience: some DVD media dies with no reas

[SydBarrett]

Maxell is the only brand of CD-RW that I ever had problems with. I had 3 from a pack of 5 die after the 1st erase, And a Dual Layer DVD-R produce a pretty expensive coster. The only other brand to always mess up was some no-name junk that an old employer had sitting around for free, can't remember the name of it but had a ugly white and tan label, I think it was Computer Peripherals.

The only other CD I know that is not working is a TY disc, ink printable white label burned about 1 year ago. No scratches at all on it, but always skips no matter what player I use.

Re:Simple Solution - Stay Away from the shit

[Anonymous Coward]

The storage density of your alternatives are pathetic. Given our technological advancements, people should expect reliable storage densities of at least 1 MB/mm^3.

DVD media quality guide

[noky]

I have found this dvd media quality guide to be extremely informative. Yes, Taiyo Yuden is always ranked at the top (and is what I use), but they are not readily available at local retailers. It really helps to have a detailed comparison of various media instead of just saying "brand X is best".

Re:DVD media quality guide

[PhotoJim]

If local retailers won't sell me the products I want to buy, I'm more than happy to mail order them. I agree that MCC media is extremely good, though.

Only a lifetime solution

[Valacosa]

See, the problem with that is it works only so long as you're around. I think the point is that the archives are supposed to be both zero-maintanence and they're supposed to outlive you. Who's going to push your bits around when you're gone?

Re:Only a lifetime solution

[7macaw]

>Who's going to push your bits around when you're gone?

I don't see why I should care :)
But seriously, if my children think that antique porn and pictures of mom's and pop's adventures in Grand Canyon are important enough, they'll keep rewriting them on their uber-cool storage media. If not -- well, I'm dead anyway.

Shoddy Article

[Anonymous Coward]

TFA isn't really well written. The conclusion is mostly correct, but some things really put me off:

With pressed media, the pressing method causes pits to reflect the laser's light away from the sensor, and the lands to reflect it back at the sensor.

Actually, the pits have a depth of 1/4 wavelength of the laser, so that the light that is reflected from the bottom of the pit travels 1/2 of the wavelength longer and cancels out the light that is reflected from the land.

roughly .0000000038th of a second

That would be about 263157894s. or about 3046 days - more than 8 years.
Additionally, I really don't get his argument about the ATIP and wobble. If your DVD-R has degraded so badly that you can't read the ATIP before burning it, you probably don't want to use the disc for long term archival anyway. He goes on about the error correction of the data in the ATIP, but as far as I know, the ATIP is only used to determine the recording strategy, and should be of no relevance to reading the DVD - after all, the relevant part shoud follow the same standard as pressed DVDs so that the "new" DVD-R and DVD+R media are compatible the old drives that predate the recordable media. The same compatibility argument holds for the encoding of the ATIP data itself - if it differs from the pressed DVDs, it can't really be important for reading the medium.

Taiyo Yuden & Japan all the way

[Venner]

Yes. There is a program out there that identifies a disc's manufacturer, and all the Japanese Fuji's I've bought have been Taiyo Yudens. Consequently, they're all I buy. I've never had one fail yet (and have had lots of other brands fail. (Such as, oh, about every other disc made by CMC Magnetics. Horrid.)

I've also had good luck with the Verbatum "VideoGard" line. (DVD Identifier says the batch here on my desk is made by Mitsubishi Chemical.) Their particular gimmick is a hard, scratch-resistant coating which is particularly helpful for high-use media. (i.e., non-archival.)

Whenever I archive my most important data (such as all of our old family movies) I create two copies on different-branded media for safety. And then I make a duplicate set to store off-site (safety deposit box.)

Re:Taiyo Yuden & Japan all the way

[dangitman]

Such as, oh, about every other disc made by CMC Magnetics

I think that would be expected when they try to make a magnetic medium work in an optical drive. The magnets, they do nothing!

I have had Imation CD-Rs fail...

[emil]

...when the label peels off. I don't know who the OEM was. I usually buy Sony CD-Rs now (and avoid Imation).

I have also previously read that DVD-R was slightly more compatible with most readers, but the error correction discussion below makes me want to switch.

The DVD-R specification states that for every 192 bits, 48 of them are not protected under any scheme, 24 of them are protected by 24 bits of parity, and the last 56 bits are protected by another 24 bits of parity. This weird (to put it mildly) scheme allows you to easily scramble or lose 25% of the data that is required to read your disk!... The DVD+R specification, however, states that for every 204 bits of information, it is split into four blocks of 52 bits containing 1 (shared among all blocks) sync bit to prevent misreading because of phase changes, 31 bits of data, and a 20 bit parity (that protects all 32 bits).

The secret to spotting Taiyo Yudens

[traindirector]

I've seen Taiyo Yuden CD-Rs and DVD+/-Rs in a number of retail stores under various brand names. I'm hesitant to publicize my trick, but I suppose the Slashdot community should know. Here's how to spot Taiyo Yudens quickly in the store, without checking each label for "Made in Japan":

The spindles all have a unique bottom lip. Whereas most plastic spindle coverings are the same diameter from the top of the spindle to the bottom, Taiyo Yuden cases have a "lip" on the bottom of the plastic covering that starts about an inch from the bottom. The bottom of the clear plastic covering sticks out just a bit and then recesses to the diameter of the rest of the spindle. Taiyo Yudens comes in these cases no matter how they are branded, and I have never seen a spindle of discs with this bottom lip that are not Taiyo Yuden. I guess Taiyo Yuden supplies the plastic spindles as well as the branding on top of the disc.

In any case, I have had better luck with the consistency of Taiyo Yudens than any other brand of DVD+R. I'm not sure what the case is now, since I've only been using Taiyo Yudens for the past few years, but when DVD recording was first becoming affordable, the compatibility of much DVD media with various recorders was so terrible as to be useless (and endlessly frustrating). Taiyo Yuden makes quality discs, and it's always nice to spot them in the store when there's a deal going on.

Re:The secret to spotting Taiyo Yudens

[drwtsn32]

Not sure your method is completely accurate. I have a spindle of DVD+R media from TDK with the lip you describe. The media is indeed Taiyo Yuden. I then checked a spindle of CD-R media without the lip. Nope, not Taiyo Yuden. So far your method looks good! But I have another spindle of CD-R media (Maxell) with the same lip as the DVD+R TDK media and Nero shows the disk as being made by Ritek. Damn!

Re:The secret to spotting Taiyo Yudens

[resistant]

The trick I used a few months ago to spot Taiyo Yuden inside 50-pack spindles of Sony DVD-R diskettes was to look for octagonal hub-retainers (if that word's correct). Checking for "Made in Japan" served as a confirmation.

I forget now the number that was supposed to also be printed on the hub ring, to be spotted with a tiny LED flashlight. (All eight spindles had that number).

Out of direct sunlight, cool dark place

[blantonl]

please keep all media out of direct sunlight, in a nice cool dry dark place, in acid-free plastic containers; this will triple the lifetime of any media

Well, it's good to know that 95% of slashdotters are already following this practice by inherently storing their media close to themselves, next to their computers.

Re:Out of direct sunlight, cool dark place

[Anonymous Coward]

Dry? I've seen one of those basements... like the inside of a stillsuit, yuck. I didn't even know that a monitor could get a yeast infection.

What about DVD+RW?

[pestilence669]

There's no dye fade with RW formats. He didn't even bother to touch on rewritable media. I imagine they'd be much more resistant.

Re:What about DVD+RW?

[TeknoHog]

He didn't even bother to touch on rewritable media. I imagine they'd be much more resistant.

He did mention briefly that RW media are out of the question for archival purposes.

To make a completely unscientific point, I'd imagine there's a reason for having different technologies for R and RW. If RW media outlasted R, why would there be R media at all? For example, many audio CD players don't play CD-RW, but they play CD-R perfectly. This should tell something about the relative merits of the media.

Re:What about DVD+RW?

[Anonymous Coward]

I agree. The article seemed to associate rewritability with impermanence.
Changing the state of silicon requires significant energy - I've seen several
+RW brands including sony advertizing the archival properties of their products.

Re:What about DVD+RW?

[pestilence669]

I came across this:

"One important difference between R and RW media, however, is that the re-crystalized alloy of RW media will gradually decrystalize, taking the data with it" (Wikipedia)

Re:Simple Solution - Stay Away from the shit

[Anonymous Coward]

Speaking of which, here is what you can do fucktard.

Instead of using a fucking computer for your fucking letters, use a fucking typewriter.

Instead of using a fucking computer for your fucking research, use a fucking library instead.

Instead of fucking living, go fucking kill yourself fucktard.

Re:my experience: some DVD media dies with no reas

[Anonymous Coward]

What DVD?

-Maxell

TY fake disks

[goombah99]

How do I actually buy TY disks? When I try there's so many counterfeits on the market that I don't know how to assure myself I am getting these. The problem gets far worse when one wants to minimize the price one pays so one is looking at on-line discount office supply companies without the reputation and high prices of the big chains.

Re:TY fake disks

[Hatta]

Can't you check the ATIP? Or do they fake that too? I get all my TY media from American Digital, and have never had any problems.

Re:TY fake disks

[RKThoadan]

You could always RTFA, since he addresses this very problem, but I'll be nice and save you the trouble. He recommends SuperMediaStore.com as the only store that guarantees thier TY is genuine.

Re:TY fake disks

[WuphonsReach]

Don't know if the ones I bought were counterfeit, but at least they came from MWave. (Where I'd think it would be unlikely that they'd sell grey-market goodes.)

MWave Taiyo Yuden

I tend to buy the 100/pack, tape-wrap, inkjet printable. As long as you have some empty cake boxes around to transfer them into, the tape-wrap style works fine.

Very interesting read

[Endo13]

However in all the hundreds of CDs and DVDs (both -r and +r) that I've burned I've never had reliability issues, with any type or brand. Of course I'm sure this is partly due to the fact that none of my data burnt to optical discs is 'archival', and generally only sticks around for a few months, or a few years at most. So for my purposes, I tend to just watch for those rebate deals that leave me with just the sales tax to fork over.

DVD/CD Storage Cases

[eieken]

No mention of these yet, so thought I'd let everyone know. The DiscSavers cases are great for fragile DVDs. It keeps them safe, and gives you easy access to them. Try dropping one of those on the ground and you'll see it doesn't result in scratches and broken cases. Highly suggested for anyone planning on storing any burned DVDs for any length of time.

Again?

[jbrader]

Is it just me or does this subject come up once a month or so?

par2

[MMC Monster]

I create par2 files of all the videos and other large files I burn to DVD. It takes a while, but I know that I can have a 30% failure rate and still generate all my data.

Maybe someone could create special hardware to make par2 take a reasonable amount of time?

Re:par2

[springbox]

dvdisaster does the same thing basically but is easier if you intend on making error correction codes for entire discs.

Re:par2

[skidrash]

You put the PAR2 files on the same disk?
If not, how will you ensure your disk-and-par2 management scheme will work for the long term?
I've found that after a move or 2 lots and lots of stuff - paper and CDs and backups ends up as a confused muddle.

Re:par2

[MMC Monster]

Actually, I do keep the par2 files on the same disk. I figure that if I have a failure, it's going to be a read error from a random sector. So long as I can read the vast majority of the files (say, 70%), I'm fine.

Of course, if a disk gets a read error in the sector holding the FAT, I'm toast. :-)

Re:par2

[WuphonsReach]

You put the PAR2 files on the same disk?

Depends on what I'm archiving to DVD-R.

If it's stuff that I found on the web, and could probably dig up again if I had to, then it's no worry to burn a single copy and put 10% PAR2 data on the same disk. From my experience, the odds are very slim (1:500), that I will encounter errors trying to read the disk after a few years. And even when I do encounter errors, I can put the disk into a 2nd machine and pull the missing bits off. Or else, I only end up with about 1% damage which is easily repaired by the 10% parity data that I put on the disk.

(Again, the whole advantage of parity data is that it gives you a recovery window. If you test regularly, at a higher frequency then the disk is failing at, you can catch errors early enough to repair. If you want a lower frequency of testing, then you should put more parity data on the disk. Maybe 20-25% of the disk devoted to parity data. If you are less concerned, you can get away with 2-5% parity data.)

For more important stuff: That always ends up spread across multiple archival snapshots before it is permanently removed. For example, I'll backup up the previous 6 months of data every month. Which means that a single piece of data should end up on at least 5 of the disks. The PAR2 data on each disk is more of a time-saver and double-check. Just enough PAR2 data to recover from minor scratches / blemishes and I can use the PAR2 data to double-check the disk health.

If I'm doing a very large snapshot (say when I image my laptop HD) and don't want multiple copies... Let's say I have 8 disks worth of data in the snapshot. I'll spread it across 10 disks. The 10th disk will be 100% PAR2 data. Disk 1-9 will be 89% data and 11% PAR2 data. Which I think works out to 2 disks (25% parity) worth of PAR2 data. I can lose an entire disk out of the set without worrying.

Re:par2

[WuphonsReach]

Of course, if a disk gets a read error in the sector holding the FAT, I'm toast. :-)

Maybe not. PAR2 files store the filenames as part of the recovery data. As long as the TOC track (innermost track) isn't kaput, you can recover the data even if both the UDF and older 8.3 file tables are blitzed. If the TOC is busted, you'll have to get a professional DVD reader or go to a recovery service.

(The "how to" is over on the QuickPar forums. Basically, you rip the disk at the sector level to a pair of files. Rename the one with a PAR2 extension, then feed it the 2nd file as source data. QuickPar will find all of the data blocks and reconstitute the original files.)

Why CD/DVD

[really?]

There was a time, years ago, when CD/DVD archival made sense. These days when the HD prices are so low, I just don't see the benefits.

I have a box sitting in my friend's "office" in a different country - could just as easily be in a friend's house in a different town ...as long as it's not in an area that is likely to be hit by the same disaster as your home.
Currently, in addition to the drive the OS is on, it has a mirrored pair of 700 GB drives - this is enough for _MY_ current needs. It holds compressed ISO images of CDs/DVDs - stuff I used to burn in the past.

I have an "identical" setup here - same amount of storage space, different OS and hardware; eggs ... basket > 1.

So, every time I would normally burn a CD/DVD I do so to an image which then gets copied to the local and the remote archive servers. Makes management WAY easier for me. For example, until recently I used to have 250GB drives in the mirror. I plugged in the two new drives, initialized the mirror, copied the data over, removed and re-purposed the older drives. WakeOnLan makes sure that the remote box is only powered when I actually need to push new data on to it; saves power and wear and tear. I find I only archive stuff about once a week or even less. This is NOT my backup server, it's the archive server, completely different purpose.

YMMV and all that.

Drive + Media

[Anonymous Coward]

A good burn is created by good media in a good drive. I like to visit http://www.cdfreaks.com/ before purchasing new burners. It's not I'm scared of losing $40 on a bad one. The reason I do it is the same one behind why I check the data twice once it's burned and make three copies; I'm a little paranoid despite the fact that I have CD's from 1994 that are still in perfect condition.

Verbatim MediDisc

[Two99Point80]

I got a batch of Verbatim MediDisc CD-Rs a few months back. (multiple listings for MediDisc at buy.com including DVD-Rs) Given their intended use - archiving medical data - I'm hoping they'll preserve my data well. FWIW they're made in India and feel thicker and less flexible than other CD-Rs I've seen. They ain't cheap and the spindles of CD-Rs are currently backordered.

The usual vague personal testimony...

[dpbsmith]

Oh, I'm so tired of these articles. Everyone concentrates on dye fading, because I guess it's easy to measure and quantify. If dye fading were the failure mechanism for these disks, they'd last twenty to two hundred years... according to vendors and researchers.

Everyone says "I've never had any trouble with brand ABC," but the thing is, ABC varies depending on what you read or who you talk to. Some people insist they've never had any trouble with the cheapest generic products they buy at Staples. Some say any name brand is OK. Some say Verbatim is good. Some say to stay away from Verbatim. The more sophisticated will tell you not to use anything but phtalocy- pthalocy- pffthal- the Mitsui stuff. Others (like this guy) are partial to other dyes. Some say you're a fool to use anything but Mitsui Gold... some say they're an overpriced waste of money.

It's all authoritative sounding talk, talk, talk and no two experts say the same thing.

In reality, I don't think anyone understands very well what actually causes these disks to fail in the real world. I've had disks fail in less than two years--maybe only a couple-three in many hundreds, but certainly not zero--and I've never seen any obvious pattern as to which of them fail.

The thing that really bothers me is that drives and/or their accompanying software drivers never give you any indication of what the signal quality of a particular disk is. If they did, you could detect that a disk was deteriorating before it failed, and make a copy. As it is, they just keep silently keep correcting errors behind your back and you have no warning until there is utter, catastrophic failure.

Re:The usual vague personal testimony...

[epine]

Have you ever heard of bait and switch? It's a deeply engrained trait in the human species. Under one set of conditions, such as not having much credibility to begin with, an organization will work very hard to establish the reputation of a product line. Then under another set of conditions--major stakeholders change chairs, new management team recruited, under a short-term cash-out-now incentive structure--all the expensive magic that made the original product good is discarded, and the newly watered down version of the reputable product continues to be marketed with no mention that it was changed at all.

Do you need that again in a short sentence? Brands suck. The brand is not the product.

I used to buy a lot of bottled pasta sauce under the Classico brand. No added corn sugar. Tasted like food. The tomatoes tasted like they were delivered to the factory on a flat-bed truck in the kinds of crates they picture on the front of the bottle. Then the situation turned evil. Some new brand manager decided to move the brand upscale. The best flavour of all, the plain onion and garlic, was replaced with a roast garlic that tasted like crap. Prices went up. The tomatoes began to take on the appearance of a puree. Now it is impossible to open a jar without thinking the tomatoes were delivered to the factory in a tanker trunk. The body of the sauce now conveys the impression "ultrasonically homogenized". All the bottles have fancier flavours and labels than ever before, and the price is higher for an inferior product.

This has nothing to do with experts whatsoever, and every to do with the human necessity of people needed to make themselves look good (e.g. in their role as the marketing director) at the expense of end result.

Brands exist to convey the message that you're still getting the good stuff long after the good stuff has taking the building with it, and the only left is debasement.

Re:The usual vague personal testimony...

[infolation]

You can obtain meaningful PIE/PIF/jitter (etc) error information using certain plextor drives which in include error scanning modes (e.g. the 716a) and software such as plextools professional or PXSCAN/PXVIEW.

Alexander Noe's site also includes error readings from the main brands of DVD+R/-R (including TY) for comparison to your own discs.

Use Lowest Quality DVD-R for Archiving

[RonBurk]

I highly recommend doing your archiving on the cheapest, poorest quality DVD-R discs you can find. The reason is simple: thinking you have high-quality media makes you complacent about testing and migrating archived data

Re:Use Lowest Quality DVD-R for Archiving

[skidrash]

If possible data loss is all I have to worry about, I won't test and migrate.

So, how about hiring someone to whack you upside the head with a 2x4 or an aluminum baseball bat if you don't do your testing and migration.

THAT'S motivation.

Attribution.

[Anonymous Coward]

I hope Patrick gave credit were it's due?

All this long-life media are worth nothing...

[Anonymous Coward]

...if you store the content in formats that wouldn't be read after 50 years. Will I be able to look at the pics of my youth in 2076 if they were recorded in some propietary format?

Gold DVDs do exist

[the eric conspiracy]

The author makes a statement in his article that gold layer DVD's are not feasible. In actuality they are offered by Misui / MAM-A in a couple of different grades. I'd much rather use them than Taiyo Yuden for important data.

Why the Guessing Game?

[Slugster]

Why not just order them online, and know what you're getting?
A 50-ct spindle pack costs about 25 cents per disk (plus shipping) as I've seen it.

...In any case, I have had better luck with the consistency of Taiyo Yudens than any other brand of DVD+R....

Taiyos do work well, but then--have you ever try Mitsui golds?
At roughly a dollar a disk for a 50-pack spindle, not many people have. The last tests I was (a year or so ago) MAM-A still edged out Taiyo in long-term reliability. ~5 years back there used to be issues with early CD-R drives not burning them well, but that isn't a problem with current hardware.

And yes, I do need to order them online, noplace local carries disks so expensive.
~

MOD PARENT UP!

[mrchaotica]

He speaks the truth.

Web Bug in Article

[Anonymous Coward]

The article body text has a 1px gif leading back to afcyhf dot com. Kinda funny to have it within the article paragraphs, rather than the regular page code. Check the end of section "Why Taiyo Yuden".

afcyhf WHOIS's as part of ValueClick. Most modern HOSTS files block it.

Risk

[fm6]

I have yet to run across a burned CD I cannot read due to this sort of degradation. Maybe at the decade mark, some of my discs will fail me and I'll change my mind but right now I'm not too concerned.

When a product has an expiration date, that's just a promise that it probably won't fail before then, not a guarantee that will fail. If your carton of milk expires on Thursday, it won't necessarily be sour on Friday. By the same token, your archive disks are probably OK well past the 10 year mark, but you have no way of knowing exactly when they will stop being readable. If you have any data on them you can't afford to lose, you should make copies before the expiration date.

Apropos of last week's flamefest about safe driving: this logic also applies to the risk you take every time you exceed the speed limit or tailgate. Maybe you've done it a thousand times without mishap — but the risk is still there. When you make a bet, the odds are not the only measure of whether the bet is a safe one. You should also consider whether you can afford to lose.

Printable media

[smoker2]

Never, ever use printable media for backups. It always lets you down. Nuff said.

way way way WAY WAY WAY better medium

[ILuvRamen]

one floppy sized holographics disc from InPhase is about $45 and holds 200 GB, probably more now, I haven't checked lately. The drive is like $3500 and only available commercially though :( But the read/write speeds kick everything's ass as well as the data density on the medium itself. They keep longer too. Every company should use them!

He DOESN'T recommend and disk for archival storage

[ffflala]

FTFA:

So, you're probably now wondering, in simple terms, what media do I recommend?

To begin with, I do not recommend CD-RW, DVD-RW, or DVD+RW media in any form for permanent storage. This is mostly a no-brainer, but those discs are meant to be able to be changed after burning, and they are simply unsuitable for long-term archival storage.

Archival storage media --if by "archive" you actually mean, you know, a real archive-- has to be last decades, even centuries. Tape media is the current standard for digital archival storage, and is probably not suitable for over a century: it's still too young to tell. With media an extra difficulty is the access mechanism, and it is very possible that while the tape may last several decades the tape deck will not -- ask NASA.

Ten years and no errors on some discs you've burned is nice, but that's not anywhere close to proving a suitable archival media.

Re:Price is all that matters in the short run

[toadlife]

"I just go to WallyWorld/ChinaMart and get me the cheapest 4.2 gig DVDs money can buy, "

That would be a great plan if Taiyo Yuden DVDRs ordered via the web actually cost more than the cheap-ass ones you buy at bigboxmart.

People like stores. It's not a guessing game.

[traindirector]

Why not just order them online, and know what you're getting?

Sometimes you need media immediately. Some people like to pick up media at the store. If you're going to be buying it at the store, why not get the good stuff, especially if it's the same price?

Additionally, it's not a guessing game once you know what you're looking for. If it comes in the Taiyo Yuden "That's" packaging and says "Made in Japan", it's Taiyo Yuden (unless it's a cheap imitation, which is still identifiable and not very common). People over at the media section of the CD Freaks Forum have discussed this to death for years. I will save you from hours of mostly non-productive reading by telling you that the packaging is unique to Taiyo Yuden, that fakes are not hard to spot and will not normally say "Made in Japan", and that most of the people on the CD Freaks forum, who have entirely too much time on their hands and enjoy obsessively testing CD-Rs and DVD+/-Rs with many official and hacked firmwares for dozens of recorders, agree that Taiyo Yuden is the best.

price / lifetime correlation?

[Anonymous Coward]

What gets me is that there is, in my experience, absolutely NO correlation between price and lifetime. The worst cd's I've ever encountered were expensive, brand-name, gold (I was young and foolish) cdrs... that started to show errors after just 1 year! The best, which I have *never* had a problem with, were relatively cheap no-names... burnt about 7 years ago and used semi-regularly since. Even the ultra-cheapo dodgy bulk ones I bought a while back, while not perfect, have on average lasted longer than the brand-names despite obvious discolourations.

It would be so easy to choose if expensive == better. Unfortunately it doesn't, so it isn't.

Re:my experience: some DVD media dies with no reas

[Anonymous Coward]

I have had bad experiences with Maxwell myself. I got a 25-pack of Maxwell DVDs and ten of them wouldn't even burn. TDK, on the other hand, is dirt cheap at Fry's (20 cents a disk in lots of 100) and I haven't had a single problem with them.

www.blankmedia.ca

[Frostalicious]

In Canada you can get Taiyo Yuden from www.blankmedia.ca

Re:He DOESN'T recommend and disk for archival stor

[vga_init]

He DOESN'T recommend and disk for archival storage

Actually, it says in the article that he recommends CD-R and DVD+R. He even mentions a specific brand (Taiyo Yuden). You will notice that his two recommendations are missing from his list of items he cautions against. That list does not imply that he does not recommend this type of media.

What I posted there

[Anonymous Coward]

I posted the following on the article's comments section:

I'm curious what you (the author) think of this link:
http://www.digitalfaq.com/media/dvdmedia.htm
I was always of the mind that TYs are the best, with no competition. The article above also cites Pioneer, Hitachi Maxell, and Mitsubishi Chemicals in their top 5 archival quality media. I got the link from Lifehacker and was surprised to see anything but TYs as da bomb.
Lifehacker link:
http://www.lifehacker.com/software/dvds/choose-the -best-blank-dvds-220373.php

Anyone here care to comment on the digitalFAQ.com article? I'm curious if anyone even halfway qualified agrees with them. Should I only stick with TYs or can I trust these other brands that they claim are as good?

Why bother?

[snoggeramus]

Why bother with discs anyway. Just email the data to your GMail accounts.

myarchive1@gmail, myarchive2@gmail .....

... And it's free!

I wish for a cheaper tape solution

[mlts]

DVDs right now are the most cost-effective way to backup/archive data. For backups, I personally use Retrospect (because it uses a packet writing format, does incremental backups, and offeres certified AES encryption). For archives, I use WinRAR with parity/recovery volumes, each around the length of the DVD, then burn each .rar or .rev archive to an individual DVD, so if one goes bad, I'm not totally cooked. I also use two different types of DVD burners and check if one burned in one will work in the other, and vice versa.

However, for a large backup set (100+ gigs), its a pain. Sometimes (and this using pretty much any media) one disk will just fail on verify, forcing me to set it as "lost" in the backup set, then redo another backup set copy.

Tape is great, but with modern capacities, you will not be achieving price/performance ratios worth your time unless you pay $500 or more for a new tape drive.. Even the old "tried and true" 75-80 GB DLT IV tapes are not worth it these days. A drive will cost you 200-300 used on ebay (of course trusting unproven ebay tape drives with irreplacable data is another issue), and you will need to find an old SCSI card to connect it to most machines. Then, for the uncompressed capacity of 10 DVDs (which cost $5 or so), you have to shell out $40 a tape.

A new tape drive that is minimally decent (A DAT-72, with 36 gigs native) will cost you $640 to $740 depending on your choice of internal or external, and tapes are $16.00 each. A solid DVD burner will run you $150, and blanks are 25 cents to a dollar depending on how much you want to pay. For 100 gigs of data, a DVD burner + media will set you back $162. For 100gigs worth of data on tape, you are paying $698. Of course,this scales better the more media you get, and this doesn't factor in the time repeatedly swapping blanks which is a big thing.

The best solution to bridge the gap in cost/capacity seems to be Blu-ray media once drives start coming down in price in the next couple years. Right now, the drives and media are too expensive (not mentioning the fact that they are so new that nobody knows how long their archival life is.) However once these come down in price, they should replace DVDs for data archival.

Taiyo Yuden FAQ

[Foresto]

Taiyo Yuden FAQ, for those who want guidance in finding these discs.

TY, Ritek and TDK and the Gamecube

[aliquis]

Well, here in Sweden you have to pay some sort of copyright taxes of media bought in Sweden and Taiyo Yuden (and many other brands) are much cheaper from example SVP in england. I can get Taiyo Yuden media there for no more or very little than whatever DVD-R in other webshops.

Another dye I've been told was good was the Ritek G04 and G05 since that worked in the Gamecube, I don't know if Taiyo Yuden does, the reason should be that the discs are more "perfect" than others I suppose, but I read later that they age very fast and the amount of errors can get high even after a few months so it might be a sad choice now or a year later. Anyone tried the Taiyo Yuden discs in Gamecube?
I only own a LG gma-4020b 2x DVD-R burner so I'll get a Pioneer DVR-111d or something aswell so if +R discs are better I will get them, but do they work in my Gamecube?

The same store also had TDK discs which was much more scratchproof than others according to the advertisment, I think the price was around twice as much as for the other discs but on the other hand they could handle much more uncare handling with the same amount of errors.
Good for people like me who stores some of their discs on the floor and such ;D

Burned verses Pressed

[KrisKlee]

Up to this time, I had thought of my burned CD/DVDs as a "permanent" backup or record. This article has changed my thinking, and while I still like to buy "pressed" music CDs and DVD movies, and even download music from iTunes, I will now think of them as more permanent and acknowledge that MY burned discs will have to be re-burned from time to time. I am using LaCie CD/DVD burner and have had little problems with it and found it to be most reliable. As time goes along, storage space, via ethernet, is getting cheaper and cheaper and will provide more hard-drive space for direct storage of music and movies. I am preparing DVD storage of about 14,000 book titles from the Gutenberg Project and make them available to schools and libraries and compatible to Macs. Query: Should I learn how to make these discs "pressed" instead of burned? I am doing this under the guise of Scriptorium Library(TM) and will pay a 20% royalty to Gutenberg should the project get off the ground. So far, I have enough titles for about four DVD discs (about 4.5 gigabytes each). Cordially, Kris Kleeberg

UDO Archive Appliance

[Anonymous Coward]

Why not just get a NAS that has RAID? That would make more sense. When a disc dies, you can replace it, rebuild your array, and everything is fine. PLUS, you could expand your archive over time.

How about NAS RAID and UDO all in one system, the Archive Appliance.

independence

[chucken]

Am I the only one to be suspicious about the impartiality of this article? Check out the links. Quite a lot go through the weird ass domain name JDOQOCY.COM. Do a whois on this domain and you'll find the registrant is "Commission Junction". Hmm, impartial, NOT.

All you need is a deity

[Sigg3.net]

I think the archiving method that is best in terms of historical proof and durability is oral memory.
Take the epic of Gilgamesh, for instance.

I read a post further up about writing 1s and 0s on stone tablets. That's step one.
Step two is taking them down from the montain, and three is convincing your cohorts that these tablets were in fact handed to you by a higher power.

This type of archiving has been found to be the longest lasting in human history; backed up in fresh copies for every new generation learning the recitations. All you need is a deity.

OK, so what's the difference ...

[constantnormal]

... between DVD-R and DVD+R media? From reading the referenced article, it appears that the difference between DVD-R and DVD+R is all in the encoding of the bit stream.

Clearly, that's not the whole story, as when I insert a blank disc, the software can tell me whether it's a DVD-R or a DVD+R.

From the info presented by Wikipedia, there is a series of physical pits between the grooves that is used for addressing and tracking purposes -- I suppose that there is also some sort of identification code that indicates which type of disc it is, but have no idea if there is any other significant difference between the +R and -R flavors of media -- such as types of dyes, or differences in the sizes/shapes of the dye cells on the discs. My guess is not.

Drives that measure quality

[dpbsmith]

All well and good, and thanks for the information, but why don't all drives do this?

And why don't the standard OS drivers push this information visibly up the user?

You know, like the little wires that are built into brake pads so that you will hear a nasty scraping noise well before the brake pad wears out?

The current situation is like tires that not only don't have wear bars, they have little flexible plastic shells that hide the tread and make it impossible for you to see whether the tire is wearing out.

Agggh, you know what I meant...

[dpbsmith]

I meant "push this visibly up TO the user..." of course...

(Must remember to hit preview... must remember to hit preview...)

Re:my experience: some DVD media dies with no reas

[freakmn]

I have had bad experiences with Maxwell myself. I think the problems with Maxwell are probably due to their bad brand reputation. You should really stick with the real brands. Genuine ones like: Sorny, Magnetbox, or Panaphonics. That might help a bit.

just burn twice

[ClioCJS]

I burn everything to TWO dvd-rs. I also use RAR to add 8 percent parity data..... so in theory even if the disc is only 92% readable, parity will be able to regenerate the missing data. But more likely than that is I would simply mosey on down to my 2nd copy, (and burn a 3rd copy to keep "two working copies").