Automated CD/DVD Archival?

An anonymous reader asks: "Our department used to use a Cedar Technologies Desktop CD-R Publisher for fully automated backup of data (~2 CDs per day) controlled by a Linux PC. The publisher just broke and we are looking into a new backup solution to automatically burn and print CDs or DVDs. Solutions for CD/DVD duplication are available for Windows and Mac (for example: Primera and Rimage [which acquired Cedar in 2000]) but not for Linux. While a Mac would be OK, none of the manufacturers seems to offer scriptability or a command line interface which is essential for our task. Tape and HD backup are not an option - the data is already mirrored on RAIDs. Has anyone set up a similar archival system using Linux?"

Comments

Let me tell you about the Rimage systems

[everyplace]

Recently at work I had to recommend an automated cd printing / burning system, and I went with the Rimage 2000i. We're a mac-only design shop in nyc, and needed the machine not for backup, but more for one-offs with automated labels, in a machine that was networkable.

One of the things on the Rimage website that's kind-of misleading (at least it was to me) was that it NEEDS a windows pc in order to share the rimage machine with other machines, like a mac. But once it's setup, the machine works wonders.

What's interesting about this machine though, is that despite the ridiculous setup hurdles, after it all works they provide a fairly decent way of writing your own scripts to control the machine.

The entire device uses xml files in order to handle job requests, and the client they ship it with is actually just a beast of a java app. But the xml files are used for the imaging orders, the production reports, everything. They also have a fairly extensive sdk that allows you to do pretty much anything.

I had an unfortunately difficult time setting this thing up, but the tech support (while their english was a little lacking) were actually incredibly knowledgeable. One of the things they told me was that almost no-one who buys this device uses the provided client. It is designed to be integrated into custom work solutions, so for you this might actually be appropriate.

If you're looking for a solid dvd archival device / printer that has an autoloading function and is fully scriptable, the Rimage 2000i (or any of their devices higher end than that one) could work.

Xor?

[yppasswd]

If he says "Tape and HD backup are not an option", well,
I'd think he has considered all the other solutions.
There are cases where CD archiving is the only solution.
For example:
- Legal requirements of read-only media. My case.
- If archives must be guaranteed readable by common
hardware (and I mean COMMON, try buying a tape reader
in your favorite supermarket...)

Sure, backup on CD is a pain, but this was not
his question.

Re:Xor?

[bluGill]

Several companies make write-once tape drives. We use Sony AIT drives at work, but there is no reason to assume the others won't work. (I know DLT has a write-once mode)

Tape drives are common hardware for enterprize backup, more common that CD/DVDs. CD/DVD is consumer media, I it easy to show in court that they are not up to the other standards that anyone who must save data must maintain.

Needlessly elaborate solution

[AndrossUT]

take an old linux box with a burner. Set up an automated backup burn... and use a LEGO Mindstorm setup to pull out the disc, slap an autmatically printed label on, and put a new, blank disc in. Sure it's not the most cost effective or efficient way, but it would be damn cool looking.

Re:And?

[Kz]

As pointed by another poster, there are situations where CD/DVD are superior to either Tape/HD.

but the existance of RAID mirrors have nothing to do with that.

a mirror protects you from hard disk failure, not against data corruption, for that you need a different thing.

also, the original poster uses 'backup' and 'archive' words. these are _totally_ different things! I think he means: i need archive, i had archive with CD/DVSs, i don't want tape/HD backup. but it seems he hadn't checked up the possibility of an HD archive.

I've recently replaced a wall full of DVDs (around 6000 discs) with a near-line HD archive system, at just 1.56$/GB total cost!

of course, for offsite backup of the archive, it still burns DVDs

Splitting Discs

[Wiwi Jumbo]

I'd like to do (somewhat) automated backup's as well... tho more on a homesize scale...

I'm trying to back up my pictures and my mp3's on a regular basis, but at 15 Gig's for one and 56 Gig's for the other even putting them on DVD can be a pain.

Every app I've used (at least on XP) can automatically "split" the files over multiple discs but they all use their own format for it making recovering the file difficult if not impossible with out the original program.

Is there anything that will split on the file structure so I can just read the files like they were burned normally?

Right now I keep adding and removing files from the "to burn" list and try and get as close to each disc limit and then do the same for the next disc... makes me put off my backups for longer then I should....

Ack Thpt

[binaryspiral]

Burned DVDs and CDrs have piss poor shelf life for archival material. RAID arrays can disappear at the snap of a raid card's whim (yes - it happened to me last week).

Please - re-evaluate your solution before you actually need to recover data or get (gasp) audited and need every single file to work 100% for the auditors.

Recordable optical media sold retail are not up to the standards of archival requirements of most governing bodies (like the SEC). There are optical WORM drives that are used by medical data centers and hold only 30GB on a huge platter. And most of those are getting retired for other methods.

(this is not a plug, it's just what works best for us)
There are many tape solutions like Exabyte's VXA-3 with 160GB native storage space on an $80 tape. Granted $/GB is higher than a DVD-R - the tape will not let you down. The tape is equal to ~35 DVDs and writes at 500MB+/minute.

We have an Exabyte autoloader with 10 tapes on a weekly rotation - and it was as close to heaven when we needed to restore a server. We also backup >400GB of data weekly from a few of our database servers - and need it to be there when something fails.

The entire rig will set you back about $3000 including tapes. This will give you over a TB of backup space. And the tapes are archival ready.

For your use - 1-2CDs/Day each tape will last you about 3-4 months. But it will also allow you to rotate your backups off site and give you much better utilization of space and much higher chances of recovering this data years down the road.

Please reconsider your backup solutions... if it's worth saving at all - it's worth being able to get it back later.

Pioneer DRM 3000

[Ayanami Rei]

Product Brochure

This is probably overkill, but it is a really cool piece of equipment, and it doesn't rely on shitty windows software to do it's job. Unfortunately it costs $10000 fully loaded with 4 DVD-RW drives.

RAID != A Backup Method

[seigniory]

RAID isn't going to help you if your building goes up in flames.

RAID isn't going to help you if a file is deleted accidentally.

RAID isn't going to help you when someone comes in and steals your boxen.

In 15 years I have never, ever, EVER recommended that someone back up to optical media as their only recovery method. DAT drives can be had in the sub-$200 range, and the $/MB cost is cheaper than DVD media - and much more reliable.

I realize that this doesn't really answer your question, but it's an important point that shouldn't be overlooked.