Slashdot Mirror
Might Flash Memory be a Viable Backup Medium?
General Books asks: "Rather than fuss over mechanical failures and damaged media, why not use flash memory for backups? We maintain about 100 servers distributed to customers' sites. Each night we copy a backup of critical data (generally less than 128MB) to removable media in case the hard drive fails. We have experienced high failure rates with CDRWs and so now I am considering some sort of flash memory like a USB key drive. They are solid-state and you can get a 128MB device for $20. They seem ideal to me, but I can't find solid evidence. One question is how would they endure a lightning strike (perhaps not as good as an optical medium)? Admittedly, there is a wide variety of CDRW drives and media but don't they all seem risky compared to a solid-state device? More info about my circumstances: We have no network for backups. A second hard disk is not viable because it could not be rotated offsite. Tape drives are relatively expensive and overkill for our volume of data."
You mentioned that CD-RWs weren't all that good, and I have the same experience, but why not try normal CD-R discs? They're cheap and pretty reliable. You can even make multiple backup copies if you want, or a multi-session disc to store several backups on one CD to cut costs.
I suffer from attention surplus disorder.
A USB or Firewire external drive would store far more data and could be rotated offsite.
When that place burns I want to be sure all the evidence goes up in smoke.
134340: I am not a number. I am a free planet!
So let me think this through... you are trying to store small amount of data on individual items. Why? Once in storage, are they kept apart fro one another?
I think a RAID will suffice. Locally. If you need to keep it on the cheap, pump your small amount of data to 3 other simple boxen offsite. I mean, for small amounts of data, there's no reason to muck around juggling the physical medium. One can duplicate that data faster and more reliably than boxes of little memory cards with scribble on them.
If you need to go cheaper, try floppies! W00T!
mug
If you carry it with you when you aren't actively backing it up, you don't have to worry about lightning, fire or whatever. If you are struck by lightning, burnt in a fire or in some other way destroyed, you won't really miss the data.
Or why not get yourself a few of those neato caddies that hold a HDD, and allow you to swap them out (internally; I am not talking about external enclosures) they are available for IDE, and the more expensive ones (claim to) allow hot swapping, even (I cannot personally verify how well the hot swap feature works or doesn't ...). I have seen them that even allow you to lock them in place with a key; how cool is that?
... i.e. just pull a drive, and put a fresh one in, let the card rebuild it for you; backup your whole system if you like. Restoring doesn't get much easier than that, either.
Much cheaper in the long run, in terms of media costs, at least for large quantities of data. Especially if you score some inexpensive smaller drives (like a surplus batch of 10 GB or so)
Hell, if you went all the way and just put an inexpensive RAID controller in there, it might pay off in the simplification of your backup procedures
Here is one made by 3ware
Here is one made by Promise
There are plenty of other, cheaper ones out there, too
I cross my fingers (no, not literally) every time I've inserted a piece of Flash Media into my camera, PDA, or USB drive. I find that about one in every 20 times the disk comes up empty. I take good care of them (three different media types), but I don't find them reliable at all. I'd sooner use CD-Rs (though now I might start backing those up every 2 years on new CD-Rs.
Alex.
I work for a university as an educational content developer. We are in the same situation in that our whole codebase doesn't break the 100 meg mark but needs to be kept extremely safe given that it represents months and months of man hours.
We had a lot of bad luck with CDRW's and ended up dropping that idea and moving to a dual backup system. We do intremental backups to CD-R's and make two copies. Secondly we push the content to an FTP site that is elsewhere on campus. The FTP site is backed up seperately onto tape as added an precaution.
Just my $.02
Before you give up on CD-R/CD-RWs, try this:
I don't see any reason why USB flash media wouldn't work for backups if the OS supports it. The only problem is that USB flash media is more expensive then CD-Rs. $20 will buy you enough CD's for a monthly 'archive' (12 CDs/1 per month), plus a weekly backup/incremental daily backup (4 per month), even if you don't reuse the weekly backup media (personally, with the cost per CD, I wouldn't). However, to implement such a system with USB flash drives (assuming $20/drive), would cost $320 dollars.
I love my USB flash drive, but its not cost effective for backups.
one of these perhaps?
No TiVo and no caffeine make me something something...
Is the "generally less than 128MB" before or after compression? A nice compression package like 7-zip might get the files down to a size that can be emailed off-site each night.
What operating system are you running? How long do you need to keep each days data? Does it need to be stored offsite? Is this data security sensitive?
.bat files on a Windows server would work similarly. Make directories on the backup HD corresponding to day of week/month and automate a job to copy the data to the appropriate backup directory. Automate a job to tar/gzip or zip said backups. Automate a job to FTP those zip files to a remote server. Hard drives are dirt cheap, and with the tiny amount of data you are talking about you could hold a whole years worth of backups on a $70 dollar hard drive. Need redundancy? Do the same process to yet another machine with another $70 hard drive. Never buy a tape or CD-RW media again. Never worry about backups again. Set it and forget it. Have the backup machine(s) email you the results of each night's backup just to be sure they happened. Make a test file on the server and delete it and restore it every so often to ensure you can recover last data. Burn monthly to a CD-R for even better sleep at night.
An extra hard drive and cron works wonders on a Linux or Novell server. I assume the task scheduler and a set of
Rather than fuss over mechanical failures and damaged media, why not use flash memory for backups? We maintain about 100 servers distributed to customers' sites. Each night we copy a backup of critical data (generally less than 128MB) to removable media in case the hard drive fails.
Both of these would be my recommendation. I use flash media to boot firewalls, routers and embedded servers that run from RAM drives (nearing 100 deployed at customer sites and in our network). But I automount a partion on CF modules for logs. Flash memory is very reliable; it's rated at about 100,000 destructive writes. Read that as wiping it out, reformating it, not as I wrote to /var/log/messages for a week and the media toasted because somebody's machine caused the firewall to log crap every 2 seconds for a week. If it wasn't reliable, Cisco wouldn't use it for non-volatile storage (neither would I).
The way we handle server backups is for servers to backup via a script to a tar.gz file over a private T-1 for servers. Granted, this amounts to a lot of GB for us but if you use something like rdiff-backup or a more simple script that backups up your files across the net through an SSH tunnel, you should be in pretty good shape. CDRWs are a poor choice if you can't or won't rotate media routinely. Especially since their lifespan for writes is low. You or your customer will have to rotate if you use CDRWs.
you can always Print out the info. If you're really worried about lightning you can use a laser printer and marble
You can read the Compact Flash FAQ
A quick google search returned these links, that may be interesting to you
IDE to Compact Flash Adapter
Flash Storage Solutions
Read all this thread if you will be storing sensitive information
How Compact Flash can keep your data safe?
This guy has an opinon different from mine. He says that, all of a sudden, he lost hundreds of picture. Well, I've been working with Compact Flash for more than one year, now, and the ONLY time I gost corrupted data was when I took the card off the camera while it was writing. Then the camera could not read any picture. They seemed to be lost. But later I put that CF in my CF reader, and ran a chkdsk. It found lost chains, that I saved as files. And recovered ALL pictures except for the bottom half of the one it was writing at the very moment when I removed the CF. It probably corrupted the FAT (same way as hard disks, when the computer is not properly shut down).
And I do think CF is more reliable than Microdrive.
-
Roses are #FF0000, Violets are #0000FF, find / -name '*base*' |xargs chown -R us && mv zig greatjustice