Slashdot Mirror
USB Flash Drives for Backup/Long-Term Storage?
cyberdigm is curious about this issue: "I am writing two books and have just recently gone through the trauma of having my hard drive flake out (physical damage to several sectors). Fortunately, while the OS instance was trashed, the file system is still intact, so I have been able to recover my files.Given that, I am now much more aware of the needed to regularly back up my files. I'd be interested in any opinions about the suitability of USB flash drives to help me solve this problem. The idea would be to store copies of all my files on a USB drive and back them up every day. I like that USB drives are generally fairly cheap. My concern is the long-term wisdom of this approach. Are there (practical) rewrite limits for USB flash drives? Is there a chance that the data would degrade on the drive over time? Other alternatives I am considering include external/USB hard drives. Of course, an overarching concern is that I'd rather not spend a lot of money."
Software RAID on a bunch of different hard drives (preferably SCSI, but you can also use IDE/ATAPI/UDMA/USB) and automatic off-site (e.g. a remote ftp or scp) backup cron job should do the job.
Sincerely,
Pan Tarhei Hosé, PhD.
"Homo sum et cogito ergo odi profanum vulgus et libido."
Almost all flash drives are made using cheaper & smaller NAND flash (rather then NOR flash which requires more transistors per bit of storage). This type of flash wears out much more easily then NOR flash.
If you decide to use these flash drives as a backup medium, you should definitely use some sort of encoding that allows for bit-corrections. Possibly some sort of Forward Error Correction. Or use a RAID parity/striping method.
Back up your data off site on GmailFS.
Win32 Version
*nix version
there are two sides to a magneto optical disk: a floppy-like side and a cdrom-like side. a laser heats the cdrom-like side until it hits a temp. where the magnetic portion directly below that hot point can be changed. a magnetic head then changes the polarity of that hot spot on the magnetic side. the disk is read from the optical side, and the dot on the optical side reads differently depending on the polarity of the dot directly oposite it on the magnetic side of the disk. this gives you over 1e6 rewrites, and the disk won't demagnetize under a certain (high) temperature. also, the shelf life is 50-100 years, in part due to the plastic (3.5" floppy like) casing mentioned by the parent. the disadvantage being that these drives are slow and expensive... slow because the drive checks what it just wrote and corrects the write if it's faulty (i think on a per dot basis). the upside is reliability.
magneto optical discs get anywhere from 128mb to 5.2gb that i've seen, and they come in three varieties: minidisc, which is primarily for audio, but a few data ones are being sold, the old version holds ~1/5 of a cd, so ~130mb, the newer version (uses multiple layers) holds 1gb, and i don't know if they have a data version or not. 3.5" mo discs come in 128mb-1.3gb that i've seen. slightly older drives accept 640mb discs while the new ones take 1.3gb discs. this value may have increased since i last looked for a mo drive. 5.25" mo discs come in sizes up to 5.2gb so far as i've seen; this value might be bigger now.
These is my comments based on working in flash storage industry. There is an endurance limit to flash drives. The nand flash media are rated at about 100K erase/write cycles but in reality it can do more. There is additionally ECC correction to extend the life and preemptively recover with a sector goes bad. Once all the spare sectors go bad drive would likely prevent any further changes but still be readable. To give the most even wear and entend life, use a backup strategy where you erase all files and then rewrite or add incremently. Random write are the worse. Lastly flash failure decreases greatly with temperature so store at stable temperature.
When your house burns down, or floods, and your drives are underwater, including the backup drives?
Here is how I would probably do this.
First, check out RCS. You have one file, you want to keep a record of revisions.
Second, depending on how much I'd want to spend, I'd either back up to CDR once a week (keeping the old backup "off-site", say, on in my vehicle parked on the street, at work, etc) or else buy an online storage space for a few dollars a month.
A quick google search shows 50 MB for $3/month, which is a lot of plain text. If you are using some funky word processor format and/or images, half a gig is available for $10/month. It even supports rsync!
The only practical way to maintain integrity, given that data was stored offline (in fireproof safes) on mag tapes was to use a grandfather/father/son system of backups, with special tapes reserved for end-of-month or end-of-year processes.
I know we've come a long way since the days of batch-processing, but having a series of backups on separate media is way safer than relying on a single flash drive, CD-RW or whatever. The latter is bound to fail at the critical moment when it is most needed.
I run weekly backups on my home computers out to DVD-RWs which are overwritten in three-weekly cycles. More interim stuff on a day-to-day basis gets run out to USB flash drive or to CD-RW. Oh, and I set my backup scripts to alert me if there is even a hint of bad I/O an any media. Anything that is at all iffy is replaced immediately and the backup re-run.
I know this is a pedantic way of going about it, but I've rarely had any trouble with data loss as a result, while others seem to take it as a matter of course that they will lose their stuff when a drive fails.