Slashdot Mirror
USB Flash Drives for Backup/Long-Term Storage?
cyberdigm is curious about this issue: "I am writing two books and have just recently gone through the trauma of having my hard drive flake out (physical damage to several sectors). Fortunately, while the OS instance was trashed, the file system is still intact, so I have been able to recover my files.Given that, I am now much more aware of the needed to regularly back up my files. I'd be interested in any opinions about the suitability of USB flash drives to help me solve this problem. The idea would be to store copies of all my files on a USB drive and back them up every day. I like that USB drives are generally fairly cheap. My concern is the long-term wisdom of this approach. Are there (practical) rewrite limits for USB flash drives? Is there a chance that the data would degrade on the drive over time? Other alternatives I am considering include external/USB hard drives. Of course, an overarching concern is that I'd rather not spend a lot of money."
Experiences in using flash memory for backups on PocketPC:
#1: Rewrite limits. Currently, flash memory has about a 1 million rewrite limit theoretically. In practice, though, I had a CompactFlash card (no real different technology from the USB drives other than interface to system) fail after about a year of daily backups.
#2: Time-to-destruction- I once left pictures of my honeymoon for nearly 6 months in my digital camera, also using flash memory. After 6 months, the files had a 50% corruption rate. So I wouldn't consider this a very long term storage solution- at least not without refresh.
Asside from those concerns, it's a very cool idea- especially if you kept the backup software on the key and increased your potential by using say, 7 keys (one for each day of the week) and kept the backups off site.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
Software RAID on a bunch of different hard drives (preferably SCSI, but you can also use IDE/ATAPI/UDMA/USB) and automatic off-site (e.g. a remote ftp or scp) backup cron job should do the job.
Sincerely,
Pan Tarhei Hosé, PhD.
"Homo sum et cogito ergo odi profanum vulgus et libido."
which do you prefer?
why do people not realize that costs for the physical item are only part of a price tag?
comment directly in my journal
Almost all flash drives are made using cheaper & smaller NAND flash (rather then NOR flash which requires more transistors per bit of storage). This type of flash wears out much more easily then NOR flash.
If you decide to use these flash drives as a backup medium, you should definitely use some sort of encoding that allows for bit-corrections. Possibly some sort of Forward Error Correction. Or use a RAID parity/striping method.
Second, since we *are* just talking about text, it might be worth your while to use email. I've got a community network email account that I do this with - email them important info, they store up to 15 megs worth of data, where it will pretty much sit forever.
Third, little more expensive: last I checked, a 512MB USB disk drive costs about $70 CAD; you can buy a brand new low GB HD for about that much. Just run two HDs on your system, sync the data every night, and there you go. HDs don't tend to flake out as often as you think, and this way if one goes, you've still got an onsite copy. Then just buy another $70 HD and keep going.
Condemnant quod non intellegunt.
Back up your data off site on GmailFS.
Win32 Version
*nix version
The more media types and locations, the better, but concentrate on the small percentage of your data that you couldn't recover by just buying or downloading another copy. Concentrate on your own personal data, and the job will be much easier to manage.
The USB drive will be fine as one approach--use it daily--but don't leave it at that. It's a convenient medium, but you want it to be able to fail without hurting you much. If you combine it with other media, you can enjoy the convenience without exposing yourself to whatever risks there may be.
Then get yourself Web hosting from some reasonably good quality host and FTP your files to your own website. If you're not very technical, the Web host can tell you how. You don't need to learn how to build a website to FTP files. The more important the data is, and the harder to recreate, the more you need to have it on the website. If you can just get it up there, they will do your backup for you. This assumes that you don't have tens of gigabytes of personal data, which could be a mistaken assumption if you are talking about photography, for example, instead of writing.
From time to time, burn a CD-R of your files (or multiple DVDs if you have gigabytes of important personal stuff), make multiple copies, and stash them in different locations: with your parents, in a self-storage locker, or whatever. That's too much bother to do very often (if you include the offsite stash), but it's a good thing to do occasionally. Just to be sure, check and see if the CDs/DVDs you burn can be read in your parent's, friends, or kids' computers.
"Those who have never entered upon scientific pursuits know not a tithe of the poetry by which they are surrounded."
Any kind of backup solution is preferable to a single point of failure. USB Keys may not be the best solution out there, but it is certainly better than having only one copy on your hard drive. I would look at more than just USB keys, there are certainly plenty of options for copying and making backups of your data, from CD-Rs, and iPods, to zip disks, and external USB/Firewire hard drives.
While a USB key is certainly portable and convenient, it may also create another problem - easy theft of your work. How easy is it to lose a USB key, or have it stolen. And what happens if the finder claims your work as their own? If you did use a USB key, I would definately not keep it with you, but store it in a safe and secure location.
In that vein of thought, since you are working on a book, do you keep hard copy backups? I know it would be a pain to OCR all those pages back in if you lost everything, but it would be better than starting from scratch. If kept in a fire retardant safe, they would fare much better than digital media would.
I haven't lost my mind!
It is backed up on disk...somewhere...
A bit pricier, but you get what you pay for:
External harddrives
(sorry if the link is bogus)
USB flash drives are pretty small... less than 512 mb max, generally 128 mb normally. I'm assuming, then, that you need to store about 128 MB of stuff. I'll also assume from your statement about external drives that you've got a laptop with very little room inside.
The best option for you would be network backup. Seriously. Either pay a little for a little space on a remote server somewhere, or squeeze a folder out of your friendly publisher network technician / techy friend / college kid / local bum. Either way A: your data is safely backed up on a system more stable than yours, itself probably backed up elsewhere, and B: your data is offsite, making recovering from a stolen suitcase / fire much easier. Other options are possible too, like using G-mail as your remote file store, or starting a "backup pool" with a colleague, etc.
The other option is a raid of mirrored ATA drives, much cheaper these days than you would think assuming you have a desktop... really no more expensive than getting a USB HDD.
The ______ Agenda
I've used a 256 meg flash drive to back up a website or two and some other files intermittently over the last year, maybe a little more. It's handy because I can carry the backup offsite immediately, so in the instance of absolute mission-critical data even the remote chance of some sort of physical catastrophe (fire, natural disaster) I'm covered. The newest generation of USB flash drives are very fast, handy, and cheap.
:)
My only worry has been actually losing the drive itself. Then again, I'm doing intermittent backups, not daily. From the minor problems I've had over the time I've used the drive, I'd consider replacing your memory every 6 months or so. At current prices, 100 USD/year isn't too bad for superior, fast and easy backup
ACs are modded -6. I don't read you, I don't mod you, I don't see you. Don't like it? Don't be a coward.
Sure Flash drives will work but you really need redundancy and it sounds like Gmail might suit your needs. E-mail me if you need an invite.
Nothing in the world is more dangerous than sincere ignorance and conscientious stupidity.
(or any other webmail with sufficient space)
Email yourself a daily backup.
Reasonably secure, readily accessible storage, plus fodder for a future book: "How I wrote the Great American Novel - Stages and iterations."
My computer has 6 usb ports, get 6 usb flash drives and RAID 5 them. If one dies replace it ;)
1 gig of space, just send yourself an email with the latest version every day when you're done. You can clean up after yourself if you do run out of space.
(email me at ralf at muhlberger.com if you need/want an invite)
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt.
-Bertrand Russel
For almost free (at least a fair amount cheaper than USB keychains) loyou have alot of (probably) better options:
1) Web storage (encrypted to your heart's content)
2) Floppies/cds (yes, they arent great for long term, but if you make semi-frequient backups, that should be no problem.)
3) Hard copies. Just take the files down to kinkos/office depot/whatever and print the entire work off. (re-entry might be hard, but non-recoverable data corruption would be harder, also sometimes editing by hand is a nice alternative, especially if you've been staring at a moniter all day.)
4) If you have more than one computer (office/home/etc.) you could back the files up on each machine. The odds of both HDs failing simultaniously is rediculously low.
And if you've got the cash for a USB keychain, you might as well just buy another hard drive.
If cost is a real deciding factor, then (depending on the size of the files you need to backup) thumbdrives can be expensive. Reliability is a problem too as these little thumbdrives can get damaged/lost or stolen quite easily, however the chance that a thumb drive will fail at the same time your system fails will be fairly low.
The Email option is a good standby and can be relatively cheap (some one on this thread offered you a GMAIL invite, there's 1GB of online storage for nothing), also Yahoo offers a 2GB account including disposable email addresses and 20MB emails for only £12 a year.
You asked about reliability and wanted low cost, I would suggest that if you use a thumb drive it should not be trusted as a sole backup system and you should definitely use the free/cheap email storage option as well.
If you have or can afford a CD/DVD writer, use the backup software provided regularly (once a week for data perhaps) and keep regular off site backups (at the office or at a friend's or relative's house). As with any backup software do a 'confidence' test, backup your data, delete a (created for the test) file and restore it from the backup. If you are setting up a machine from scratch (or are prepared to) then you should do a system restore confidence test too, this way when the inevitable happens you will at least know how the restore works.
Seriously, CD-Rs are probably cheaper, and almost certainly more reliable. If you multi-session they'll be even cheaper.
Under capitalism man exploits man. Under communism it's the other way around.
These is my comments based on working in flash storage industry. There is an endurance limit to flash drives. The nand flash media are rated at about 100K erase/write cycles but in reality it can do more. There is additionally ECC correction to extend the life and preemptively recover with a sector goes bad. Once all the spare sectors go bad drive would likely prevent any further changes but still be readable. To give the most even wear and entend life, use a backup strategy where you erase all files and then rewrite or add incremently. Random write are the worse. Lastly flash failure decreases greatly with temperature so store at stable temperature.
When your house burns down, or floods, and your drives are underwater, including the backup drives?
Here is how I would probably do this.
First, check out RCS. You have one file, you want to keep a record of revisions.
Second, depending on how much I'd want to spend, I'd either back up to CDR once a week (keeping the old backup "off-site", say, on in my vehicle parked on the street, at work, etc) or else buy an online storage space for a few dollars a month.
A quick google search shows 50 MB for $3/month, which is a lot of plain text. If you are using some funky word processor format and/or images, half a gig is available for $10/month. It even supports rsync!
I have no idea how long the data will stay on a USB flash drive, or how many times you can write to one. I put my USB flash drive through the washer and dryer on accident. The drive contained my master's thesis in it. When I connected the drive to a computer, I discovered that not a single bit of data was corrupted. Doesn't that rock!
The only practical way to maintain integrity, given that data was stored offline (in fireproof safes) on mag tapes was to use a grandfather/father/son system of backups, with special tapes reserved for end-of-month or end-of-year processes.
I know we've come a long way since the days of batch-processing, but having a series of backups on separate media is way safer than relying on a single flash drive, CD-RW or whatever. The latter is bound to fail at the critical moment when it is most needed.
I run weekly backups on my home computers out to DVD-RWs which are overwritten in three-weekly cycles. More interim stuff on a day-to-day basis gets run out to USB flash drive or to CD-RW. Oh, and I set my backup scripts to alert me if there is even a hint of bad I/O an any media. Anything that is at all iffy is replaced immediately and the backup re-run.
I know this is a pedantic way of going about it, but I've rarely had any trouble with data loss as a result, while others seem to take it as a matter of course that they will lose their stuff when a drive fails.
email them important info, they store up to 15 megs worth of data, where it will pretty much sit forever.
Have you tested this version of foreverness?
My (expensive) USB flash stick died a few weeks ago, after two years of usage. I could still read it, but it refused to be written. Gladly, my vendor gave me a new USB stick for free, but this is definetly not the media I want to store my important files on.
My backup is an external firewire case with a large ATA harddisk, combined with a script that writes all modifications in the main RAID-1 filesystem to the backup harddisk. Tape drives are too expensive, and tapes wear out rapidly. Burning complete backups to DVD-R(W) or CD-R(W) takes way too long, and you need a huge pile of discs to backup just a few ten Gigabytes. The downside is that there is yet another fragile harddisk that may die some day. My hope is that not all harddisks die at the same moment. That's why the backup harddisk is usually disconnected from both power and firewire.
Tux2000
Denken hilft.
One of the plan9 file servers uses block hashing for it's file store. Thus if you save the same file twice it doesn't use any extra disk space (modulo housekeeping meta data)
thus you could backup 100 windows machines and you would consume the sum of their overlapping data, repeated data not repeatedly stored
in this way plan9's full daily snapshot backup system consumes something approaching the minimum disk space required, one can step through one's file system a day at a time
this would be a useful way to utilize your flash as it can, in certain situtations, reduce the number of writes
good luck
There are places where the networks are not touching,and there are places where they are-Boeing's Lori Gunter
I've had a USB flash drive fail completely after about two months of every-day (not heavy) usage. First the drive wouldn't allow me to overwrite a file, then it stopped mounting, and then it stopped being recognized as a valid device - all in rapid succession over 5 minutes.
This was with one of the Sandisk Mini Cruzer 256MB drives. I replaced it with another Cruzer (newer model), and after another three months it's mostly reliable, however I've had it become "unplugged" on its own a few times recently.
just because you're running some cheapo raid setup with no redundancy doesn't mean that proper, hardware independant raid isn't a good idea. it's there to provide AVAILABILITY, not backup though: delete a file off RAID and it's gone: this is why you should have backup as well - e.g. to tape kept offsite.
Mine failed too after 2 months of not so heavy usage... the first few blocks in the root directory are no longer writeable/deleteable... so that's -128KB from my sandisk minicruzer and -60MB from my sony microvault...
while microsoft bashing is always great here, lets put it in perspective. people who have paid accounts like myself dont have these problems. I also have 2gb of storage and a rather large attachment limit. Now you may say, look google has thes features and they're FREE, why dont you switch? I say this,
1) I've had my hotmail account for many years, and since its my primary accoutn I couldnt switch if I wanted to, I need it - too many people have that address. The only downside I have with hotmail is they dont want you to check it with pop. I solved this with a free copy of entourage for laptop curtesy of my school (and its a dammn fine piece of software actually) and hotway daemon on my desktop.
2) gmail is still beta. thats right folk, who knows, maybe in a year or so when they're done free accounts will be crappy and you'll have to pay for good service.
lets be blunt here people, you get what you pay for. Free may not necessarily be the greatest (free software doesnt count, its pay comes in the form of many people putting in many hours to make it work).
so go ahead and bash M$ by all means. I hate 'em too. But dont needlessly bash. Your free hotmail account does not have that many gaurantees on it. If you pay the $25 a year you'll get an incredibly good email service.
"goodbye and hello, as always" ~Prince Corwin, from Zelazny's Amber series
RAID=REDUNDANCY/AVAILABILITY
this is not the same as backup, you fool. Disks come and go: your array remains. What's stored ON THAT ARRAY is your problem, and you maintain it despite the stupidity of your users via OFFSITE BACKUPS. THE END.
Check again. 128MB was the norm 6 months ago, but now 1GB drives are available for $100. The industry is moving *very quickly* it seems. I bought a 1GB "Intelligent Stick" a month ago for ~$80, and two weeks later observed another brand's 1GB drive going for ~$60 after rebate. You can now get 4GB drives.
sigs are a waste of space