Slashdot Mirror
How Do You Backup 20TB of Data?
Sean0michael writes "Recently I had a friend lose their entire electronic collection of music and movies by erasing a RAID array on their home server. He had 20TB of data on his rack at home that had survived a dozen hard drive failures over the years. But he didn't have a good way to backup that much data, so he never took one. Now he wishes he had.
Asking around among our tech-savvy friends though, no one has a good answer to the question, 'how would you backup 20TB of data?'. It's not like you could just plug in an external drive, and using any cloud service would be terribly expensive. Blu-Ray discs can hold a lot of data, but that's a lot of time (and money) spent burning discs that you likely will never need. Tape drives are another possibility, but are they right for this kind of problem? I don' t know. There might be something else out there, but I still have no feasible solution.
So I ask fellow slashdotters: for a home user, how do you backup 20TB of Data?" Even Amazon Glacier is pretty pricey for that much data.
Asking around among our tech-savvy friends though, no one has a good answer to the question, 'how would you backup 20TB of data?'. It's not like you could just plug in an external drive, and using any cloud service would be terribly expensive. Blu-Ray discs can hold a lot of data, but that's a lot of time (and money) spent burning discs that you likely will never need. Tape drives are another possibility, but are they right for this kind of problem? I don' t know. There might be something else out there, but I still have no feasible solution.
So I ask fellow slashdotters: for a home user, how do you backup 20TB of Data?" Even Amazon Glacier is pretty pricey for that much data.
Crashplan has unlimited storage. I use their home plan; it's unlimited for up to 10 machines. I think I am backing up about 6TB there now.
I have a 16 TB media collection at home that I just back up on more hard drives.
External hard drives in USB cases + Robocopy works great for me.
I don't respond to AC's.
I agree, I've been using Crashplan for three years and the unlimited space it's really great BUT... ...I'm not sure about the bandwidth they provide: how long it will take to upload 20 TB?
Anyway, I don't see what's the problem in using external drives for backup. Here in my lab I've realized that the best way to backup X Terabytes is to have another storage with X Terabytes...
These "unlimited" claims always turn out to be lies. When will we learn?
My friend paid for an "unlimited" account from JustCloud for backup. He stored 1.8 TB on it and then they "fair use"'d his ass and canceled his account. They didn't even give him a refund for the rest of the money he prepaid.
At 10 characters per second, the backup would take 63,419 years(*) and require 659 TJ or 0.2 TWh of power to complete. I have a customer that still uses paper tape. It lasts and lasts, and I have only replaced the reader once. The punch needs a new power supply every 20 years or so.
However, 63,419 years is a long time to wait for a backup to complete.
(*) this assumes that 1 TB = 1,000,000,000,000 bytes. It takes almost 70,000 years if you add the extra 10%.
Punch the hole and you can flip them over to double your capacity.
Do not look at laser with remaining good eye.
At the very end of the 5 1/4" floppy era, the "High-Density" floppy used the same data rate, tracking, and recording density as the 8" 1.2M floppies. They were, in fact, 1.2M 5 1/4" floppies. Which is why their formatted capacity was different from 3.5" "high-density" equivalent, 1.44M.
Other than electrical needs (as 8" floppies often had their spindle motors directly powered by 120VAC line current), the high-density 5 1/4"s were used as a drop-in replacement for 8" floppies in the hobbyist retrocomputing community. (Not collectors, though; they'd want to keep the gear as cherry as possible.)
Welcome to the Panopticon. Used to be a prison, now it's your home.
The dataset isn't that huge. Tape can write at speed at least as fast as disk - LTO-5 writes at up to 280MB/sec - far faster than you can read the source at which isn't likely to be fast disk. The seek for a single-file restore will be slower than disk but after the initial seek, the read will be as fast as from a typical archive disk (no, you're not archiving 20TB to SSD, nor are you storing the source data on SSD either)
However, the change rate for this application is likely to be low. That makes it very feasible to do random testing from the new backups where a minute to do the tape mount/seek is not a problem. You won't be writing more than a single tape in any single run (LTO-5 is ~1.5 TB of uncompressed data).
For $2K, you'll have the LTO-5 drive. Add $500 for 20 tapes and you can back up the entire set (once) plus a bunch of incrementals. I haven't done the math with LTO-6 which is faster and holds more data. If you want multiple generations, tape is a lot cheaper per TB than disk. The initial drive cost hurts but after that, the price is good at $15/TB or so.
The problem with RAID-5 is that you are 2 disks away from failure and rebuilds often kill the disks.
No. The problem with RAID-5 is that during a rebuild, there is a reasonably possible chance you could have a UBE, and lose one bit, making perfect recovery of the array impossible. Only a stupid controller would consider a UBE to be a failed drive and trash the entire array. On RAID-6, you still have the same possibility of a UBE, but the chances that two separate drives would experience one on the same exact block during a rebuild are so astronomically slim as to be irrelevant.