Slashdot Mirror
Advice on Remote Backup Services?
a-freeman asks: "Faced with the prospect of doing automated weekly backups for several servers with some 200 GB of files each, I have been looking for a remote backup solution. A couple of recent articles consider backup to hard drives, although I feel this still fails the 'separate snapshot in time' aspect of good backup policy, since with many of the solutions that I have seen, you will likely lose all your backups if your array gets corrupted. However, CD-Rs and DVDs are just too damn small. Can anyone recommend a remote backup service or interesting combination of hosting service + FTP/RSync/etc., or am I stuck buying a tape drive?"
"...or am I stuck buying a tape drive?"
Whats wrong with a tape drive? It is a medium that was designed for backups. If you are going to be backing up large amounts of data you need a tape-library and remote backup software. If you want the convienence of harddrives then attach the tape-library to a machine with a whole lot of disk. You can backup to the disks first and then archive whats on the backup-server to tape. Most backup software programs allow you to do this.
-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GIT d? s: a-- C++++ UL++++ P++ L+++ E- W++ N o-- K- w--- O- M+ V PS+ P
I'm dealing with pretty much the same issue right now as you. I've set aside a dedicated backup server (cheap K6/2 400) with a lot of disk space which uses rsync to backup the other servers in the office. Then I'm using rsync on an offsite server to keep a backup of the backup. Seems to work well. Having some sort of a raid setup on this box would be even more insurance I suppose.
I'm not using tape because the office I'm doing this for doesn't have a dedicated IT staff, and I'm not going there nightly/weekly to rotate tapes.
I have an Rsync backup walk thru. Once everything is on a large array, you can run backups to a localy mounted tape drive.
Rsyncing keeps the network traffice to a minimum, and the local tape speeds up the backup to tape.
Rdiff backup, does incremental snapshot.
http://rdiff-backup.stanford.edu/
I've had enough abrasive sigs. Kittens are cute and fuzzy.
rdiff-backup is based on rsync but allows you to keep incrementals as well as full backups. great for disk based backups while maintaining lots of history.
for redundancy and recoverability just use it to multiple backup disks at whatever level of redundancy you need. each one will have its own full set of incrementals so if you lose one, no big deal.
"It's hard to beat the bandwidth of a stationwagon filled with DAT tapes"
If you're talking: 1) several servers and 2) 200ish gigs per, welcome to needing a real backup solution.
One thing to keep in mind is the three 'kinds' of backups. You will need to cover (or choose not to) all three.
1) DR. Disaster recovery. A full image of ALL data, usually duplicated so you have a in house copy and a remote copy. Full system images, and a software package that can blast a full system image to a box, or full data and config backups that require an OS install before your restore. Usually this is somewhat light on tapes, since you'll only keep 2-3 weeks of them.
2) File Recovery. Someone deleted something that they shouldn't have and need it restored. Or the Database equivelent: "We dropped this table 5 weeks ago, and discovered just now this random important process that hits it every 2 months. Can we restore the DBF file so we can get that table, data and schema back?" Sometimes DR feeds into File Recovery. You just keep the tapes longer. More expensive in tapes though, and you have data you'll not use (like OS images) wasting tape space. It's easier though.
3) Archival. EG: The IRS mandates that we keep this data for N years (where N is usually greater than 7). Thankfully, this is a thin ammount of data, but it's important none the less. CD/DVD rock for this, but tapes are good too (so long as you're under 10 years. Media and reader issues will kill you after that).
Good luck. Backups are a huge pain. Be sure to test the DR portion of it at least once a year. You'll be thankful you do.
Zapman
Arkeia has a nice backup suite that while not open source does have a free (as in beer) edition. When I was evaluating it, it seemed to work great with my ADIC library.
I would be using that now except for that our company already had a license for the Veritas backupexec software for windows so I was able to just download the linux client software for free.
-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GIT d? s: a-- C++++ UL++++ P++ L+++ E- W++ N o-- K- w--- O- M+ V PS+ P
No it doesn't. It even says it doesn't in the online FAQ.
"Talent does what it can; genius does what it must."
There is an old joke. "How do I move 3TB's from NYC to LA?" Answer, FedEx or UPS.
If it's multiple remote sites as in WAN sites all over the country and you have 200GB's approx. per site to backup. Then you need to have something like a Compaq DLT Tape Library on each server and someone to rotate the tapes for offsite storage.
We have many field locations so we backup to these DLT Tape libraries and either have an outsourced company like UNISYS or Siemans go to these offices and rotate the backup tapes. (A DLT hold many tapes so it's about once a month or twice a month). The tapes are shipped to an off-site archival storage company. Reports are sent to the Lan Center to track the tapes locations. If we need one and it's off-site then it's couried to the LAN Center for restoration.
Yeah, it's expensive, yeah we are a big company. But to do it for the amount of data we need backed over up over the WAN would cost more in bandwidth then it does to have someone pickup the tapes and store them for us.
Alternatively, you could train a primary and secondary onsite person to rotate the tapes and ship them to you. But you would have to trust them and that can be tough depending on the data. Also short turn-around time may mean you need to constantly train new people. Attempt to automate it as much as possible. Also ensure the person is not an idiot that will jam a tape and get it stuck in the DLT auto-loader, etc.