Slashdot Mirror
Backing Up 100 Gigs in an Hour?
cybrthng asks: "I am faced with finding a backup solution capable of archiving to tape about 200 gigs of a financials data in a 2 hour window. I originally looked into DLT8000 Jukeboxes with 2-4 drives but have recently discovered the new LTO drives. I am interested in knowing real world experiences with these drives as there has to be a catch. I mean there is a 3 fold performance increase in data transfers, two fold increase in tape capacity and a minimal price increase overall. With these drastic differences is there something I'm giving up with LTO over DLT or vice versa? Which backup applications are more geared to handling volume and integrate with Oracle RDBMS? Restoring speed is even more critical then backup speed so i'm curious about how these two drives compare and which applications are best geared for this much data on a nightly bases. Mind you there will also be about 500 gigs of data in an end-of-week backup as well."
Does the backup medium have to be tape? Hard drives are in fact more reliable than some tape, and would have a faster data transfer rate. A pair of hard drives hooked into a RAID array could backup over 200 GB of data and then be taken offsite just as easily as tape. Considering the fact that the drives would likely cost $400, tops, and could be reused many more times than tapes, I don't understand why people bother with tape anymore.
Remember, there were no nuclear weapons before women were allowed to vote.
You never mentioned it, so I thought I'd ask. If you don't need something easily removable, you can still have the data backed up to the other side of the data center, or even possibly they other side of the campus. With a storge array on a fibre loop you can back data up hourly, all 100GB in a full backup. Even a Gbit ethernet link could do this in under an hour, provided the link is not shared too much. Then you could run daily or twice daily tape backups off of that archive to send to you offsite safe archive location.
It's by no means an easy feat, but the following should probably get you there.
First, you need to consider how the data is getting to your backup server. This looks like a job for gig-e. (since you don't really want to run you DBs on the same machine as your backup server.) You should use multiple streams. (either break it into multiple smaller jobs or enable the multiple streams option in your backup software if it has one. Many do.) It's hard to flood even a 10base network with a single TCP/IP connection. (your bandwidth utilization decreases in inverse proportion with your latency. I forget the exact formula though.)
Next there's how you're getting it to tape. I recommend running the backups to disk first if you can. This means you won't stall a network connection if you change tapes, or the like. But it does mean you need a lot of storage on the bkup server. Also, if that's a 2h from DB to tape window, this might not be useful. However, barring using a SAN and snapshots (or the like) your only other option is to go straight to tape.
To go straight to tape you'll need at least 6 DLT drives, assuming you can keep the tape streaming and get 6Mbytes/s, and you balance them across a wide enough SCSI and PCI bus(or whatever system bus you choose) This will give you N+1 redundancy and meet your bandwidth requirement of 28Mbytes/s.
As for the LTO/DLT trade off. We're moving to an LTO solution where I work, and it generally seems to be the way to go. It's worth evaluating, but I don't think your choice of tape either way should be your restricting factor. And there is something to be said for the reliability of DLT.
By no means do I know a whole lot about backup technologies or any of that, but I do have a suggestion that kindof takes a different angle on the hard drive idea.
I understand that you would want and need to keep the data off-site on tape (requirement). However, getting that transfer rate is going to be difficult. Perhaps you could do something like this:
Use the hard drive backup (SCSI RAID perhaps?) idea to backup the data quickly and reliably. THEN, you've got it backed up in your time limit. Now, you can back up that back up with a tape, but you don't have the incredible time requirement. Get it?
Concept:
Original Data on Hard Drive
--> Back it up onto a separate Hard Drive within the time limit
--> Now, back up that hard drive that has just backed up the original. You have a backup done already, so you've met the time needed. Now, you can back it up with tape or whatever without having to do it within such a short amount of time. You can use the technology you desire to back up the hard drive copy while the original data drive keeps working.
Then to restore, you can do it from whatever the removable media is.
Again, I don't know a ton about this, but it's just a thought of another way to accomplish this.