Slashdot Mirror
Best Backup Server Option For University TV Station?
idk07002 writes 'I have been tasked with building an offsite backup server for my university's television station to back up our Final Cut Pro Server and our in-office file server (a Drobo), in case the studio spontaneously combusts. Total capacity between these two systems is ~12TB. Not at all full yet, but we would like the system to have the same capacity so that we can get maximum life out of it. It looks like it would be possible to get rack space somewhere on campus with Gigabit Ethernet and possibly fiber coming into our office. Would a Linux box with rsync work? What is the sweet spot between value and longevity? What solution would you use?'
Cue usual discussion about defining the problem correctly, choose the right tool for the job, etc.
Specifically:
"Would a Linux box with rsync work?" - It depends on the objective business requirements you've defined or been given. If those requirements include "has to be implemented on Foo operating system", then those requirements are not just for a backup solution.
"What is the sweet spot between value and longevity?" - Simple: Graph accumulated TCO/time based on quotes from internal and external service providers. Throw in some risk/mitigation. Find the plot which best meets your cost/time/business requirements.
"What solution would you use?" - Almost certainly not the solution you would use, because my needs are different. What is your backup strategy? What are your versioning requirements? What are your retention requirements? (How) do you validate data? Who should have access? What is an acceptable speed for access to archived data? What's an acceptable recovery scenario/timeline, etc.
If you do not already know the answers to those questions, or how to find reasonable answers, ask neighboring university TV stations until you find one which has implemented a backup solution with similar business requirements as your's, and copy and paste the appropriate bits. You'll likely get better answers from people who have solved your exact problem before if you search Google for the appropriate group/mailing list for your organization's level of operating complexity, and ask there instead of asking generalists on slashdot, and hoping that someone from your specialist demographic is also here.
There are 1.1... kinds of people.
Actually, I'd suggest using OpenSolaris so that you can take advantage of ZFS. Managing large filesystems and pools of disks is *stupidly* easy with ZFS.
You could also do it with Linux, but that would require you to use FUSE, which has a considerable performance penalty. I'm not sure about the state of ZFS on FreeBSD, although I imagine that the Solaris implementation is going to be the most stable and complete. (For what it's worth, I've been doing backups via ZFS/FUSE on Ubuntu for about a year without any major problems)
-- If you try to fail and succeed, which have you done? - Uli's moose
You might have mentioned the Slashdot article on these from two weeks ago.
Don't use rsync to make backups. Because you don't just want to backup against spontaneous combustion â" inevitably, there will be accidental deletions and the like occurring in your studio. If you use rsync (with --delete, as any sane person would, otherwise your backup server will fill up in days, not years), then when some n00b runs `rm -rf ~/ReallyImportantVideos`, they'll be deleted from the backup too.
Remember that pro photography website that went down, because their "backup" was a mirroring RAID setup? Yep â" they lost all their data on one fell swoop when somebody accidentally deleted the whole lot. Don't make the same mistake.
Use an incremental backup tool. Three that come to mind are rdiff-backup, Dirvish, and BackupPC.
I would think that rdiff-backup would suit your needs best. I currently use BackupPC at home, which is great for home backups, but I think that it's overkill (and possibly a bit limited) for what you want.
Hope this helps!
The parent is absolutely right. We don't have enough details to really make a recommendation, but if the question is 'can rsync replicate 12 TB with an average rate of churn over a 1 Gbps link reliably'? The answer is an emphatic and resounding YES!
I used to maintain an rsync disaster recovery clone that was backing up multiple NetWare, Linux, Unix, and Windows servers to a central repository in excess of 20 TB over primarily 100 Mbps links. We found that our average rate of churn was 1% / day which was easily accomplished. It was all scripted out with Perl and would notify on job status each night or failures. Very easy to slap together and rock solid for the limited scope we defined.
When you get into more specifics on HA, DR recovery turn around times, maintained permissions, databases and in use files, versioning, etc.. things can get significantly more complicated.