When Does Data Backup Become a Full Time Job?

nasteric asks: "Myself and 5 co-workers assume a number of responsibilities at my current job, including monitoring our tape backup jobs that run nightly. We do a good job of keeping non work related items (mp3's, wav files, etc.) off our servers, but our users keep eating up disk space very quickly. File storage, along with the fact that we have numerous projects 'going live' that each require one or more servers means more investment in our backup solutions. Fortunately, we have the capital to expand our backup solution (media, drives, autoloaders, software, etc.) but my boss cringes when I suggest hiring a full-time person to handle the backups. Ensuring the integrity of our nightly backups is critical, but my teammates are being spread thin due to the rapid expansion of our company. We really feel the best solution would be dedicated backup person. We currently backup approximately 3.5 terabytes of data and our enterprise expands to over 4 states. Does anybody have any suggestions as to when data backup becomes a full-time position? Are there any resources that specify when a full-time person dedicated to data backup should be hired? It would be nice to have some resources to refer to when proving my point to my boss."

Comments

With all due respect...

[aaarrrgggh]

Is there a business need for the level of backup that is being provided? Specifically, is the backup strategy in-line with the value of the data? No timeframe is given for your 3.5TB, so it is difficult to discern your exact needs.

Maybe the easier battle to fight is to reduce the level of data backup, and show how it can save the company money. Chart the growth in number of tapes in your backup journal, and see if it is getting out of hand.

Of course, if you ever DO have a problem, you will be fired, but... what are the odds? ;)

Re:automated solutions?

[madajb]

Interesting, but it doesn't work like that in The Real World(tm).

In The Real World(tm), users bitch and moan to their boss, who bitches and moans to his boss, who will then bitch and moan to your boss, who will bitch and moan to you.

When this happens, you _will_ drop what you are doing and restore said file.

My solution to this problem in the past has been simple. Start writing a log, simple spreadsheet, whatever, of how many hours per week you spend working on the backups (changing tapes/checking the logs/restoring). Do the same for your coworkers. If you and each of your coworkers spend 6 or so hours a day, that's about 40 hours, or one full time person. If it ends up being only a few hours a day, it might not be worth hiring a full timer.
Of course, this doesn't take into account intangibles, like how productive you are if you keep getting interrupted every hour to restore something. Or how much you know/don't know about optimizing backup solutions.

Another, perhaps more palatable option, is the rotating shift. Just like a rotating on call schedule, one person is responsible for all backup related issues during a given week. Presuming you have an automated system, this should really be limited to changing tapes, the occasional backup, and checking the logs to make sure things didn't break in the night.

===
-ajb

1/2 person

[Dukebytes]

We have about 12 TB and about 45 servers that we maintain in the DC. We have one guy that does data loads and backups.

The backups takes about 1/2 of his time. We have a backup guy for him but hs is only used when the other is away.

We do both tape and "DR" types of backups. We are testing a system right now from Avamar that does snapups, etc... and is a disk to disk system - pretty fast and works good for semi static data. Might want to check them out - www.avamar.com. If you got the cash to build a good backup system - it really should only take about a half a person to maintain it.

Hope that helps.

Duke

Re:1/2 person

[GigsVT]

and is a disk to disk system

Speaking of disk-to-disk, Maxtor's MaxLineII that will be out in a couple months is aimed at the mass archive/backup market.

250 and 320GB ATA hard disks, Rated the same MTTF as SCSI, 3year warantee, $400 MSRP each for the 320GB. 10TB for under $20,000.

For 1 to 10 TB this is a cheap and good solution, combined with rsync/rdiff incremental backup smearshots onto either a Linux NAS with 3ware serial ATA or direct attached storage in the form of something from ACNC or an AXUS ATA-SCSI box.

With the direct attached storage, you could scale it up past 10TB, 4.4TB per 16 disk RAID5 with hot spare, string those together on as many SCSI channels as you need. Each AXUS 16 disk box costs about $6000, 16 of the 320GB disks costs $6400, so 4.4TB will cost about $12,000. Use software RAID0 to tie them into larger volumes if you need to.

Anyway, the potential is there for low maintenence, very cheap, and automated backups using this roll your own solution.

I can't wait until the 320GB disks come out!

Several Problems, Several Solutions

[Crypt0pimP]

If you have 3.5 TB AND the capital to get an autoloader, then by all means!!
Depending on your environment (you made no mention of what OS's, where the capacity is concentrated, etc.) you should be able to call one of the b/u software vendors directly (CA, Veritas, Legato, Bakbone, CommVault, etc) or call one of their channel partners and start sniffing for a deal. I work in this sector and man, everyone is slugging it out and dropping their prices to bottom just to win business.

Also consider some storage management software like SRM- It will let you know EXACTLY what you are backing up, who's hogging space (by user and group) and where your capacity is allocated (an mis-allocated)

The short version is this: With some SRM tools and a good automated backup system, you WILL reduce your workload. I know, I set this stuff up for a living.

Disclaimer: I am a tech for a Computer Associates reseller. I am biased towards their products, but will be the first to admit that they sometimes do suck. Hard. Like turbine of an F-16 hard.

If you'd like to, I will give you a format in which to present your Request For Quote (RFQ) to the various vendors.. It will give you an unbiased measuring tool for all that FUD and MarketingSpeak.

slineyp (at) hotmail (dot) com