Searching For A Reliable Backup System?

InfoSec asks: "We run a great deal (read all) of our current business on Linux, and we have found that tape backup is far too unreliable for our purposes. We have used Quantum DLT 7000s (two of which died), and we have two VXA drives (11 hours to restore three tarballs). Is there a better solution?"

Comments

forget Travan too =)

[compwizrd]

Obviously with DLT and the VXA's, you're quite a bit upscale from what i'm fighting with at the office, but add HP 7/14, and Seagate 10/20gb travans to the Stuff That Sucks(tm) list.

Drives die within about 6-12 months, and tapes die in a few months. I'm praying i can get better life with some 12/24 DDS-3 HP drives. Seems to work pretty good on the main fileserver, but then again, that's a bit better controlled environment than what the Travan's are mis-operating in.

Re:forget Travan too =)

[Anonymous Coward]

I really wouldn't bother with any of the HP 12/24 DAT drives. I'm a support engineer and I've found they're totally f**king usuless. There's nearly 90% failure rate within 18 months. I've got so fed up with replacing these drives in the past 12 months. It's not as though they're in over used systems either. I think they're over suseptable to dust or something since they take more and more cleaning to keep them working. In the end you end up taking it to bits to clean it properly and then you only get another few week from them. The sony DDS-3 drives seem to fair alot better, though I have dealt with less of them. I have a very low opinion of DAT drives in general, they seem to be very expensive for what is a low life span, esp. where HP is concerned! Plus they aren't really a disaster recovery tool as alot of companies use them as, esp. in NT systems. P.S. don't touch ArcServe with a very, very long pole! J Bullet

Re:forget Travan too =)

[compwizrd]

Maybe i'm lucky then, because i've got a 12/24 at home, a 12/24 in operation at one of our plants, and a 12/24 in the main server at the office. They get cleaned every 3 months or so with a cleaning tape(yes, not until the stupid light blinks, i'm that lazy). I'll have to do some research into the Sony's. Our HP's are up around 2 years now, far as long term goes. How does the Sony load the tapes? The Hp's just pull it in a bit, leaving the door open, so if you leave the tape in there all the time, you're FUBAR if you have a dust problem. Does the sony pull the tape all the way in and close the door?

And yes, i've heard ArcServe is evil =)

I'm still using the backup that comes with NT, mainly cause every company wants far too much money for their NT Server backup software.

Backup programs ?

[mbyte]

Speaking of tape backup .... Which software do you use ? We have a quite HUGE amount of data (70-80 gb), that we want to backup on a MLR tape (tandberg MLR1)

We did try various software:
- tar: has problems when tape is full. not easy to restore single files
- arkeia: strange interface, need to buy costly updates regular
- tapeware: has problems with hardware compression
- bru: had some problems I forgot ..

are there any GOOD backup solutions for linux out there ????


Samba Information HQ

Re:Backup programs ?

[larien]

Try Amanda. It's free, runs under most versions of *nix and will even back up SMB shares (if you have samba installed). It is also fairly intelligent and will run different levels of backup depending on how much has changed on disk (eg, if not a lot has changed, it'll do an incremental; if a lot has changed, it might as well just do a full dump etc).

We use it here to backup our Unix stuff and it rarely needs poking; it just chugs away in the background and the only intervention we have to do is change tapes every day (which you'll get on any backup system without a tape array).
--

Re:Backup programs ?

[jbridge21]

Ugh, we used that at the place where I worked and it was kind of ugly, and didn't work all that well. Try it, but I wouldn't expect too much...

-----

Re:Backup programs ?

[njet]

AMANDA
the Advanced Maryland Automatic Network Disk Archiver, is a backup system that allows the administrator of a LAN to set up a single master backup server to back up multiple hosts to a single large capacity tape drive. AMANDA users native dump and/or GNU tar facilities and can back up a large number of workstations running multiple versions of Unix. Recent versions can also use SAMBA to back up Microsoft Windows 95/NT hosts.

Re:Backup programs ?

[bogomipe]

Legato Networker can handle almost any OS known to modern man and has far too many options for your own good.

It is also commercial and expensive, but then, losing your data has a price tag too.

I find it useful (weekly backup amount some 400G).

Re:Backup programs ?

[lushmore]

I second the recommendation of Amanda... I have used it for three years to back up 1TB on 25 machines and 300+ filesystems. I use a a 588-slot library with 4 DLT4k's and can produce any file from any night of the last 90 inside of 15 minutes. I have never been unable to fulfill a restore request.

Redundant tape arrays

[Tet]

tape backup is far too unreliable for our purposes

So use redundancy. RAIT is the tape equivalent of RAID for disks. Basically, your data is written across an array of tapes with varying amounts of redundancy (from simple parity, all the way up to mirrored ECC stripes, depending on how much you want to spend :-). There uses to be a CLARiiON DLT array, but since EMC's buyouy of Data General, that seems to have been discontinued. Still, there are plenty of other suppliers of tape arrays. Here's one from Adic, and here's one from Compaq.

This is Not a Flame (TINAF) Post

[Brazilian Geek]

I use ArcServer on an NT 'cause I need a backup tracking system that's easy to configure (ok, Arcserver's not easy but after hacking at it for a while I managed to understand it) and Arcserver's doing a great job for my needs. It deals out about 11Gbs of backup daily on a DDS-2 Seagate and I haven't had any bad experiences with it yet.

What I want to know is - remember that TINAF (This is not a flame) post - what's so wrong about Arcserver?

--
All browsers' default homepage should read: Don't Panic...

kludge works

[Senjiro]

We have a primarily RedHat environment and aren't using tapes due to cost. What we setup is an x86 box with an Adaptec 2940 SCSI adapter and 3 17 Gig drives. Amanda - http://www.amanda.org backs up the boxes onto these scsi disks (holding disks in amanda terminology). It requires a bit of disk monkey maintenance, but we can keep about 2 weeks of backups this way. If we wanted/needed more than 2 weeks back (we don't) we could just buy more disks. Restores are lighting fast from SCSI. Amanda is pretty darned reliable, and only requires the occasional kick. I have it backing up RH, Solaris, 98, and FreeBSD.

Re:kludge works

[compwizrd]

What happens when a group of angry *insert political party* people gets upset that *insert another political party* won the election and rampages through *insert your town* and sets your building on fire in the fun? Onsite backups work, till you don't have a site.

Re:Strange?

[tiberus]

We are using a using an Overland Library Express with two DLT 7000's and a cartridge that holds 10 tapes (we had some cash and I am basically lazy) with ARCserveIT.

While I really can't stand ARCserveIT, I do like the Overland Unit. The drives and the tapes have proved to be highly reliable and easy to use. ARCserveIT might not be so bad if I was only backing up NT servers.

I would have preferred to stick with AMANDA. While AMANDA can be a bit of a bear to get running, of course this was my first install, it is highly reliable and stable. Support from the developers and users was always timely and helpful. One of the many things to like about AMANDA is that in the event of a catastrophe that native Unix utils (tar, gzip, dump, ufsdump, etc.) can be used to recover the data from the tapes.

AIT-1

[sharkey]

We are running a Sony SDX-300D AIT-1 drive. It has worked fairly well for us, though it needs cleaning before it will tell you. The performance is limited by our software situation (Backup Exec on NT), but is still well above the acceptable line. Backing up Win9x, WinNT/2000, Linux, MSSQL and Exchange live, we get an average of 100+ MB per minute. It's enough for us, anyway. It's SCSI, so there should be no problem hanging it off a *NIX-alike box.

--

ArcServ vs Retrospect

[PhunkySchtuff]

Well, IMHO ArcServ blows chunks.
i've been forced to have it installed on my NT Server by the powers that be, and i don't like it one bit.
It's top heavy, installs all kind of shite on the system and, well, i just do not like it one bit.
If you are looking for good quality, ease of configuration and _most importantly_ ease of restoration, look no further than Retrospect from Dantz, they've even got a free 30 day trial version there.
I've used it in an environment where the DAT backup was long-term offline storage of high res images, and it's simply the easiest software i've used.
for hardcore users, it may not have all the features of something like ArcServ, but for day-to-day use, and sheer ease of administration, you can't beat it

-- kai

duh

[Zurk]

i use a AIT-2 sony drive in an autoloader (around $17,000 for a 30 tape autoloader) and use lonetar to back up from alpha linux boxes. gives me around 40MB/min on a good day.

What are you *doing* to those poor drives?

[Mr. Foogle]

We have used Quantum DLT 7000s (two of which died), and we have two VXA drives (11 hours to restore three tarballs). Is there a better solution?"

Drives die within about 6-12 months, and tapes die in a few months.

whatinthehell are you people *doing*? clean those drives on a weekly basis, retire you tapes after xx hours of use, and you'll be fine. And for gods sake, using another box for backups is *fine* - until your entire site burns to the ground, and your whooptedoo backup system with it. kids these days.

Re:What are you *doing* to those poor drives?

[compwizrd]

good luck getting a user at the site to do so :P
i have enough trouble getting them to send me the monthly tapes. environmental concerns is a major thing to consider

Re:What are you *doing* to those poor drives?

[Mr. Foogle]

good luck getting a user at the site to do so :P i have enough trouble getting them to send me the monthly tapes. environmental concerns is a major thing to consider

i've had that problem. make sure that the remote site guys have written procedures to follow - and a number to call if anything untword happens. have your senior boss person speak to the senior boss person at the remote site. impress upon both of them that if *they* won't follow your written SOP, then when *their* data goes south, it's going to be a long time to recover. write up a FAQ detailing *how* long it takes to recover, and the revenue lost.

Seriously, *they* don't really see why they are doing what you ask, they don't care. But they do speak 'downtime' and revenue lost, and they *must* understand that you can't wave a magic wand and make it better . .

Re:What are you *doing* to those poor drives?

[compwizrd]

Heh. =)

I had one lady at work, refused to let me replace a 1.2 and 1.0 gig drive, even when i told her she was going to lose data on them, within a year or two. She's the plant manager, so it's very hard to override her(short of sneaking in when she's not in the building, and doing it anyways, she wouldn't notice the difference). I'm pretty much forced to VNC my way in at midnight or so from home, for any routine maintenance that has to be done.

Anyways, hard drive died. She loses data, bigtime. Claims I never told her i wanted to replace the drives, and furthermore, claims I tried to replace the drive, but couldn't(???) Then on top of that, tries to send the drive away for data recovery, when I told her it wouldn't do any good(bits were randomly flipped throughout the filesystem)

I just spent 17 hours rebuilding the CEO's machine at the office, bad hard drive. Due to bad backup procedures, and to pre-existing hardware issues, it took longer than it should have.
<RANT>
(THANKS MATROX FOR THE BUG IN THE G400 BIOS THAT PREVENTS THE DAMN MACHINE FROM SHUTTING DOWN WITHOUT A BSOD, I ALWAYS WANTED TO RUN SCANDISK EVERYTIME I REBOOT!)
</RANT>
It's not fun. Takes less time to rebuild from scratch, than to try and salvage data off a dying hard drive. With a proper smbtar, i can have the machine back up and running within 20-30 minutes.

Yet, we're not likely to do anything about it. Frustrating how upper management will refuse to do anything, until something happens, and then it's a half-assed measure, implemented badly.

Spend the money now, get something that works, instead of spending even more money on the overtime for your overworked tech's. It's less stressful on them. <grin> And that's not even counting the value of the data...

Re:What are you *doing* to those poor drives?

[Tet]

THANKS MATROX FOR THE BUG IN THE G400 BIOS THAT PREVENTS THE DAMN MACHINE FROM SHUTTING DOWN WITHOUT A BSOD, I ALWAYS WANTED TO RUN SCANDISK EVERYTIME I REBOOT!

Hmmm. I've had no such problems with my G400, under either Linux or Windows. Strange...

Re:What are you *doing* to those poor drives?

[compwizrd]

It's likely your bios revision. It shipped with version 1.6-something, and my update to 1.9-something fixed it. It's even documented in the latest bios notes.

Re:Backup programs...from a guy who does backups

[TBone]

If you're running a large installation of servers, I'd have to recommend Veritas NetBackup. I'd also have to !recommend Networker, as I've had no end of problems with it in all the places I've worked for anything but the single-server, local drive version.

Pros:

• Distributed architecture scales with absolutely no problems. We run NetBackup on about 800 servers spread throughout the entire eastern part of the United States, from workstations to large NT and UNIX servers.
• Largest supported platform list that I've ever seen. NB 4.0 has been verified on most Linux distributions as a client, as well as just about every UNIX, Windows, and Mac platform, and some platforms I didn't even know existed (WTF is Auspex? People still run UNICOS on CRAY servers? And why would anyone run Data General's DG/UX?).
• Widescale device support, from standalone drives to huge libraries.
• Non-OS client software out the ying-yang. Pick your database, there's software to get it to back up to NB. Have Network Appliance boxes? Get the NDMP client.
• In it's simplest setup, every image on tape is a GNU tar tarball. If you lose your master server, grab the backup tape, forward to the file you want, and tar it off by hand.
• Lots of additional software that works with NB. Want to manage duplication and offsite tape storage? Get the Vault extension to do it automatically. Need spiffy reporting? Get the Advanced Reporter package.
• Pick your interface to the software. Commandline, X, or Java. I believe the Java interface will even run through a web page.

Cons:

• The NetBackup Server software doesn't run on Linux. It does run on just about every other UNIX, and Windows NT and 2000
• Learning curve is steep. I've been one of the primary admins here since we installed the product last year, and I still have to go running to the docs every time there's a problem I don't have memorized yet
• License Keys. They're a pain in the keester. I don't like dealing with keys when I have a site license for the damn software, I just want to install it all over the freaking place.
• $$$$. I believe our site license cost us just over $1M. Contact your local rep to find out how much you'll get reamed :)

I've used Legato Networker, SCH Reels, various dump utilities, and Veritas NBU, and NBU is by far the best product I've seen so far. Any questions, drop me a line