Mid-Size Business Tape Library Suggestions?

MPankau asks: "My current company is quickly outgrowing our current tape library and I'm looking for some advice on where to start looking. We backup approximately 12TB of data per night with about 3TB of that going to a disk backup on an EMC Clarion CX600. We're primarily looking for something that will give us some room for growth and be cost effective. What tape formats and library solutions would Slashdot readers recommend? Also, are there any other data backup solutions out there that may be better than tape?"

Comments

AIT

[Dante]

What ever library you choose. Make it AIT4

Re:AIT

[suso]

What ever library you choose. Make it AIT4

Major funding for this message was brought to you by Sony.

Re:AIT

[fimbulvetr]

Actually, I'd agree with him for tape devices. Of all the mid-sized tape drives/changes I've ever dealt with, the AIT class has always won, no contest. Although data silos and other high end storage put them to shame in big-data environments, they are certainly not to be looked over in small to mid sized areas. I've ran every thing from DLTs to Travans to drives that aren't even around any more. The DLT drives I run, even with regular cleaning, need the drive replaced every 12-18 months and the tapes are only slightly better. I've taken over AIT2 drives that were a year old, and worked for the next 3. I've since left the company but recently visited and they were still using them. That's 5 years. Same drives. The AIT3 we purchased at that company is now about 3 years old. No problems there, either. I can't wait to start using an ait4. Awesome storage capabilites, excellent speed, good compression, amazingly reliable and not too expensive.

P.S. I also usually passionately dislike Sony.

Re:AIT

[Anonymous Coward]

yeah stay aways from a standard like LTO... Go all the way to sony

Re:AIT

[sirwired]

What makes me nervous about AIT is my lack of faith in Sony's commitment to the product. The predecessor to AIT, DTF was supposed to be Sony's long-term format, but they changed their minds, making any investments in the old format obsolete. (DTF used gigantic tapes, and Sony was right to change their minds, but that didn't help those with DTF libraries.)

You are tied to Sony drives, and since the form factor is not even close to LTO, DLT or 3590/3592, your selection of libraries is also limited.

SirWired

Re:AIT

[grub]

No kidding.

We use LTO2 and are looking at LTO3. It's an open standard, unlike the GP's Sony adver^Wsuggestion.

Re:AIT

[Sporkinum]

We use AIT3 to backup medical images at the hospital I work at. We have had tons of problems with them. Our AIT2's and DATs have been rock solid though.

Supposedly AIT5's are due out later this year. I think they are supposed to hold 500GB uncompressed.

Call EMC back, write another check

[Gothmolly]

Seriously. Yes, you could cobble something together with 'tar' (this is afterall Slashdot). You already have EMC gear. Buy more. That's what they DO. Your company does $FOO, EMC does storage. They would buy $FOO from you, since its not what they DO.

Re:Call EMC back, write another check

[Wayne247]

Given that they already do business with EMC, the fact that the author states they want to keep "cost effective" might mean that they already ruled out more-EMC as being too expensive.

Or they're just too cheap to do it, and turn to slashdot for a quick fix that will probably not do it.

Re:Call EMC back, write another check

[mabinogi]

no one said anything about cobbling together tar scripts - any company that's reached > 1TB per night knows that's a bad idea already.

He's asking about the hardware side, not the software.

Re:Call EMC back, write another check

[Monkelectric]

Yea ... this has death march project written all over it :)

You want this

[austad]

I have a Sony DMS-8400 petabyte storage array. It's about 3 years old and cost $1.2 million new. I'm no longer using it and it's not doing me any good. It seriously holds a full Petabyte of storage, 1000 terabytes. Drop me an email at austad( at ) signal15 dot com if you're intersted.

Re:You want this

[St. Arbirix]

Wow.

Where else does someone being helpful offer to sell you an unused something they paid $1.2 million for?

Re:You want this

[Anonymous Coward]

hmm, i could sell you 1717 SCO licenses that I'm not using.

Re:You want this

[MobileTatsu-NJG]

"Where else does someone being helpful offer to sell you an unused something they paid $1.2 million for?"

I'm surprised he even mentioned a device. I was expecting a bunch of "Go to Google you nitwit" replies.

Re:You want this

[T-Ranger]

I have one Ill sell you for $1.1 million, and I'll include brown paper and string, and shipping! Some places charge $200k for that alone!
.

.
.
.
(thats what, a 30 year old MAD reference?)

Re:You want this

[BJH]

You really have something that weighs about 1.5 tons, is three or four full racks in size, and sucks down about 10KW of power just sitting around?

Re:You want this

[this great guy]

Do you have something bigger (for my porn collection) ? Thanks !

Re:You want this

[Profane MuthaFucka]

Watch it buddy, that's my wife you're talking about.

Re:You want this

[cdrudge]

Maybe because it's 3-4 racks in size and weights 1.5 tons, they couldn't move it once they realized it consumed 10kW of juice.

Tape Backups

[ryan1825]

What kind of backups are you doing? Incremental or differential? Monthly full backups plus nightly and/or weekly? Knowing which you do and what kind of schedule you run backups on will determine what is the best way to backup essential data as this can dramatically change the way you make your critical backups...

Depends on what you want to do

[Xross_Ied]

IF you want long term archiving, still need tape.

BUT IF you want weekly, monthly, quarterly, yearly backups then a virtual tape library (VTL) is a better option. For most servers, the change in the dataset is small and gradual so a VTL stores one full compressed back + diffs for incremental/differential/full backups. Also, VTLs look for redundant data across servers; 10 similar linux servers will have the almost identical binaries.

I am currently looking at http://www.datadomain.com/ VTL to replace a 72 slot dual drive LTO 1st gen library.

A VTL costs a bit less than a regular tape library + all the tapes you need but the increased throughput and no more tape handling is what makes it worth it.

0.02cents.

Re:Depends on what you want to do

[ivan256]

VTLs are great, but...

It's hard for a small business to keep offsites with a VTL. For a big business with a dedicated circuit to a remote datacenter, it's just fine, but most companies can't afford that. He'll still need a tape solution. It would certainly be faster and cheaper to use the VTL for nightlies and only produce a single set of tapes per week though. It may even make a cheaper tape solution more tolerable.

If he doesn't care to continue using a legacy backup software package, then a VTL is useless, because there is no need to maintain the tape paradigm and the virtualization layer, and he could start using snapshoting instead.

Re:Depends on what you want to do

[Xross_Ied]

Most VTL have support for remote replication but it is usually extra.

Depending on requirements, broadband link is all that is needed.
More bandwidth can be had with point to point microwave links (300km limit).

Re:Depends on what you want to do

[thedletterman]

I worked as an architect on a project that needed access to 120 tb, and had strict performance requirements. It wasn't a daily backup, and our solution was a Disk/WORM library. However, I did get a chance to play around with FileNet, which created an interesting layer, in that it will intelligently determine which files are more commonly accessed/updated and store them online on a disk cache and schedule the other files to automatically move to a library.

In your scenario, this could eliminate the need to differentiate between incremental and full backups, as it will automatically move to nearline anything that isn't being updated on a file by file basis, and your often modified files will be kept online... meanwhile, the entire filesystem will be available 24x7. This replication and data storage solution proved to be worth millions of dollars in my scenario, and sounds like it could save you a ton of money and headache as well.

Re:Depends on what you want to do

[Jeff DeMaagd]

Depending on requirements, broadband link is all that is needed.

That would not be a very high requirement. A broadband link is quite a reduction from the 12MB/s that I see on tape libraries. With good outbound DSL, you might be lucky to get 512kbps, but less than half that is more likely.

More bandwidth can be had with point to point microwave links (300km limit).

How much does that cost? That doesn't sound cheap.

Re:Depends on what you want to do

[ivan256]

Why bother virtualizing tape then? Why not do snapshoting with replication, or CDP.

VTLs are only interesting for interfacing with legacy software.

Re:Depends on what you want to do

[Nefarious Wheel]

IF you want long term archiving, still need tape

Perhaps. EMC has some low-cost high density stuff used by the finance industry a lot, but I'd still look into MAID disk arrays (massive arrays of inactive disks). They have better response than tapes and don't chew anywhere near the power of a large RAID array; disks with info not being used don't spin except for an algorithm-based spin-up exerciser. Iirc they also use acoustic sensors and other interesting bits to pro-actively determine a failing disk, which they then route around as if it were a bad sector. Copan makes them I think.

I would pick LTO

[Bombcar]

LTO is the way to go, it has a current capacity of 400 GB per tape, and is available in autoloaders of all sizes, is made by four manufacturers, and has a roadmap clear through 3.2 TB per tape.

Re:I would pick LTO

[hey!]

Yes, but what if you have QSD because your VCC is acting like a VCO... Oh, wait. QTA.

The best solution

[elrick_the_brave]

This is pretty much the only real sofware: Symantec (formerly Veritas) NetBackup Enterprise Server 6 Hardware: Recommend a StorageTek L80 with whatever drives you see fit. And of course a server to match.. recommend Sun Solaris. If you want, I'll set it up for ya :)

Re:The best solution

[Anonymous Coward]

This is pretty much the only real sofware: Symantec (formerly Veritas)

Wow, a Vertias product now being sold by Symantec? I'm wondering if there was possibly a worse combination of vendors? Maybe Symantec could sell it to SCO to guarantee it would be the absolute suckiest software in the world.

Re:The best solution

[Anonymous Coward]

Actualy, symmantec in a stroke of (rare) genius, has pretty much left the Entperise Support from veritas together. As usual, make sure you call late and get a hold of the Australians. They rock.

Re:The best solution

[Anonymous Coward]

Wow, a Vertias product now being sold by Symantec? I'm wondering if there was possibly a worse combination of vendors? Maybe Symantec could sell it to SCO to guarantee it would be the absolute suckiest software in the world.

I am personally thankful that IBM/Lotus did not buy Veritas and made NetBackup part of Notes/Domino. Specifically the Notes UI.

Re:The best solution

[tigersha]

"My Killbot has a Machinegun AND Lotus Notes!"

Professor Wernstrom at a Robot Convention, Futurama, 3004, AD

Do you really need TAPES?

[rtobyr]

Companies like http://www.idealstor.com/ use ejectable hard drives instead of tapes. This has several advantages, including the ability to have backups in native format: just browse to the ejectable shared drive and copy your data. I don't have the stats on the newer tape systems, but last time I checked, the shelf life of a tape is not much more than 18 months. The shelf life of a hard drive is much longer. 400GB hard drives don't run any more than LTO2 tapes, so it doesn't cost more money. From the web site

Idealstor is a complete backup system with ejectable inexpensive IDE ATA disks which are used for backup. Idealstor uses disks like tape, giving the best of both the technologies; Idealstor offers the speed and reliability of disk and the portability of tape. Idealstor offers a backup platform that for the first time outperforms tape drives and libraries in performance, reliability, scalability, ease of use and functionality.

The one drawback is that they haven't yet developed a way to allow one backup job to span multiple disks. However, the last time I checked on this was before my Marine Corps reserve unit got activated to come to Iraq, which was June of 2005.

Re:Do you really need TAPES?

[ivan256]

Yes, you really need tapes for archival backup.

The shelf life may be longer on hard drives, but the chance of the tape surviving the move to the offsite storage facility is way higher. The infrastructure for connecting a large number of hard drives (switch ports, backplanes, caddys, etc...) ends up costing more than the drive (BTW, 400GB drives cost between five and ten times more than an LTO2 tape), and the automation just doesn't exist (changer robots, barcoding, etc...).

Virtual tape is great, but it's not archival, and it's not offsite. If you're not already tied to tape and tape software, there is no point in using those solutions when you could do snapshotting or CDP instead.

Re:Do you really need TAPES?

[fimbulvetr]

The 1 Bay comes preinstalled with Windows XP professional and offers excellent performance for data recovery and backup.

Great. As if the workstations weren't enough of a headache, now we need to patch, maintain, reboot and license the "tape drive". No thanks. Get a linux box with hotswappable sata. Better yet, it'd be less of a headache to reboot the linux box and swap the drive than go through the hassle of running that crapfest. Not to mention Linux could easily support AES encryption of the drives without more crappy software.

Re:Do you really need TAPES?

[sirwired]

but last time I checked, the shelf life of a tape is not much more than 18 months.

You must not have checked very recently. If stored under proper conditions, LTO has a shelf life of 30 years.

I would NEVER trust my archival data to a 400GB drive that cost about as much as a LTO2 tape.

SirWired

Re:Do you really need TAPES?

[eric76]

last time I checked, the shelf life of a tape is not much more than 18 months.

Huh? 18 months? You must be rewriting to the same tape daily. Or more often. The barrier that you are running into is the maximum number of read/write passes rather than any limitation on how long the tape will hold data.

What you need really depends on how long you need to keep the data. If you want to store data on something that you can set aside and reload years later, tape is the way to go.

Ideally, you should use both. Use disk drives for your first level of backup for short-term and short-mid-term backup needs. For mid-term backup to long-term archival, use tape drives.

The problem with disk drives is that you not only have potential media errors, but you have all the electronics included on the drive. That results in far more single points of failure.

With a tape drive, if the unit fails, you replace it and you can still read your backup media.

Of course, you need a good quality tape drive. With cheap tape drives, you just about need to have two so you can use the second to see if the tape written by the first is actually readable.

Re:Do you really need TAPES?

[dbIII]

I have tapes from 1982 sitting next to me which I could have on a disk in a couple of hours after dusting off the right hardware and grabbing a few cables (which will still plug into a new SCSI card). Hard drives sound good - but can they be trusted in the same way?

Re:Do you really need TAPES?

[grub]

I would NEVER trust my archival data to a 400GB drive that cost about as much as a LTO2 tape.

We bought a bunch more LTO2 tapes for ~CA$75. Way less than a 400 GB drive and far less things to worry about ("Will the motor spin up if I need it in 2 years?" etc etc)

Re:Do you really need TAPES?

[jrmcferren]

Why not use a removable hard disk cartridge. The Iomega Rev Drives have autoloading capablity, 35 GB (I know this is a little low) capacity, and the major advantages as tape. The disk and spindle motor are stored in the cartridge. For more info go to http://www.iomega.com/direct/products/family.jsp?F OLDER%3C%3Efolder_id=26891275&ASSORTMENT%3C%3East_ id=63191&bmUID=1143400161901

Re:Do you really need TAPES?

[Sporkinum]

I don't have the stats on the newer tape systems, but last time I checked, the shelf life of a tape is not much more than 18 months.

I don't know where you got that, but we occasionally pull data off of 10 year old dat tapes. They are very reliable.

Re:Do you really need TAPES?

[ayden]

I would never trust my company's data to Iomega. Ever. They simply don't compete in the enterprise market.

And 35 GB is 'a little low'? That's less than 1/10th the capacity of an LTO2 tape's compressed capacity. I can buy prelabeled LTO2 tapes for $38 each.

Re:Do you really need TAPES?

[Anonymous Coward]

Are you _sure_ your tapes are to be trusted like that? Sure the data is there, but have you actually performed a restore? Archiving policies that I am aware of involve reading data in and writing it to new media (and updating indexes and catalogs) at least once every 5 years. The paranoid among us do so every 2 years.

For some reason, people have this unending faith in tape backups. Let me tell you as someone who works with a lot of tapes (over 50,000 in rotation at last count), these things fail. More than you would think. Sure, they last longer if they are infrequently accessed, but they still will corrupt over time. The worst part is that unlike online disk arrays you will not know that you have lost the data until you need it.

My final issue with situations like this is that you are most likely going to have issues with the tape drives themselves. Especially a drive from 1982. Replacement hardware is going to be a challenge to locate, and with older media the alignment of the read heads might need to be readjusted to match the out-of-alignment characteristics of the older drive which wrote the data. Of course, finding a technician able to perform this task on hardware that old is not going to be easy either.

Re:Do you really need TAPES?

[dbIII]

Are you _sure_ your tapes are to be trusted like that?

They are not the only copy (just the ones I pulled out of storge to get to the original paperwork in the same box) and there are copies on newer tape formats. Also, the drive is not from 1982 but the original tapes certainly are. Unfortunately some of the tapes written last week are using drives that are no longer manufactured - IBM and exabyte decided to stop making certain types of drives without making an adequate replacement. Even a truly ancient 8mm reel drive and a variety of things that came after it (3490, 3590) can more reliably stream data than SDLT and LTO and can do tricks like parallel writes to different drives as the data is streaming in. I probably could get data from most of those tapes and would most likely have trouble and expense getting data from a hard drive of the same era, and wouldn't even know where to start.

All that said - is there a reliable tcopy for linux out there? More and more tape drives are getting hooked up to linux boxes and dd just doen't do the job in every case.

Re:Do you really need TAPES?

[willjwill]

I agree. We use the Idealstor 8 Bay and it is far better than the tape system we relied on before. We have a rotation of 500GB disks for our daily,weekly and monthly. We kept our old tape drive around to run backups from the 8 Bay to our tape for our annual archive.

Re:Do you really need TAPES?

[Sylver Dragon]

...last time I checked, the shelf life of a tape is not much more than 18 months.

The 1980's called, they want their tapes back.
Seriously, tapes last way longer than 18 months. Also, they store well. While a harddrive backup is nice for quick recovery, for long term storage it's a non-starter. First off, I can go over to Dell and buy a 20 pack of LTO-2 tapes for $1,000 or about $50/tape (we use a Dell tape library). Compare that to $88 for a Western Digital WD2000 (200GB ATA100) from Newegg (400GB drives are $180). Considering that I use 13 tapes for my weekly full backup we're already $430 more expensive for 1 week. Figure, I keep a rotating set of 4 weeks we're at $1720 in extra capital costs. Plus, we have the offsite sets (2) and quarterly archives. This is starting to get expensive in media fast. Then we start having to look at the reliability of the media. I can drop an LTO-2 tape and be reasonably certain that I'll still get my data back. If I drop a hard-dive, it's not quite as certain. Granted, you should avoid dropping either, but accidents happen.
This solution of yours sounds like a load that one of our vendors is trying to shanghai us into. It's a newtwork attached disk backup solution that runs around $100k. Considering that we recently paid about $140k for our SAN (including fiber switch and head-end server and CX500), I'm wondering why we don't just kick in a little bit extra and have a RAID 5+1 on our SAN (we're currently RAID 5). When you consider that we are using one tray of Fiber Chanel drives in our array, which we would only need ATA drives to backup, we might get the cost down to around $100k (We'd probably have to buy a second fiber switch for redundancy, but we could skip the server).
Disk based backup sounds nice, but tapes are still cheaper and highly reliable. And let's not even get started on the question of automation. The library we use (Dell 132T) has 2 drives and shuffles 8 tapes for me. And, with luck we should be upgrading that by the end of the year, so I don't have to swap out tapes at all.

Ultrium

[paugq]

Go for a HP StorageWorks Ultrium 960 Tape Drive. It stores 800 MB compressed (2:1) data.

Re:Ultrium

[emerrill]

Umm, thats GB, 800GB. It would be pretty useless at 800MB :)

Re:Ultrium

[paugq]

Absolutely correct :-)

Re:Ultrium

[fimbulvetr]

It would be pretty useless at 800MB

I only have 478MB of data to back up, you insensitive clod!

Re:Ultrium

[afabbro]

Ultrium, yes (perhaps - this is really a broad question). HP, no. There is no point in buying anything HP in storage these days because it's just rebadged equipment (disk, tape, etc.)...with a healthy premium for the HP logo.

The big three in enterprise-class tape library manufacturers are IBM, StorageTek (now part of Sun), and ADIC. Buy from one of them. Don't waste your time with HP.

My personal favorite are IBM's 3581/2/3/4 line. I've worked with all of them and they have some nice features...partitioning, WWN at the drive slot level rather than the drive, virtual I/O ejects, expandability by stacking on frames, highly-available pickers, multiple pickers for high-use environments, etc. Some of the other vendors are catching up, but that's the key...these are all features IBM had in the 3584 five years ago.

Re:Ultrium

[bernywork]

Depends on what your discount schedule is I guess..

I know for a fact that it costs me 20% less to buy HP than IBM because of my discount schedule with my reseller. On top of which I also get a great deal on support, 3 year 24x7x6 warranties cost me next to nothing with HP compared with IBM.

Re:Ultrium

[afabbro]

Yes, if you're dealing with middlemen, then what/how much you buy, what they carry, etc. can skew the results.

I work for a Fortune 500 company and we're large enough to buy direct from both HP and IBM. HP is always more expensive. The hilarious thing is that on disk, HP is significantly more expensive than Hitachi, when it's the exact same equipment (HP just rebrands Hitachi, e.g. XP1024 = Hitachi 9980, XP12000 = Hitachi Tagma, etc.). We're talking like 20% more. Likewise, Hitachi tape equipment is more expensive than IBM, STK, or ADIC every time I've priced it.

HP has evolved into a white-box reseller...in enterprise computing, they are just packaging other people's products and marking them up.

Re:Ultrium

[ayden]

There are 3 major manufacturers of LTO2 drives: HP, IBM and Quantum. HP actually makes the LTO2 drives in my StorageTEK SL-500. The company we acquired last September also uses STK libraries (L80, L40 and 3 L20's) all of which have HP drives.

anything that doesn't rhyme with..

[Abstract_Me]

anything that doesn't rhyme with "ibm" and "tivoli"... that about covers it..

Depends on what you need...

[sirwired]

How much of that data is simple backup, and how much goes off-site for long-term archiving? For on-site backup, Virtual Tape Libraries going to dirt-cheap SATA arrays are becoming rather interesting and useful choices.

For off-site archiving, you really need tape. There are any number of expandable libraries available from any number of vendors. Personally, I am most familiar with the IBM 3584. This can be expanded to a rather large number of drives and slots, and the LTO drives it usually is equipped with are pretty darn solid. (And the 3592 drives you can buy if you have a LOT of money even more so.)

What you REALLY need to pay attention to when building a tape backup solution (which most customers ignore), is environmental and storage conditions, for both your data center and your off-site storage (if any). I think this is a far more important thing to focus on than what brand of library or drive you purchase. Pay VERY close attention to the data sheets for the tapes and drives. Tape can be easily fouled by humidity that is too low (static), or too high (sticking). Same goes with temperature. Stacking the tapes improperly can result in edge tracking issues, which in turn causes little bits of tape to fly around your drive when the drives rollers break them off when shoving the badly-tracked tape at high speeds past the heads.

For software, again, you have a lot of choices. On one hand, you have "traditional" backup applications like Veritas and Legato. These perform your traditional full, differential, etc., backups. On the other end, you have full-fledged data management apps like IBM's TSM. TSM can be a pain to configure, but if done properly, it is very tape efficient, and it has great support for live DB backups, staged backups, file versioning, data expiration (as opposed to mere tape expiration), etc.

Good Luck,

SirWired

Re:Depends on what you need...

[The Blue Meanie]

Give the man a cigar (or a mod up if you happen to have the points).

I've been responsible for tape backups in most of the positions I've held over the last 12 years or so. I've worked with most of the major tape formats including QIC, 4mm DDS, 8mm, AIT, DLT, and LTO.

I'm currently using an IBM 3584 with 3xLTO2 drives. It's almost a pleasure to work with. It doesn't jam. It doesn't lose track of what tapes it has loaded. It's fairly fast. Every other autoloader I've ever worked with has been a pain. ESPECIALLY the DLT loaders. I don't think I've EVER seen a DLT drive last longer than a year before simply crapping out and needing to be replaced. I can count almost a dozen DLT drive failures I've had to cope with. I have yet - in 3 years of continuous use - to physically lose an LTO drive (although I admit all three of mine did lock up at one point due to a firmware bug).

I've also suffered with all of the major backup packages including ArcServe, BackupExec, NetBackup, Legato, and TSM. You know what I've discovered about choosing backup software? It's like picking who to vote for in an election. It's impossible to pick ANY of them based on any sort of positive criteria. You simply have to settle for the one that SUCKS the LEAST. And after being forced to use all of these packages, I can say without a doubt that TSM far and away sucks the least of all of them. You could not pay me enough to run a backup system based on NetBackup EVER again. I wouldn't trust it (or most of the other alleged "backup" systems) with data that had ANY value to me or my employer, whatsoever. I've seen more than one NetBackup installation simply implode, taking the entire catalog with it and needing to basically be rebuilt from scratch, having each and every tape in the inventory re-cataloged from beginning to end. And even when the catalog was still intact, I've had less than a 70% success rate in getting NetBackup to actually RESTORE something I needed restored. Almost a third of my attempts to get data back out of a NetBackup backup system resulted in random, unexplainable failures with cryptic numeric result codes that basically translated to "unknown internal error" according to the docs. On the other end of the spectrum, using TSM, I've successfully restored whole directory trees that were accidentally deleted in just a few minutes, whole Oracle databases that were damaged beyond recovery in a few hours, and I've done a bare-metal restore of both a complete Solaris server and a complete Novell server to a fully functional state in less than 4 hours each. Those last two were scheduled recovery exercises - I don't have ACTUAL failures that need restores very often. We have a bare-metal restore DR exercise for a Windows 2000 system scheduled for the early part of next month, and I expect it will work almost as easily as the other two.

Plus with TSM's Disaster Recovery Manager feature, offsite tape management is brain-dead simple. The system automatically keeps one copy of your data hot and ready in the tape changer on-site, so restores of accidentally deleted or corrupted files/databases can happen immediately, and another copy is fully maintained and rotated to offsite storage by the DRM for a disaster scenario in which the on-site equipment is destroyed. The daily outbound and call-back reports are generated automatically, and plugging them into the offsite storage company's infrastructure is pretty easy. All I usually have to do is take the tapes out of the changer, and put the call-backs in the changer when they're dropped off.

With my current 3584(LTO)/TSM setup, I can safely say - for the first time in over a *decade* of working as a system admin - that I am TOTALLY confident in my ability to restore our data center to 100% functionality in a total-loss scenario. I'd love to find out how many SysAdmins working with any other backup technology have that same level of confidence. I know I personally never had this level of confidence in my backups with any other backup software, and I was always at least a little concerned when using the other tape formats.

Re:Depends on what you need...

[Anonymous Coward]

Frankly, if I only had three tape drives, I would mostly agree with you.

But as you scale up to the tens and hundreds of drives, with a couple hundred clients of various age and capability (let alone environments 10x that size), Netbackup starts to come into its own. The little asterick here is that I have not ever worked for an employer that did bare-metal restores.

You are right in the error codes Netbackup generates, but medium to large IT shops will have people who's entire job is devoted to managing the netbackup systems. These technicians understand the quirks and issues that frequently occur.

Basically, managing backups for an enterprise is a full time job. This means people learn the product (netbackup in my case) in and out. When this happens, your success rates rise to 100% and these odd error codes are either memorized, or in documentation readily at hand. Personally, I've never had a problem restoring any data requested of me. If we have a backup of it, that is. There is still a _lot_ of process and procedure outside of the technical aspects that are much more important in the end.

That said, TSM is very nice. Almost all of the enterprise backup products are once you climb deep inside, and truly understand how the beast works. As with many things in this environment, success is mostly limited by your personell and procedures, rather than the tools used to do the job.

It is a bad workman that blames his tools.

Re:Depends on what you need...

[Anonymous Coward]

I've got a 3494 and a 3584 buzzing just a short distance away from here - mid-sized University. All agree that the frame, accessor, what have you is an excellent piece of hardware but honestly I'm a bit iffy on the 3584's drives.

The tapes seem a bit fragile, particularly that little metal pin that the drive grabs to pull and unwind. Just a little bit off and the drive gets completely shredded.

Granted, we could be doing something wrong. In fact, we do a *LOT* of things wrong here, but I won't get into that...

Re:Depends on what you need...

[thempstead]

I would have to agree with this and the other reply to the post. If you have the money and the time to set it up properly then a 3584 teamed with TSM produces a very very good backup environment.

But make sure that you have it configured correctly and that you understand what it is doing. Also do not reply on vendor consultants to set it up for you as ... well ... in my experience we had three over time and we got refunds on two of them as they did not do anything and knew less about the products they were supposedly expert in than we did. Also knowing other people who are currently using TSM where it has been setup by the vendor it is not necessarily set up correctly (i was shocked at the configuration).

So its a great combo, but make sure you know what you are doing with it and are able to cope in a DR when you have lost your TSM server and onsite library and pools.

t

Re:Depends on what you need...

[nbvb]

You get a cigar too.

TSM's a great *GREAT* piece of software. Lots to love, especially the fact that your data magically comes back out of it. :)

Portable Backupsets are awesome. If you haven't tried them yet, what are you waiting for? :)

Here is an idea...

[chris_eineke]

My current company is quickly outgrowing our current tape library and I'm looking for some advice on where to start looking.

Okay, here is an idea you should consider:

Open your address book and look for $sales-contact at $storage-vendor and remember her phone number. You think that sounded easy? Okay, here comes the hard part:

Pick up the phone, dial her number, and talk to your sales representative.

I'm sure if you tell him/her that you are in the ballpark for $very-big-storage-solution she'll give you a dozen free ipods with your order. Seriously man, don't be a prick. Talk to your business affiliates before you ask slashdot.

Re:Here is an idea...

[mOdQuArK!]

Yeah, like you can trust a sales representative to give you an honest broad picture of available technical solutions. "Surprise, _our_ products are better than anyone elses!"

You can ask a saleperson to boost their own product or service, or disparage the competitor's products or service, but if you completely rely on them to pick something for you, then you are being incompetent at doing your "due diligence".

I'm not saying that most advice that you get from Slashdot will be much better, but with a broad cross-section of geekdom willing to rant on about their favorite "solutions", if you're willing to sort through the noise, you can at least get a diverse set of leads on product/services/processes to evaluate.

not that implausible

[Anonymous Coward]

he might work for the government...

What backup solution do you use? NetWorker?

[Fallen Kell]

I am just wondering, as the solution you use to do your backups needs to support your tape archive that you use.

If compatible, I would look at something like:
http://www.storagetek.com/products/product_page238 9.html

Business requirement

[layer3switch]

We backup approximately 12TB of data per night with about 3TB of that going to a disk backup on an EMC Clarion CX600. We're primarily looking for something that will give us some room for growth and be cost effective.

That's a lot to backup and mouthful. This question seems to direct more toward business requirement. If the rate of the backup required excedes and outgrows the rate of revenue gain or loss due to insufficient backup, the risk must come from the management, I think. The backup cost investment variable must be equally or comparably matched against the growth of the business revenue, hence parallelly effective, not negatively affected or wasted against the entity's objectives and investment.

So the $64,000 question, I think, should be, how much cost can your business bare for a backup solution with the rate of your backup requirement in comparison to the rate of your business revenue growth? If $2 million sounds like a drop in the bucket for your mid-size business, the best approach is to find a best solution that fits that amount of investment.

Perhaps the best solution may not be a backup, but more time or less backup or extension of your current backup infrastructure.

LTO and StorageTEK SL-500

[ayden]

I disagree strongly.

LTO3 is the way to go: 400 GB native 800 GB compressed.
http://www.lto-technology.com/

Look at the StorageTEK SL-500. The library is modular and can be expanded (up to 500 slots and 15 drives) as you requirements dictate.
http://www.storagetek.com/products/product_page228 3.html

I run our company HQ on 5 LTO2 drives in 142 slot library. Weekly full backups about 5 TB. Daily incremental backups take another 3-4 TB per week.

Re:LTO and StorageTEK SL-500

[Anonymous Coward]

What software are you using?

Re:LTO and StorageTEK SL-500

[Anonymous Coward]

Not the OP. I use a single fairly cheap 20-tape (well, it's got more than 20 slots, but I have 20 tapes plus cleaners) LTO2 library (Dell PowerVault something-or-other, strangely it's mostly IBM parts inside, don't ask me ask Dell), and AMANDA. Works like a charm. LTO3 should be pretty much as nice.

Re:LTO and StorageTEK SL-500

[subsoniq]

We recently started switching from SDLT320's to LTO3 libraries, and the oldest LTO3 libraries we have are starting to have problems. We've already replaced maybe 5 of the 35 drives we've bought so far, after 8 months. It's still too soon to tell if this was just an abnormality or if we can expect to have more problems with the drives/tapes, but for nervous buyers I'd say go with AIT for now.

Also, avoid quantum libraries like the plague. We've had constant problem with all of our quantum libraries, about 15 around the globe.

Re:LTO and StorageTEK SL-500

[ayden]

We run Veritas NetBackup Enterprise 5.0 Maintenance Pack 5 for UNIX and need to find time to upgrade to v6.0 MP 1. We run a master/media server at HQ on a SunFire 280R with 3 dual channel LVD SCSI cards and 2 GB RAM plus copper gigabit into our network core.

We also have 2 media servers in Atlanta and Bangalore running AIT3 on Windows 2003 Standard also gigabit connected. Backups through these media servers are much, much slower than the LTO2 backups.

Re:LTO and StorageTEK SL-500

[ayden]

We use HP LTO2 drives and have only replaced one over the past year. I recommend buying a support contract with StorageTEK (or whoever your vendor is). STK replaced the drive for free the same day it failed.

Does anybody have information on the performance of IBM drives?

VTL might be the way to go

[jbourne]

Take a look at http://www.falconstor.com/vtl.asp

They produce a VTL appliance that emulates many different libraries and uses (IIRC) your own hardware. You can then attache a library behind it and move tapes off to that as you need (for off site storage).

Regards
James

Dell ML6000

[zerocool^]

Ok, Ok, I know dell buys it from someone else, like they buy their fibrechannel SANs from EMC... but, we just bought one of these for the Computer Science department at Virginia Tech. Without the additional library component, it's i think 5U's, and holds 20 or 24 tapes. We are using 400GB/800GB LTO-type tapes. We back up to a gateway SAS (Scsi Attached Storage) array (it's slick - 2U, 12x500GB SATA drives, SCSI320 interfaces in back, manages all raid onboard), and flush from there to tape. We bought 75 tapes, so in theory that's 60TB of storage.

The machine its self keeps track of the tapes - it comes with a set of stick on / color coded bar codes, and it scans the tapes you pop in with a barcode reader.

It's cool. It may not be enough for your setup, but it was only like $50k for the autoloader and tapes.

~W

UDO Archive Appliance

[Anonymous Coward]

How about a UDO Archive Appliance?

It's "tiered" storage combining RAID and Ultra Density Optical 30GB disks (soon to be upgraded to 60BG). There is a range of Archive Appliances going from a few slots to over 600, from one internal UDO drive to about 8 or 10 IIRC.

The idea is, the Appliance appears on your network as conventional Network Attached Storagae using FTP, NFS or SAMBA (the RAID part). You put your files on the RAID and the files are migrated to the UDO disks (two copies) which can then be taken away and archived.

UDO disks are guaranteed to last at least 50 years. UDO2 will double the capacity of the disks to 60BG. They're in a 5.25" cartridge and data is written to both sides.

It has a nice web gui for administration and recovering archived data.

And best of all, it runs Linux :-)

How about UDO?

[turgid]

The shelf life may be longer on hard drives, but the chance of the tape surviving the move to the offsite storage facility is way higher.

You might find UDO discs better than both.

12tb to Tape

[chivo243]

YIKES, your company likes to live dangerously.... put the tape idea on the floor, back away slowly, close the door, and lock it, lose the key and never think about it again... that much data to back up and it will grow.... seek the advice of a professional back up expert to come up with these solutions, I can't believe anyone in this day and age will still recomment to use tape.

Re:12tb to Tape

[Anonymous Coward]

I can't believe anyone in this day and age will still recomment to use tape.

That's only because you are a clueless twat. If you had a clue, you would know that sooner or later all large scale operations dump to tape. All!

Re:12tb to Tape

[grub]

In the Real World, tape is widely used for large scale backups. It stores well for years and has a high GB per dollar.

Tell me, what do they use where you work?

Re:12tb to Tape

[chivo243]

We used tape... it failed too many times, media errors/device write errors. We don't want to look at tape again, we use disk to disk, for fast restore, and to portable media for offsite storage. We don't reach 12tb. Btw, I used to work in VIDEO, you want to talk about tape.... and headaches. I would think that already existing technology would have left this in the dust.... We back up with out tape, but maybe 1tb/month. Also, I am not "the" back up guy, and he wants site to site replication next year, with 2 Xserves+Xraids. That will be interesting...

Re:12tb to Tape

[grub]

We used to use DLT but there were problems with it. LTO and LTO2 has been fantastic. And moving tapes offsite in case of a disaster can help you sleep better. I do like the xraid, though, very nice stuff.

Re:12tb to Tape

[chivo243]

We have on the shelf now a Adic Fastor with an IBM? mechanism, this is the 3rd one we have had in 3 years, all broke before the warranty was up, the only good luck we had with them. We have plenty of offsite plans in place, I had all the system back ups at my place, I live only a 10 min bike ride from work... very handy. My boss has all the old tapes... Just in case.

Hey Grub, thanks for being civil, unlike all the other jack-offs that could only spout their superiour supposed knowledge

~ stuff like "you twat" etc how enlightening and informative, thanks... and fuck off! I'm glaaaad I don't work with sorry asses like you lot...

Re:12tb to Tape

[grub]

I do have superior knowledge. I'm just wasting my time in talking to you.. Oh wait.. no sorry... ;)

Avamar Axion

[foniksonik]

This company Avamar is an archive to disk specialist.

Disclosure: I am a shareholder and former employee. I haven't worked there for 2 years so this info is a little out of date... but they've been improving it susbstantially in that time frame so use this as a baseline.

They are not a disk to disk staging server company like EMC. They focus on data archival on disk... have a system called RAIN (Redundant Array of Inexpensive Nodes.. or Independent Nodes, both are correct) which uses N-Level parity like a RAID array but across server nodes, 4 disk 1 or 2U servers in a rack... which allows for swapping out new nodes on the fly w/o data loss.

Additionally they use an advanced Content Addressed Storage algorithm which does CAS at the byte level instead of the file level tagging your data for date/time/location/backup etc. as well as providing a base for a technique called Commonality Factoring which factors out common data across files before it's ever transferred (happens in the background on the client during scheduled analysis windows) and for many applications, reduces storage and transfer amounts by up to 90%.... meaning you can backup a lot more data to a 1TB system than with standard compression. 3-5 TB of real world data is typical for a 1TB system, which includes all the incrementals you can stand... hourly, daily, weekly, monthly, with checkpoints at any point in time you flag or schedule and a policy management system for deleting incrementals and saving checkpoints permanently.

Their client server architecture allows for remote backups easily and for doing remote mirroring for disaster recovery scenarios. Remote mirroring costs extra, a software plugin called Replicator.

Intel uses Axion (their software) for archiving chipset blueprints. Goldman Sachs uses Axion for storing archived financial and legal data. Visa is using Axion for storing digital versions of checks. Nasa uses Axion for storing signal data from satellites and radio antennaes.

Base system comes in around 40k but call for a demo and sales pitch... they'll bring out a CAT (Commonality Assessment Test) to show you how much of your own data they can archive. The first run will take a day to get a level 0 snapshot, then subsequent backups will take between an hour and 3 hours to do what a tape system will take 12 hours or more to do every single time, even with software like Veritas or EMCs Legato acquired stuff....

They support all standard OS's used in servers and their clients support Windows, Linux, Oracle, MSSQL, Exchange and Solaris. Also they support VTL for use as a backup to your CLARION system which means the server will look like a tape system and can be managed as such....

Finally they do encryption on server, before transmission if required for remote backup. Allow for online access to data 24/7 with full search capabilities and network mapping for mounting a backup directly on your administrative client that looks like a normal filesytem... oh yeah and a web based filebrowser for doing individual or directory sized restores if you just want to grab an archived copy of something small... rather than having to do a full restore w/ all the incremental restores, etc. just to get at one file

Seek Professional Help!

[Anonymous Coward]

With the amount of money you're about to spend (and the amount you've already dumped into the CX600! :-/) slashdot isn't the right place to be getting answers. All you're going to get here is random opinions (i.e "AIT4 is the best!", "No, LTO3 is best", "Veritas will corrupt your catalog", "no, just blindly write EMC another check").

There's so much more to know in order to architect a proper solution. For example:
- What sort of clients are you backing up? Is all 12TB on 1 big-ass Sun E25K? Or is it scattered across 5,000 Windows desktops?
- Do you need/want to send it all offsite?
- What are your backup windows like?
- Do you currently have some decent backup software? Are you happy with it?

Do you know to set up all the fiber channel mess you're about to get into? Do you think you can figure out how many media servers you need and where they should be placed in your network?

A couple months ago, I ran across a guy (from the government, of course ... your tax dollars at work!) who had 20 LTO-2 drives in a library attached to a single media server with a 100bT connection. He was wondering why backups were taking so long.

My point here is it's very easy to blow your entire budget for the year on this project and not end up with a decent solution. Unless you're a storage expert (which I'm assuming you're not since you're asking for opinions from /.), you need to find a VAR that can come in and make some recommendations based on your environment. When you buy a new, big library, you're most likely going to need bigger and/or more media servers. That means more backup software licenses. The VAR can help you sort all that out. You'll most likely be needing some professional services from them to help build it, too.

Re:Seek Professional Help!

[ayden]

Uh, the original title of the post was "Mid-Size Business Tape Library Suggestions?" That's what MPankau asked for, and that's what we gave him. All the questions asked are good, but they're only meta to the original question.

Here comes the newbie opinion

[billcopc]

Having only one measly tape drive that I used for about 2 days before giving up, can I ask the general nerd population why we're still using tapes ? I'd be fine if there were some sort of cost advantage but cheap hard drives have become cheaper than tapes now, and definitely more reliable.

So here's my idea: hard drive libraries! Since SATA drives have a standardized connector layout, you could simply have a backplane and swap drives in and out as if they were tapes. As a bonus you'd get much faster read/write speeds and random access (if desired). Maybe design a cheap plastic enclosure to protect the drive and simplify robotic handling.. hell, build a cartridge that fits a dozen 500gb drives and you have a 6tb slab of cheap fast storage (that's 12tb if you still believe in 2:1 compression).

All we need now is a robotic library that's tailored to hard drives. Should be easy, no ?

Re:Here comes the newbie opinion

[OnceWas]

A hard drive library is not useful for long-term and/or offsite storage. You would need multiple libraries and cart them around to even keep a minimal backup set that goes back a few weeks or months.

Also, HDs may or may not work after coming out of storage, tapes usually do, since there's less to break down in a tape.

Re:Here comes the newbie opinion

[billcopc]

Quite the opposite actually, tapes are brittle by nature. Have you ever tried to restore something older than a year or so ? Backups work great, it's the restoration that is prone to failures. Tapes dry out, rot, stretch/deform and all sorts of nasty things. Hard drives are sealed and will work great as long as you treat them with respect (i.e. don't drop them, don't microwave them, don't ship them by UPS).

Hell, most DAT tapes crap out on me within 2 years. Crap, if I could revise history I'd spend my money on hard drives instead of giving money to some whoring tape drive manufacturer. Those freaking drives cost thousands and really don't do all that much.

Re:Here comes the newbie opinion

[tdeg]

DAT tapes are the cheapest around. In my experience they aren't made for enterprise use. For enterprise use go with LTO or maybe DLT. And buy good tapes in either one. You get what you pay for.

NetApp VTL

[Anonymous Coward]

Have you considered it? It sounds like it would fit into your setup easily. It's far superior to tape backup.

Backup tends to be a black hole....

[bolix]

Backup falls across many platforms:

    * Desktop : workstations, laptops, pda's
    * Servers : per server - NO centralised management
    * DataCenter : dedicated console and bells/whistles

To support the above, there are many types of Recovery scenarios:

    * Hot Restore : Replicate to a redundant/failover box/server/Cluster/Datacenter
    * Warm Restore : Restore capability to same or supported (redundant) hardware
    * Cold Restore : lights-out/bare-metal from scratch including alien hardware

And again to tie the two together there are many solutions:

    - SAN/Storage replication/sync (typical dump data to tape)
    - ILM or tiered storage with tiered retention (archive when necessary)
    - Synthetic backups

The problem, IMO, is that the bulk of existing "dump it on tape" apps facilitate "operational" failures NOT disasters. Most shops concentrate on ensuring (sometimes regulatory) data integrity but the restoration capacity should be the focus. In an SMB scenario, you cannot always guarantee the same hardware to facilitate a complete bare-metal restore. Locking data in a box/tape is of fuck all use when the keys are 100ft under water/slag/Cheney etc.

Your question prompts further ones from me:

    - Does your backup app support removing the hardware dependance?
    - Is your storage virtualised?
    - Can your storage snapshot/replicate?
    - Are your servers virtual?
    - Do you _really_ generate those deltas?
    - Can you improve the delta or data-sync efficiency?
    - Can you emulate a full?

What i'm getting at here, is that SMBs cannot afford to just backup and be done, SMBs encounter failure rates higher that the FortuneX guys, accommodate that first rather than a straight performance/media benchmark.

LTO3 and STK L700

[dspyder]

We have an STK SL8500 loaded with various drives. For a medium-sized shop (12TB is quite a volume for a medium shop, btw) I would definitely recommend and L700 and a couple LTO3 drives. You can always connect a second library now or later. They have a new product, the L1400 that comes pre-configured with ACSLS built-in... might save you some time and energy if you have to mix backup environments.

    --D

Re:LTO3 and STK L700

[rhaig]

the L1400 would more scalable for you if you see growth in the future. The entry cost isn't that much more than an L700 either.

Stay away from the SL500. Had one of those at my last employer. the SL8500 is nice, but has a high entry cost. Scales forever, but very high entry. (have on of those now)

depends

[rhaig]

what is your backup window?
how long do you want to keep the data around?
how much does the data change?

I'd be happy to give you some suggstions, butthose are the questions that need to be answered.

Backup to disk isn't much good if you'reconsidering LTO3. It's faster tham most disk. it only helps with restore latency.

I manage backups for about 300TB of data (mostly oracle, some file server). I'd be happy to through you some suggestions for free, or do a more formal proposal if your management likes that kind of thing. Shoot me an email.