Slashdot Mirror
Online Backup vs. Tape Backup?
hashbox asks: "I work for a small non-profit (about 100 staff members) and management has decided that they want to use an online data backup system instead of our existing tape backup system. After a meeting with one of the many vendors providing this service, I must admit that I am impressed with the promise of the technology (ease of use etc). However the sysadmin side of me has a few reservations. Has anyone here on Slashdot used an online backup service, and what were your experiences?"
Kazaa counts, right?
g
Even if you go with an online service you should do some form of local backup. What happens if you lose your connection? This sort of service is perhaps best treated as an offsite backup.
There are a couple of concern you should ask the vendors,
- How much data can the offsite carry?
- Any type of compression
- What about encryption?
- What if you need your backup, how much time will it take the vendor to bring you a copy of your data on a CD or DVD ?
- Does the compagny have a secure storage ?
You can also try "easeBackup" Easy to use, support Encryption and Compression.. I am using it..
"Only wimps use tape backup real men just upload their important stuff on ftp, and let the rest of the world mirror it." - Linus Torvalds
Very apropos.
I really hate Dan Patrick.
The popularity of both Storage Area Networks and Network Attached Storages somehow seems to show that most organisations prefers to keep backup within their own control. And tapes are dirt cheap on today's standards.
Both tapes and solid state solutions can be have cheaply and some have great user friendliness. You get excellent bandwidth thrown in as well.
The only upside I can think of for online storage is that it is truely redundant, that is you will not lose your backups in event of a fire or other calamities.
But bear in mind that vendors do have the tendency to promise you the earth and deliver pebbles.
If you are talking about 20 to 50 gig of data and monthly backup then I'd say this isn't a bad idea but if you are talking 100+ gig and weekly back then this is definitely a bad idea. You also have to factor in the security of your data while it's being transported, the time it takes to backup and the overall cost of your bandwidth use. Also, a year from now you might need to backup a lot more data so factor that in.
If you ask me it is not worth the trouble, you are better of investing in your own backup system.
-----
One is born into aristocracy, but mediocrity can only be achieved through hard work.
Remember that your online backup provider's backup versions of your files are fair game for subpoenas from the RIAA or whoever else can shell out a few bucks to a lawyer.
Spend some time thinking about the circumstances under which you might need a restore service; how often, how quickly, how to verify it works, etc. This may help to clarify the issues for you.
The thing about things we don't know is we often don't know we don't know them.
"If online backups are so good, why don't you use them?"
I think a better option would be to roll your own.
Standard PC components are insanely cheap these days. Get a 4U box, chuck it full of lots of IDE drives, an IDE RAID controller or two, and Linux. Then toss samba + tar + bzip2 + yyyrsa + rsync on a local box.
Said 4U could be located at a remote office (if you have one), or possibly find another business who would be willing to swap remote storage devices with you. If all else fails, you can get colo for $50/mbps + space.
symetrix. We are building a religion, a limited edition.
I do this all the time. Usually I backup to CD-ROM but a lot of times I forget to remove the CD from the drive, thereby leaving it online. Works great. Super convenient too.
"Real men don't do backups...But you know, real men sometimes cry" :):):)
____
nico
Nico-Live
One way or another, it doesn't really matter *what* you do. If you don't back up to tape, cd, ftp, scratch monkey, reams of paper to be rescanned later, or whatever - if if you don't backup, you're fscked. To be perfectly honest, it's like mirroring - optimally, the backup copy is in a safe location offsite that only certain people can get to, and even more optimally is that if it's something that can be dismounted from a network and put in something like a safe deposit box at your bank or something. (They're less convenient than an FTP mirror or a disaster recovery shop, but much cheaper and perfect for those who has the initiative to run to the deposit box once per week with the 'granddaddy' backup.)
This sig no verb.
1) They are a labor intensive (still think we should spell that on OK style as Labour).
2) They don't have the shelf life they claim. If it ain't 100% reliable, you better have a backup of your backup.
3) The tape hardware can be really unreliable too.
4) Make sure you can really retstore just that one file you want as opposed to the entire tap.
As for Network Backups...point 4 definitely applies. All the other posts talking about network connectivity definitely apply.
I did a contract for a network backup solution. The kept buying really high end dirk arrays. However, you probably can get away with cheap disks for backups. Assuming 1 TB of slow IDE is about $250 right now (I haven'r priced in a while) You can back up 4 TB with no redundancy for $1000. Using AMANDA, you can do the partials etc for weekly, daily, monthly.
Depending on how you set this up, it can be more or less labo intensive than the tapes.
Open Source Identity Management: FreeIPA.org
We have used stand alone tape systems, automated remote tape libraries, and a network accessible backup storage array with raid. It has been my experience that any backup system will work as long as the person responsible for the backups is making sure that they happen. I have seen several occations were one of "automated" backup systems failed and no one seemed to notice until a restore was needed. The key is making sure that the backups are happenning in the first place.
I use 2 USB hard drives. I keep one off site. ( in my truck, I can get to it quickly if i need to) One stays in the shop for easy access. I rotate them at each backup. Restores are as easy as plug and copy. Also I keep a OS disk and boot floppy, cables and power supply with the off site unit just in case. It works great for us. I can back up '98, 2000, XP and Linux boxes all with one unit. :)
Hello.
I put together an entire online backup solution once. The braintrust of the project didn't want to spend a ton of money and wanted to just find another stream of income based on the internet. I told him that he'd end up spending tons of money on hardware and it'd have to be moved offsite (all of the standard backup strategy stuff) and if he wanted to be competitive, he'd have to do it securely (encryption) and reliabily.
His answer to this was to buy an IDE raid and a T1 (no tapes!). I scoffed at the idea, but wrote the entire system for him anyway. Needless to say, the whole thing worked and he has a few customers and they get their important data backed-up fine, but if he were to get any real customers or have a crash of some sort, he'd be out of business and out of luck.
My advice (similar to a previous poster) is to foll your own. I know that backups can be costly and a pain to maintain, but getting amanda running on a linux machine with a huge raid and a tape changer is a LOT better than putting your faith in an anonymous company. If you're still stuck on the online solution, see if you can take a tour of the company to see that they actually have the capacity and hardware to back up their claims.
Just my $0.02 and no, I'm not telling the name of the backup service that I built. It's not something that I'm very proud of.
That's "roll your own". 8)
Backups are done via rsync, and restores are done with cp. (The whole system is run on linux with php and mysql). The files are backed up from/restored to windows servers over samba.
This is all great and makes the users happy. However, any intelligent business will also have offsite backups. Right now, if a (pick your natural disaster/accident) happens, the company is basicaly out of business since no data resides outside the server room.
Of course since all the client data is stored in a central location, it would take nothing to add some tape backups/hot-swap HD/etc. and take them offsite once a week.
Hi, There are 2 or 3 smaller "lower end" backup providers, and one big one called . Their product starts at 99 EUR a month (multiply by 1.2 for USD) for 2 GB of compressed storage. The might sound pricy, but they do have an insurance that pays if they can't get your data back. I also read some remarks that "if you have to do a full restore, it's going to take ages over a T1 or 1Mbps dsl line". These guys actually come to your office with a small nas-server, if you need to do a full restore...
Use disks. See : Mike Rubel's rsync backup system. You can't beat IDE disks on price/gb - Tapes are MORE expensive; They're fast, available on-line, and you'll probably be able to mount them on any machine in the next 10 years (which is not true for many tape drives).
I've looked at started such a company, it sounds like a good idea. However...
Make sure management understands the risks. I know of more than one company no longer in buisness because the outsourced accounting/HR to someone who took their tax money and ran. The IRS put them out of buisness when the money that was supposed to go to the IRS went to some crooks no longer in the country. The same situation can happen to data. Then there is the risk that your data will be sold to someone else. Maybe not a problem for a non-profit, but corporate spys exist, how do you know your provider doesn't make side money selling your secerts to someone else?
Assuming you evaluate the above risks and decide to do it anyway (and despite the above it isn't a bad idea), test everything. Randomly decide one system is dead, pull the harddrive, and request a restore to a new drive. Latter, randomly decide some file was accidently deleted, rename it, and request a restore of just that once file. Do the above at least once a year. (unless you have real incidents that cover the above)
It's a little unclear if you are looking at an outsourced backup service (i.e. backing up to a service company over the network) or a disk based backup system.
I will make the assumption that you are looking at a disk based system. There are a number of factors you need to look at:
1, Capacity required. That is, how much space will you need now and in the future? Tape systems offer advantages in being (generally) able to handle large amounts of data, particularly if it's compressable.
2, Restore time. This is very important. How quickly do you need to get data back? A disk based system will obviously have advantages in that it will give you near instantaneous restores. Tape can also provide fast restores, if you get sufficiently expensive drives (e.g. AIT-3, 9840b/c).
3, Retention period. This is related to capacity requirements, particularly in disk based systems. How far back do you want to go? Tape offers the ability to easily store media offsite and import it back in.
4, Budget. Notice that I put this last. The other considerations IMHO are more important than price, particularly if you really value your data. Spend the money and get a quality system, regardless of whether you get disk or tape.
Without knowing a lot about your situation, here are some options to think about:
- If restore time is key, look at getting a NAS (Network Attached Storage) or ATA array and use rsync or standard utility (e.g. tar, dump etc) to backup to the array. You can then use a commercial product like NetBackup or Networker or an open source utility like AMANDA to archive the data to tape if you want more control over versioning and need some offsite DR.
- If capacity is your goal, tape is the most cost effective option. In spite of appearances, high capacity tapes like the 9840B, AIT-3 and LTO2 will often give lower overall cost per megabyte than disk. You will often also get more scalability, as the libraries are easily expandable in terms of slots and/or drives. You can multiplex backup/restores over several drives to get you the pwerformance you're after.
- Use snapshots. NetAPP and other NAS vendors offer this and they backup only what changes. They also give you very quick recovery times.
I favour a hybrid disk and tape approach. That is, backup to a cheap disk array using the method of your choice and then backup the array to tape (full backup on weekends, incrementals on weekdays). Use a four week cycle for your tapes, with a monthly offiste rotation. If you have a bit of budget, then HSM software like SAM-FS or Veritas Storage Migrator make the job even easier. They are basically filesystems that use a disk cache and backup/restore to/from tape in the background. You can then NFS/SAMBA mount the filesystem on any machine. You can also write multiple copies of the same files, which is handy.
Why yyyrsa instead of gpg?
Seems like if your organization's admins already have GPG/PGP keys, it would be easier to use those. (And you can even encrypt the backups so that any one of the admins could recover.)
Wolde you bothe eate your cake, and have your cake?
No one seemed to mention the fact that online backups have the benefit of being OFF-SITE. So, if your building burns down, or whatever, you don't lose all your data. Of course, whether that matters depends upon how valuable the data is compared to the building...
I work at a largish company where we do our own 'online' backup of a remote office. Around 400Gigs over a T1 line, we use rsync (works with linux and windows systems) and keep the data on an Apple XRaid (very cost effecitve and better-supported than a roll-your-own). Since the daily deltas are rarely more than a gig, this works really well.
We did this because the tape system has a long history of reliability problems. We've since fixed the tape problem (we still do tape backups) but tape is used for off-site storage and disaster recovery at remote sites -- for typical "I deleted an excel file I didn't mean to" requests we can just drill into the xraid and pull the file over without having to mount tapes, get the offsite people to bring them back etc.
Tape might seem expensive, but you have to look at the business benefit, not the cost of drives, software, etc. For example -- point-in-time backups -- our online system is great for yesterdays data, but useless for files from 3 months ago. Our tape system has monthly tapes for a year, weekly tapes for a month, and daily (full-not differntial) for a week. This has 'saved the bacon' more than once and I highly recommend it. Another good point of tape is database backups -- sure you can dump a database to disk and then rsync it offsite, but it requires that all of your database servers have much more disk capacity, and depending on how rsync treats the backup file, it could kill the entire online concept unless you've got a T3 or something.
The killer is restores though. You have to practice them and get the process nailed, otherwise your backups (online or tape) are useless.
"But actually trying to use m4 as a general-purpose langage would be deeply perverse" --ESR
Have you looked at rdiff-backup? It's a mix of a backup system and rsync, thus makes it possible to back up entire hard disks over a small connection.
-- My signature is my passport. Verify me.
Most companies only think of dollars when it comes to backup solutions, without ever thinking of what would happen if their vendor becomes financially insolvent. If, for example, your vendor goes bankrupt, their equipment could be sold to the highest bidder.
The first problem of financial insolvency is a real problem for online backup vendors because the data storage medium (their servers) is not your company's property. Therefore, in the event of bankruptcy, your firm's data could be inadvertently sold to the highest bidder.
The second consideration is related to the first - what happens if the vendor is unable to provide the backup service? Are they bonded? Will they compensate your company for the inability to provide clean backups?
The third consideration is perhaps the most important. Because your data is sitting on a running server, it is subject to corruption or outright deletion should the server be compromised.
Offsite tape backup is probably the best way to go. It is secure - even in the worst of circumstances, your data can be retrieved by driving a truck to the vendor's storage facility. There's much less possibility of accidental erasure and corruption; the possibility of your data getting into the hands of hackers is almost non-existent. Because of the fact that the tapes are legally property of your company, they can't be sold should the vendor go bankrupt.
The society for a thought-free internet welcomes you.
In the case of having personal backups I think it would be better off having the backup in the same building as the computer that is being backed up. Think for example you are stupid and delete your 100 gig hd. You have it all backed up but if you had all that on the internet.. then you would be transfering 100 gig and eating away a path to your house. In this case it might be wise to have a usb hd or a tape backup or something like that. As far as corporate offices go however... there should be multiple backups one on site and one off site. Lets say you have a large chunk you need to update. You can do that locally without stress on the outside networks. However in case of destruction or whatever case that would prevent you from using local backups a remote one would be helpful in restoring what is needed on a "Must Have" basis.
Ignore the roll your own crap. Your job and your employer's lifeblood depends upon this information. If it didn't then there'd be no reason to back it up or consider disaster recovery or business continuance. My suggesting is to go buy two Netapp filers, cheap R200 or similar SATA devices. If you really are a non-profit then you can probably get two 8TB filers for 300k or so. If that's too much look at smaller units. Then mirror them across WAN links. Configure them as RAID5 volumes with a couple of hot spares and turn on autosupport. NetApp will know of any problems before you do. You get all the marbles that way: incredible reliability, easy management, single-file user recovery of files through read-only snapshots, total data redundancy both on site and off site, and the BEST service and support of any vendor in the market for any device. I suggest you go read up on the Data OnTap operating system and the filers themselves. Then talk to more people that administer these things. You WILL be impressed.
Democrats and Republicans are like AIDS and Cancer, I want neither!
Slowdown? Between systems on my localnet work I can get at least 2-5 MB/s while transfering 10+GB using ssh with compression.
I am not so sure that SSH would slow things down enought that you would notice.
Are you going to save old backups?
This made me think about write-once solutions. Read on for my suggestion.
that is 40 cents per GB, which is still half price of what the harddrives offers
How much per GiB does blank CD-R Data or DVD+/-R media cost in countries such as the United States that don't put a copyright levy on blank media? Of course, a DVD-R backup solution won't scale to situations where the increment is much bigger than 4 GiB, but it's still worth considering.
I'd like your permission to quote your comments about your experience with SCons (from the Slashdot thread back in July) on the SCons web site. Could you drop me a line at webmaster@NOSPAMscons.org to let me know if that's cool with you? Thanks!
Sorry for the OT post, I couldn't find another handy way to get in touch.