Slashdot Mirror
What Software Do You Use for Unix Backups?
jregel asks: "Linus has stated that
dump should not be considered a reliable backup program, and both tar and cpio have their limitations. So what are Slashdot readers doing for backing up Linux servers and workstations? (you do backup, right?)" Given this bit of news, have you used anything other than the standard Unix staple to back up your Linux boxes? If you were forced off of tar, cpio and dump, what would you use as a replacement?
I wrote my own (Perl) script, that copies all my "important" files (basically stuff in my home directory that can't be reconstructed by other means and all the system config files) to a new directory tree (using cpio) it then burns the copied tree to CD-RW and verifies the CD against the copied tree.
I operate a 4 disc system, so I always have the last four backups on CD and I keep the copied trees around (uncompressed) for as long as I have disk space. So far I've not needed the CDs (I store 2 of them offsite in case of disaster) but the copied filesystem trees have come in useful a couple of times.
The only drawback of this is it's not appropriate for backing up huge quantites of data (like lots of audio or video files) as the CD media is quite limited in size - but when rewritable holographic storage comes along I'll be able to just change my function that decides which files are "important".
#exclude <ms/windows.h>
cdbkup is a little more sophisticated - multiple levels, multiple disks.
"CDBKUP is a professional-grade open-source package for backing up filesystems onto CD-Rs or CD-RWs."
Note to ACs: I won't mod you up, even if you are being funny or insightful. So take a chance! It's not real life!
FreeBSD 5.x has fs snapshot capabilities. See http://www.freebsd.org/releases/5.0R/relnotes-i386 .html#AEN1150 for more details.
-- I love the smell of Blue Screens in the morning.
- No support for spanning tapes with a dump; i.e. if you need to dump 4GB on a 2GB and you can't compress it down, you're stuffed.
- Restoring files is fiddly
Yup, Amanda is great for small setups (I use it myself at home) but it lacks certain features to make it really usable. For example, I had to restore some files in Legato Networker; I was able to open up a GUI, navigate to the file and set the restore path (i.e. where it will restore to). With that done, it worked out which tapes the file was on and restored.I found out that solaris has a very interesting command: fssnap
If you are using EMC Symmetrix storage, you can use the TimeFinder product to create a "Business Continuence Volume", or BCV. It deltas against your last backup (at the track level, not files or blocks), applies changes to a copy of the last backup to create a consistent image, then you can dump that to tape.
I wonder if there's something like this for linux...
So long as you have one host (Solaris, NT, whatever) to run the TimeFinder client on, you can use the Symmetrix to provide storage to as many Linux boxes as you want.
Why should you need to modprobe it at all? Don't you have it set up in /etc/modules to auto-install the module at boot?
They say tar has its limitations. I really dont understand.
Ive worked with different unixen and Linux distros, so I just dont want to be dependant on something that isnt installed by default everywhere. tar already has a VERY well known format and execution parameters.
Ive lost my fair share of data to buggy harddrives and dumb mistakes like pulling off the ide cable while the system is running. So cron does daily backups using tar cfj using a file that has a list of other files to be backed up. This way I dont have to backup the whole partition. To restore a certain file, just tar xvfj backup2.tar.bz2
The cron setup renames backup.bz2 to backup2.bz2 and removes backup2.bz2 so I have the data for the past two days. Beside incremental backup which I dont need due to this setup, what else could I need? And by the way the backup.bz2 is copied off onto an NFS share elsewhere incase my whole RAID setup crashes, or the XFS filesystem bombs out. This setup can be replicated onto FreeBSD Solaris and many others.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
For backing up my FreeBSD home server I use a second (identical) HDD in a swappable IDE bracket on a standard plain ole onboard IDE controller (the 2nd channel to be precise). Though hotswapping isn't really supported on these controllers, it does seem to work :)
/home, just everything) to the hotswap HDD, using the "ssync" tool /var/log/backup
Making a backup is easy. I just plug in the bracket and start a homebrew script which:
- enables and inits the hotswap IDE channel
- mounts the partitions on the hotswap HDD
- removes system immutable flags on files on the hotswap HDD (so that they can be overwritten)
- copies over all new files (/sbin,
- resets the system immutable flags to their original state
- umounts the partitions
- disables the IDE channel
- logs the above to
- mails the log to me
A whole backup takes about 25 minutes on an almost full 30GB disk in a P200 (it only copies the new/changed files), and compared to tape it's very cheap. If the master drive fails, I swap HDDs and the whole server works again, without any configuration whatsoever.
Tivoli Storage Manager is the only "backup solution" that I have ever seen that truly works well without alot of tweaking and twiddling.
I've worked at places using Legato and Amanda, where restoring from backup was an unreliable and error-prone process more likely to be a waste of time than anything else.
TSM is not cheap, but is worth every penny. We have one full time and one part time employee handle the backup/restore jobs for about 2000 servers. Try that with Legato or Amanda.
Conformity is the jailer of freedom and enemy of growth. -JFK
Companies with money can get a netapp box for critical data. Here you can absolutly use dump, tar or cpio. They create a "snapshot" of a file before backing it up.
Unfortunately we are talking a minimum of $40k for this type of solution.
If the snapshot comcept could be written into ext4, then dumps would be great.
I always put a caveat into my backup policies to cover this issue.
Anybody out there know if Bru does any better?
Veritas have a work arund to this?
Amanda is just a wrapper around dump.
(BTW, IBM appears to have discontinued their Linux LVM project - the Sistina Linux LVM is the best way to go. This Linux LVM is quite similar in commands/functionality as the HP-UX LVM. In fact, full-featured LVMs for Linux have been around longer than similar capabilities on Solaris ...)
Tar works fine when the user actually remembers the exact path and filename.
It shouldn't be any major problem to build indexes over your tar files. What Linus argues that dump does incorrectly and tar does correctly is the reading of the filesystem. The way to read a live filesystem is through the filesystem API, not by blockwise reading of the device. All your arguments against tar and in favor of dump is about the output produced by the program. If you want output like dump produces it, then write a tool to read the filesystem in the way tar does and write output in the way dump does.
Do you care about the security of your wireless mouse?