Slashdot Mirror
Backing Up is Hard to Do?
Joe Barr writes "NewsForge is running a story this morning on a personal hardware/software backup solution for your Linux desktop (NewsForge is owned by Slashdot's parent OSTG). The solution doesn't require a SCSI controller, or tape drive, or the ability to grok a scripting language or archiving tool to work, either. It's based on point-and-click free software. Plus it includes a dead-parrot joke by Linus Torvalds."
I use rsnapshot to automate my backups to another host. Works like a dream, providing multiple virtual point in time copies (just like similar functionality from Network Appliance, etc.).
To backup: rsync -avv -e ssh . backupuser@backupserver:backuppath .
To restore: rsync -avv -e ssh backupuser@backupserver:backuppath
The most important thing about backups IMHO is that they be off-site. It's a silly amount of false security when your backup device is in the same room as the computer you're backing up; and the same failure (failed airconditioner; fire; leaking roof; power surge; angry girlfriend) destroys both systems.
Dead Parrot Sketch
The cast:
MR. PRALINE John Cleese
SHOP OWNER Michael Palin
The sketch:
A customer enters a pet shop.
Mr. Praline: 'Ello, I wish to register a complaint.
(The owner does not respond.)
Mr. Praline: 'Ello, Miss?
Owner: What do you mean "miss"?
Mr. Praline: I'm sorry, I have a cold. I wish to make a complaint!
Owner: We're closin' for lunch.
Mr. Praline: Never mind that, my lad. I wish to complain about this parrot what I purchased not half an hour ago from this very boutique.
Owner: Oh yes, the, uh, the Norwegian Blue...What's,uh...What's wrong with it?
Mr. Praline: I'll tell you what's wrong with it, my lad. 'E's dead, that's what's wrong with it!
Owner: No, no, 'e's uh,...he's resting.
Mr. Praline: Look, matey, I know a dead parrot when I see one, and I'm looking at one right now.
Owner: No no he's not dead, he's, he's restin'! Remarkable bird, the Norwegian Blue, idn'it, ay? Beautiful plumage!
Mr. Praline: The plumage don't enter into it. It's stone dead.
Owner: Nononono, no, no! 'E's resting!
Mr. Praline: All right then, if he's restin', I'll wake him up! (shouting at the cage) 'Ello, Mister Polly Parrot! I've got a lovely fresh cuttle fish for you if you
show...
(owner hits the cage)
Owner: There, he moved!
Mr. Praline: No, he didn't, that was you hitting the cage!
Owner: I never!!
Mr. Praline: Yes, you did!
Owner: I never, never did anything...
Mr. Praline: (yelling and hitting the cage repeatedly) 'ELLO POLLY!!!!! Testing! Testing! Testing! Testing! This is your nine o'clock alarm call!
(Takes parrot out of the cage and thumps its head on the counter. Throws it up in the air and watches it plummet to the floor.)
Mr. Praline: Now that's what I call a dead parrot.
Owner: No, no.....No, 'e's stunned!
Mr. Praline: STUNNED?!?
Owner: Yeah! You stunned him, just as he was wakin' up! Norwegian Blues stun easily, major.
Mr. Praline: Um...now look...now look, mate, I've definitely 'ad enough of this. That parrot is definitely deceased, and when I purchased it not 'alf an hour
ago, you assured me that its total lack of movement was due to it bein' tired and shagged out following a prolonged squawk.
Owner: Well, he's...he's, ah...probably pining for the fjords.
Mr. Praline: PININ' for the FJORDS?!?!?!? What kind of talk is that?, look, why did he fall flat on his back the moment I got 'im home?
Owner: The Norwegian Blue prefers keepin' on it's back! Remarkable bird, id'nit, squire? Lovely plumage!
Mr. Praline: Look, I took the liberty of examining that parrot when I got it home, and I discovered the only reason that it had been sitting on its perch in the
first place was that it had been NAILED there.
(pause)
Owner: Well, o'course it was nailed there! If I hadn't nailed that bird down, it would have nuzzled up to those bars, bent 'em apart with its beak, and
VOOM! Feeweeweewee!
Mr. Praline: "VOOM"?!? Mate, this bird wouldn't "voom" if you put four million volts through it! 'E's bleedin' demised!
Owner: No no! 'E's pining!
Mr. Praline: 'E's not pinin'! 'E's passed on! This parrot is no more! He has ceased to be! 'E's expired and gone to meet 'is maker! 'E's a stiff! Bereft of life, 'e
rests in peace! If you hadn't nailed 'im to the perch 'e'd be pushing up the daisies! 'Is metabolic processes are now 'istory! 'E's off the twig! 'E's kicked the
bucket, 'e's shuffled off 'is mortal coil, run down the curtain and joined the bleedin' choir invisibile!! THIS IS AN EX-PARROT!!
(pause)
The best way to create differential backups under Unix is with hardlinked snapshots. Easy Automated Snapshot-Style Backups with Rsync has a good explanation of how to do this. The best part is that restoring is as simple as copying a file. Each snapshot is a folder hierarchy on disk, and you can browse through any snapshot and find files you want.
/tmp using mkzftree if I need to restore something.
One small improvement over rsync (IMO) is to use mkzftree from the zisofs-tools package. It's designed to create compressed ISO filesystems which will be transparently uncompressed when mounted under Linux (and other supporting operating systems; it's a documented ISO extension). mkzftree supports an option for creating hardlinked forest (like cp -al and rsync), with the advantage that the files are compressed, thus saving space. ISO isn't quite as flexible as ext2 for things like hardlinks, so what I do is have DVD-sized disk images formatted as ext2 to store the snapshots. I burn the disk images directly to DVD; each one can hold ten or twenty compressed snapshots (of my data anyway). The disadvantage is that I can't read the files directly (because they're compressed, and the transparent decompression only works with ISO) but it's easy to decompress a file or folder to
It shouldn't be hard to make the transparent decompression code work with other filesystems than ISO, as long as they're mounted read-only. The files are just gzipped with a header block indicating they are compressed.
Just do an NFS or SMB mount:
/mnt/backup /mnt/drive /mnt/backup .
mount -t smbfs -o username:password \\10.0.1.111\backup
cd
tar -cvjf
(If I recall the commands correctly.) I use this all the time to make quick snapshots of my Gentoo installation before emerging some bleeding edge package.
I don't know what kind of crack I was on, but I suspect it was decaf.
Restore is also straightforward - it can be done in place, or by downloading a zip/tar file.
The Raven
Here's my solution
/home /etc
Backup:
tar -czf backup.tar.gz
Then use k3b or something to record the file to CD
Restore:
Take a wild guess:-)
Restore individual files:
Use mc to browse the tarball (slow but works)
Now, do you see me bragging about this trivial shit on slashdot? No?
Eh, wait...
Copying data to a single IDE drive and calling it "backup" is just pathetic.
He should read the Tao of Backup http://www.taobackup.com/ and be enlightend.
Actually we use rsync for incremental backups and it works quite well. Its a simple modification or scripting of rsync commands and can be all scripted away pretty easily..
b-loo
In case you have a seperate computer or a seperate drive one can use rsync to relativly easily create backups, its just a few lines of Shell:
/your_directory_to_backup \
/your_directory_to_backup to user@other_host:/backup/current/ and in addition to that keeps all the changes you did to that directory in a seperate 'dated' directory like /backup/2005-01-15, so you can also recover files that you deleted some days ago. Some other posters seem to have missed the '--backup' option, which is why I repost the rsync trick.
rsync -e ssh \
--delete \
--relative \
--archive \
--verbose \
--compress \
--recursive \
--exclude-from list_of_files_you_don't_wanna_backup \
--backup \
--backup-dir=/backup/`date -I` \
user@other_host:/backup/current/
This command mirrors everything in
Disadvantage is that you can't easily restore an exact old state of the directory which you backuped, however you can retrieve all the files very easily.
There are also floating some shell scripts around which add to the above rsync line some vodoo to hardlink the different dated directories, so that you have a normal browsable copy of each and every day while only wasting the space for the changes.
And there are also tools which optimize this whole thing a bit more, by compressing the changes you did to files, like http://www.nongnu.org/rdiff-backup/
However overall I found the plain rsync solution the most pratical, since it doesn't require special tools to access the repo and 'just works' the way I need it.
The best solution for your backup problems is to learn to prioritize. No, you don't need to save your pr0n collection. No, you don't need to save every .jpg anyone's ever sent to you. No, you don't need to save every bad joke e-mail you've ever received. No, you don't need to save... you don't need to save... don't save... don't need.
When I was young (early 20s) I saved everything. Then I had an HD crash. I started over and, several years later, my new HD inherited an unrecoverable problem. I started over and then went through a run of about 4 Western Digital Caviar drives which each lasted about one year.
So... what do I save now? As little as possible. Believe me, when you're going through your tree and say to yourself,"well... I might want to save that just in case..." do yourself a favor and hit DELETE. Anytime the word "I might" or "just in case" comes to mind, hit DELETE.
After you've reprogrammed yourself to have a sane set of backup priorities, you'll find that rsync and tar are more than adequate.
For corporate solutions... well, that's a whole different story. Then you're getting paid to feed someone else's legal paranoia. Just buy more drives.
fast as fast can be. you'll never catch me.
PAR2 can help with #3, provided your backup's filesystem remains readable.
http://flexbackup.sourceforge.net/ might help.
No Sig for you.!
mkcdrec is a really neat program that packs up your whole system and makes a recovery disk. Its something any sysop should take a look at.
See the homepage here.
backup2l does a great job at figuring out which files are new or have been modified for incremental backups. Easy to configure a very lightweight.
i've used mondo/mindi for backups from a gentoo system. a few minor quirks.
first, the mondo developer doesn't play very nice with the community, and there isn't a lot of community involvement on the mondo forums (who wants to spend time on a archive software forum).
second, mondo backups are incompatible between versions. some time back i was using mondo 1.6.x or some such. i got rid of that machine, and a month later wanted to restore some data from the backups. the new machine had mondo 2.x installed. it completely bailed on restoring from the older archive. since i was on gentoo i could easily install the version of mondo i needed to restore from. just a slight issue, especially if you're using gentoo with automated updates.
thirdly, mondo handles files in a "propriatary" manner. there's gobs and gobs of different archive files on the disk and finding which one has the file you want (outside of the mondorestore interface) is challenging. i don't understand why one big ass tgz file wouldn't be sufficient, but i'm sure there's reasons.
after quite some time w/o a backup, i'm off to re setup my mondo backups. mondo doesn't work nicely with cron either. takes a little work to grab and at script and then cron that.
i can say that having mondo backups and restoring from them has been handy, handy.
This is a very simple shell script that deletes the backup file already on the iPod, then does a 'tar czf - $HOME' and pipes it ...
No. Don't do this, as it _will_ bite you, no exceptions. Never delete the old files before the new backup is down on the platter, preferably checked in some way. Yes, you will need more disk, but there really is no alternative.
I lost ten years of email and personal files to a dying Deathstar because I was foolish enough to not be paranoid enough about my backups.
Nowadays I keep (automatic) nightly backups for at least a month, with an (also automatic) off-site copy of every dump file saved on my computer at work. (VDSL rocks.)
Nothing instills backup discipline like listening to your disk doing its russian tractor impression just when you deleted the old backup to make room for a new one.
Because RAID-1 is an exact mirror, I get a complete, bootable backup copy of my system at the time of the shutdown. Downtime is limited to the few minutes it takes to shut down and swap drives. The lengthy process of mirror rebuilding takes place while the system runs normally. And of course, RAID also protects me against random (single) hard drive failures.
This solves the full image backup problem, leaving only the more frequent partial backups you should also be doing. For this, rsync is your friend. The stuff that changes most often on my system are my IMAP folders, which I periodically (several times per day) rsync to my laptop. Besides backing up my mail server, this gives me copies I can carry around and read when I'm offline.
Tape is obsolete. It's just too slow, expensive, unreliable and small. Hard drives are so cheap, fast and capacious that there's little excuse to not run RAID on any machine that can physically hold more than one hard drive. Unfortunately, this leaves out most laptops.