Slashdot Mirror
Backing Up is Hard to Do?
Joe Barr writes "NewsForge is running a story this morning on a personal hardware/software backup solution for your Linux desktop (NewsForge is owned by Slashdot's parent OSTG). The solution doesn't require a SCSI controller, or tape drive, or the ability to grok a scripting language or archiving tool to work, either. It's based on point-and-click free software. Plus it includes a dead-parrot joke by Linus Torvalds."
Backing up isn't painful, restoring is.
I use rsnapshot to automate my backups to another host. Works like a dream, providing multiple virtual point in time copies (just like similar functionality from Network Appliance, etc.).
The best way to create differential backups under Unix is with hardlinked snapshots. Easy Automated Snapshot-Style Backups with Rsync has a good explanation of how to do this. The best part is that restoring is as simple as copying a file. Each snapshot is a folder hierarchy on disk, and you can browse through any snapshot and find files you want.
/tmp using mkzftree if I need to restore something.
One small improvement over rsync (IMO) is to use mkzftree from the zisofs-tools package. It's designed to create compressed ISO filesystems which will be transparently uncompressed when mounted under Linux (and other supporting operating systems; it's a documented ISO extension). mkzftree supports an option for creating hardlinked forest (like cp -al and rsync), with the advantage that the files are compressed, thus saving space. ISO isn't quite as flexible as ext2 for things like hardlinks, so what I do is have DVD-sized disk images formatted as ext2 to store the snapshots. I burn the disk images directly to DVD; each one can hold ten or twenty compressed snapshots (of my data anyway). The disadvantage is that I can't read the files directly (because they're compressed, and the transparent decompression only works with ISO) but it's easy to decompress a file or folder to
It shouldn't be hard to make the transparent decompression code work with other filesystems than ISO, as long as they're mounted read-only. The files are just gzipped with a header block indicating they are compressed.
1. Reach over and plug in USB 120 gig drive.
2. Become root, and go to /root.
3. Type "./backup.sh".
That is a script that goes to all the directories I care about (/root, /etc, /srv/www, /usr/local/share, and my home directory), and basically does this for each drive.
cd $DIR rsync -avz --progress --delete . $MNT/$DIR
where $MNT is where the USB drive mounts.
4. Unmount the drive and unplug it.
This is quick (a few minutes) and easy, and since rsync reads the files from the last backup to figure out what needs to be copied, it should catch it if I develop a bad sector on the USB drive.
I left it out in the above, but the backup script also, before doing the rsyncs, lists my crontab into a file, so that gets backed up.
Someone taught me a cool trick to backup up all files with the highest possible compression ratio and speed: mv * /dev/null
A hard drive crash over the holidays left me scrambling to get back to a productive desktop as quickly as possible. Luckily, I had my /home partition on a separate drive, so I didn't lose precious email, stories, research, and pictures. But it did get me thinking about my lack of preparedness. Where was the back-up system I've talked about for years, but never acquired? This is the tale of how I rectified that glaring omission, and built myself a personal back-up system using inexpensive parts and free software.
/home directory on hdd. Backing up directly to CD would be too slow and too cumbersome for me, so the first thing I needed was some new hardware.
The hardware
My desktop machine includes three IDE drives and an ATAPI CD-ROM drive. I have Debian installed on hda, SUSE on hdc, and my
In the past I've researched tape drives and found that for a decent drive, I would also have to add a SCSI controller. Those two items can be pretty pricey. I opted for a less expensive configuration.
I decided to go with a removable IDE drive, connected via USB. I bought a 3.5-inch hard disk enclosure with USB 2.0 connectivity on eBay. It cost roughly $45, including shipping. With three drives to backup, I needed a large-capacity IDE drive to hold all the data. It turns out I already had one, just waiting for me to use. I raided the stash of goodies I've been hoarding to build a killer MythTV box and found a 250GB Hitachi DeskStar -- just what the doctor ordered. I got it on sale at Fry's Electronics a couple of months ago for $189.
I have the mechanical skills of a three-toed sloth, but I still managed to cobble together the drive and the enclosure, neither of which came with directions. Four screws hold the faceplate on the enclosure, and four more hold the drive in place inside. Even I was able to puzzle it out.
The most difficult part was the stiffness of the IDE cable running between the faceplate and the drive. In hindsight, I recommend connecting the power and data cables from the faceplate to the drive before screwing the drive in place inside the enclosure. I also recommend not forgetting to slide the top of the enclosure back in place before reattaching the faceplate.
I connected the USB cable to the enclosure and the PC and powered on. Using the SUSE partitioning tool, I created an ext3 filesystem and formatted it on the Hitachi drive, using the default maximum start and stop cylinders. That worked, but there was a problem. My great big 250GB drive yielded only 32GB.
One of my OSTG cohorts asked if had clipped the drive for 32GB max, but I had done no such thing. All I did was check to see how the drive was strapped out of the box. It was set to Cable Select, which was fine with me, so I left it like that. His question worried me, though, because I had never heard of a 32GB clip thingie before.
I called Hitachi support to find out what was up with that. Their tech support answered quickly. When I explained what was going on, he agreed that it sounded like it was clipped to limit its capacity. This functionality allows these big honkers to be used on old systems which simply cannot see that much space. Without it, the drive would be completely unusable on those machines.
I asked why in the world they would ship 250GB drives configured for a max of 32GB by default, and he denied that they had. He asked where I got the drive, then suggested that Fry's had "clipped" it for some reason. There are jumper settings to limit the capacity, but my drive had not been jumpered that way. Perhaps Fry's sold me a returned drive that a customer had "clipped", then returned the jumpers to their original position. We'll never know.
The tech told me how it should be jumpered for Cable Select without reducing capacity. I opened the USB enclosure, pulled out the drive, and found it was already jumpered as he described. Undaunted, I pressed on.
On the Hitachi support page for the drive, I found a downloadable tool wh
He plugs in a USB drive, runs KDar to fill it with stuff.
/bin /usr, etc, which I then burn onto a couple of DVD9-Rs. I can run this to recreate my system.
/home.
Now, when his system borks, how does he restore? Or did he think that far ahead?
I skimmed the article, and nothing about restoring. Your backup is useless if you can't restore it.
Does he have to install and configure linux, X, and KDE just to be able to access KDar?
Forget all this jibberjabber, and emerge or apt-get or type whatever command you use to get Mondo/Mindi. Just perfect for home boxes, and most other use.
Burn yourself a bootable CD that can recreate your box, just like Norton Ghost for Linux. I have it write out the iso files and boot disk for
I run a seperate job to backup
Whats important, is to seperate system from user data when it comes to backups. This also forms my "archiving" system, since old "/home" backups stick around, so if I want to take a look at the version of foo.c I was writing 6 months ago, it's easy enough to find.
As much as I love Mondo/Mindi, it's not the be-all and end-all. AMANDA is a better choice for a corporate (more elaborate) environment. It's a PITA and not worth getting involved with for a simple user box.
I don't need no instructions to know how to rock!!!!
Restore is also straightforward - it can be done in place, or by downloading a zip/tar file.
The Raven
Is that some kind of sense that allows you to pick out other KDE users in a crowd?
With just one or two boot floppies, I can back up and restore my Linux drives to either: other internal IDE drives, other parititons on the same drive, external USB1 and USB2 drives, burnable CDs, or burnable DVDs.
Heck, it is so fast and reliable, I've been known to backup the drive just before even *trying out* new software or options, and if I don't like it, I just Ghost it back to how it was.
Now, I know it isn't free, or even Linux based, but it is hard to argue with cheap, reliable, and fast backup procedures that just work all the time...
I've read the whole article. My! You'd better be a geek to have to cope with all the little worries..
Getting cheap AND working hardware on E-Bay. My mom will not do it for the sake of her computer.
32GB limitation by jumpers. Not obvious for an end-user.
Booting up *nixes from various drives in order to access the limited drive, then fiddle with partitions. I still don't dare touching my configs for more than OS at a time. Let alone various OSes on various drives.
Compiling KDart?! Compiling what? What do I have to do? "Comp..??" You have to admit, it's not for the dummy kind.
Definitely not "Backup made easy" but "Made not so expensive" since the price tag still reaches 300$ (drive + box from e-bay + screws + shots of valium to calm you down when your machine refuses to boot after all the offence you just did to it).
I bought Linux Hacks. This, Webmin and a remote machine accessible using Samba or sftp does the daily backup just fine.
In case you have a seperate computer or a seperate drive one can use rsync to relativly easily create backups, its just a few lines of Shell:
/your_directory_to_backup \
/your_directory_to_backup to user@other_host:/backup/current/ and in addition to that keeps all the changes you did to that directory in a seperate 'dated' directory like /backup/2005-01-15, so you can also recover files that you deleted some days ago. Some other posters seem to have missed the '--backup' option, which is why I repost the rsync trick.
rsync -e ssh \
--delete \
--relative \
--archive \
--verbose \
--compress \
--recursive \
--exclude-from list_of_files_you_don't_wanna_backup \
--backup \
--backup-dir=/backup/`date -I` \
user@other_host:/backup/current/
This command mirrors everything in
Disadvantage is that you can't easily restore an exact old state of the directory which you backuped, however you can retrieve all the files very easily.
There are also floating some shell scripts around which add to the above rsync line some vodoo to hardlink the different dated directories, so that you have a normal browsable copy of each and every day while only wasting the space for the changes.
And there are also tools which optimize this whole thing a bit more, by compressing the changes you did to files, like http://www.nongnu.org/rdiff-backup/
However overall I found the plain rsync solution the most pratical, since it doesn't require special tools to access the repo and 'just works' the way I need it.
I make daily differential backups (via AMANDA) to a rotating set of 12 tapes. If I accidentally delete /etc/shadow or some other important file, I have nearly two weeks to discover the problem and restore a previous version from tape. Your idea gives you, oh, until about the time that rsync discovers the missing file and dutifully nukes it from your "backup" drive.
What you're doing is certainly better than nothing, but it's not a backup solution by any definition of the term beyond "keeps zero or one copy of the file somewhere else".
Far, far better would be for your script to use dump or tar to create incremental backup files on your USB drive and to rotate them out on a regular basis.
Dewey, what part of this looks like authorities should be involved?
I have a 20GB iPod, but only about 12GB is used. My $HOME is about 2GB, including a bunch of digital photos, but also a bunch of documents, my email, and other stuff I'd rather not lose.
My solution is simple:
This is a very simple shell script that deletes the backup file already on the iPod, then does a 'tar czf - $HOME' and pipes it into gpg using circular encryption (that is, a passphrase.) The encrypted, compressed tarball (about 1.7GB) is written directly to the iPod. Takes about 20 minutes.
I've used this backup copy to do restores, and it's really as simple as plugging in the iPod, using gpg to descrypt the file, piping that into 'tar xvzf -' to re-create my $HOME. I can move all my stuff back to where it needs to be after that.
(For those who wonder: I always make an encrypted backup file in case my iPod is ever lost or stolen. Sure, the bad guy can probably run something to brute force the passphrase, if that's something he's interested in doing, but it's a tough passphrase. I don't worry about it so much, and it's "only" email and family photos.)