Backing Up is Hard to Do?

Joe Barr writes "NewsForge is running a story this morning on a personal hardware/software backup solution for your Linux desktop (NewsForge is owned by Slashdot's parent OSTG). The solution doesn't require a SCSI controller, or tape drive, or the ability to grok a scripting language or archiving tool to work, either. It's based on point-and-click free software. Plus it includes a dead-parrot joke by Linus Torvalds."

Comments

Easy

[harlows_monkeys]

Here's what I do:

1. Reach over and plug in USB 120 gig drive.

2. Become root, and go to /root.

3. Type "./backup.sh".

That is a script that goes to all the directories I care about (/root, /etc, /srv/www, /usr/local/share, and my home directory), and basically does this for each drive.

cd $DIR rsync -avz --progress --delete . $MNT/$DIR

where $MNT is where the USB drive mounts.

4. Unmount the drive and unplug it.

This is quick (a few minutes) and easy, and since rsync reads the files from the last backup to figure out what needs to be copied, it should catch it if I develop a bad sector on the USB drive.

I left it out in the above, but the backup script also, before doing the rsyncs, lists my crontab into a file, so that gets backed up.

In case of /. ing

[Pig Hogger]

A hard drive crash over the holidays left me scrambling to get back to a productive desktop as quickly as possible. Luckily, I had my /home partition on a separate drive, so I didn't lose precious email, stories, research, and pictures. But it did get me thinking about my lack of preparedness. Where was the back-up system I've talked about for years, but never acquired? This is the tale of how I rectified that glaring omission, and built myself a personal back-up system using inexpensive parts and free software.

The hardware

My desktop machine includes three IDE drives and an ATAPI CD-ROM drive. I have Debian installed on hda, SUSE on hdc, and my /home directory on hdd. Backing up directly to CD would be too slow and too cumbersome for me, so the first thing I needed was some new hardware.

In the past I've researched tape drives and found that for a decent drive, I would also have to add a SCSI controller. Those two items can be pretty pricey. I opted for a less expensive configuration.

I decided to go with a removable IDE drive, connected via USB. I bought a 3.5-inch hard disk enclosure with USB 2.0 connectivity on eBay. It cost roughly $45, including shipping. With three drives to backup, I needed a large-capacity IDE drive to hold all the data. It turns out I already had one, just waiting for me to use. I raided the stash of goodies I've been hoarding to build a killer MythTV box and found a 250GB Hitachi DeskStar -- just what the doctor ordered. I got it on sale at Fry's Electronics a couple of months ago for $189.

I have the mechanical skills of a three-toed sloth, but I still managed to cobble together the drive and the enclosure, neither of which came with directions. Four screws hold the faceplate on the enclosure, and four more hold the drive in place inside. Even I was able to puzzle it out.

The most difficult part was the stiffness of the IDE cable running between the faceplate and the drive. In hindsight, I recommend connecting the power and data cables from the faceplate to the drive before screwing the drive in place inside the enclosure. I also recommend not forgetting to slide the top of the enclosure back in place before reattaching the faceplate.

I connected the USB cable to the enclosure and the PC and powered on. Using the SUSE partitioning tool, I created an ext3 filesystem and formatted it on the Hitachi drive, using the default maximum start and stop cylinders. That worked, but there was a problem. My great big 250GB drive yielded only 32GB.

One of my OSTG cohorts asked if had clipped the drive for 32GB max, but I had done no such thing. All I did was check to see how the drive was strapped out of the box. It was set to Cable Select, which was fine with me, so I left it like that. His question worried me, though, because I had never heard of a 32GB clip thingie before.

I called Hitachi support to find out what was up with that. Their tech support answered quickly. When I explained what was going on, he agreed that it sounded like it was clipped to limit its capacity. This functionality allows these big honkers to be used on old systems which simply cannot see that much space. Without it, the drive would be completely unusable on those machines.

I asked why in the world they would ship 250GB drives configured for a max of 32GB by default, and he denied that they had. He asked where I got the drive, then suggested that Fry's had "clipped" it for some reason. There are jumper settings to limit the capacity, but my drive had not been jumpered that way. Perhaps Fry's sold me a returned drive that a customer had "clipped", then returned the jumpers to their original position. We'll never know.

The tech told me how it should be jumpered for Cable Select without reducing capacity. I opened the USB enclosure, pulled out the drive, and found it was already jumpered as he described. Undaunted, I pressed on.

On the Hitachi support page for the drive, I found a downloadable tool wh

Heh, noob mistake

[stratjakt]

He plugs in a USB drive, runs KDar to fill it with stuff.

Now, when his system borks, how does he restore? Or did he think that far ahead?

I skimmed the article, and nothing about restoring. Your backup is useless if you can't restore it.

Does he have to install and configure linux, X, and KDE just to be able to access KDar?

Forget all this jibberjabber, and emerge or apt-get or type whatever command you use to get Mondo/Mindi. Just perfect for home boxes, and most other use.

Burn yourself a bootable CD that can recreate your box, just like Norton Ghost for Linux. I have it write out the iso files and boot disk for /bin /usr, etc, which I then burn onto a couple of DVD9-Rs. I can run this to recreate my system.

I run a seperate job to backup /home.

Whats important, is to seperate system from user data when it comes to backups. This also forms my "archiving" system, since old "/home" backups stick around, so if I want to take a look at the version of foo.c I was writing 6 months ago, it's easy enough to find.

As much as I love Mondo/Mindi, it's not the be-all and end-all. AMANDA is a better choice for a corporate (more elaborate) environment. It's a PITA and not worth getting involved with for a simple user box.

Backup Easy!

[Anonymous Coward]

Try this:

mirrordir

cron it out hourly

or

find /home/joeuser/etc | afio -o -v -Z -L /home/joeuser/backups/backup.log /dev/st0

And that is easy

[flibuste]

I've read the whole article. My! You'd better be a geek to have to cope with all the little worries..

Getting cheap AND working hardware on E-Bay. My mom will not do it for the sake of her computer.

32GB limitation by jumpers. Not obvious for an end-user.

Booting up *nixes from various drives in order to access the limited drive, then fiddle with partitions. I still don't dare touching my configs for more than OS at a time. Let alone various OSes on various drives.

Compiling KDart?! Compiling what? What do I have to do? "Comp..??" You have to admit, it's not for the dummy kind.

Definitely not "Backup made easy" but "Made not so expensive" since the price tag still reaches 300$ (drive + box from e-bay + screws + shots of valium to calm you down when your machine refuses to boot after all the offence you just did to it).

I bought Linux Hacks. This, Webmin and a remote machine accessible using Samba or sftp does the daily backup just fine.

here's how I perform backups

[ywwg]

if [ `df |grep /media/BACKUP |wc -l` == "0" ]
then
echo Backup drive not mounted, skipping procedure
exit 2
fi
cd /media/BACKUP
nice -n 10 rsync -va --exclude-from=/root/exclude $1 $2 $3 $4 $5 / .

where /root/exclude contains:

/mnt
/proc
/tmp
/udev
/sys
/media

Not the prettiest implimentation, but it works.

Re:here's how I perform backups

[Anonymous Coward]

It's begging for some shell script golf...

if [ grep -q /media/BACKUP /etc/mtab ]
then
echo Backup drive not mounted, skipping procedure
exit 2
fi
cd /media/BACKUP && nice -n 10 rsync -va --exclude-from=/root/exclude $* / .

I keep backups on my iPod

[Jim Hall]

I have a 20GB iPod, but only about 12GB is used. My $HOME is about 2GB, including a bunch of digital photos, but also a bunch of documents, my email, and other stuff I'd rather not lose.

My solution is simple:

1. Plug in iPod
   
   
2. Run ~/bin/backup.sh
   This is a very simple shell script that deletes the backup file already on the iPod, then does a 'tar czf - $HOME' and pipes it into gpg using circular encryption (that is, a passphrase.) The encrypted, compressed tarball (about 1.7GB) is written directly to the iPod. Takes about 20 minutes.
   
   
3. Eject the iPod
   
   
4. Done!
   
   

I've used this backup copy to do restores, and it's really as simple as plugging in the iPod, using gpg to descrypt the file, piping that into 'tar xvzf -' to re-create my $HOME. I can move all my stuff back to where it needs to be after that.

(For those who wonder: I always make an encrypted backup file in case my iPod is ever lost or stolen. Sure, the bad guy can probably run something to brute force the passphrase, if that's something he's interested in doing, but it's a tough passphrase. I don't worry about it so much, and it's "only" email and family photos.)

My backup script:

[chrisopherpace]

tar -lcvf /backup/$HOSTNAME-$DATE.tar /

(I'm the type that creates one big ass partition for /). The -l switch tells tar to stay on the same FS, as /backup is NFS mounted to a RAID array. Thus, I just backup the local machine, without having to specify which directories to backup, and which to skip.

Restoration, I do the lazy way:
mkdir test
cd test
tar -xvf /backup/whatever.tar

and then I grab the files (the RAID array usually has plenty of space).

I don't know where to start. What a reeking POS!

[paenguin]

So, first, we have this "Copy" system that's being called a "Backup" system. PHOOEY.

Next, he admits that the "Backup" system can't restore files or directories. OMG!!

Anybody who adopts this system of backups better be praying to the hard drive gods and be making regular and appropriate homage.

For a REAL backup method that can stand the test of time, try this:

http://www.samag.com/documents/s=7033/sam0204c/sam 0204c.htm

It's a system that compresses each file individually, writes them out to a temp directory, creates an iso and then writes the iso to a CD. This way, single bit errors in the compressed archive don't kill the entire thing, just a single file. This becomes more important as your backups begin to age because perfect playback of bits becomes more difficult as the media they are stored on ages.

Any system that creates a data stream of all files and then compresses it is prone to total loss of data beyond any significant error in the playback.

The order is everything.

Stream all files through the compress algorithm... Very Risky.

Compress each file individually and stream it to the archive... Very Safe.

Write everything to cheap read only media... Very smart.

Depending on the data, you can get several Gig's onto a single CD. High quality CD's should be readable for decades.