Slashdot Mirror
It's 2006 and Backups For Home User Still Tricky?
CranberryKing asks: "What is it about backups that always seems so difficult? I am trying to do a simple backup on my home XP system/s (about 30GB of files) that will write to my DVD burner. I don't want compression (most of it is MP3s, which don't compress well). I want a routine to simply write my selection to the DVD writer and spread it across however many discs are required (rather than me manually approximating and copying to each disc). I want the files on the disc readable from any system, so no proprietary backup wrapper or DAT files, please. My last attempt was using a free program that looked good called Simply Safe Backup, but it created two coasters before crashing with an unknown error. If I can just get a full backup to work smoothly, then I'll worry about scheduling, incremental, and encryption. This seems like a very common scenario for home & small offices. Is there an elegant, reliable & cheap (free) solution to this?"
dd if=/dev/hda1 of=mywindowspartition.image
http://en.wikipedia.org/wiki/Dd_(Unix)
That may be fine for folks like my dad that has a 30gig drive that's 80% empty.
But some folks have 3 terabytes (not porn btw) of HTPC stuff, considering how cheap hard drives are now and there's no good way to back up that much data.. And with hard drives getting cheaper by the day, it seems that the only thing to do is just keep adding more drives. You reach a point of no return where you just have to take the risk of losing your stuff.
I'm looking forward to the 1tb drives that have been promised by years end. Drop 6-8 of those in a vanilla budget box and use it as your backup, power it up only when needed.
I've been using Bacula for a while now. Backing up windows clients, and linux clients. The server side isn't too bad to setup (rpms are available), and configs are standard Linux config style. The nice part, is that it can backup to any media, hard drives, tapes, DVD, CD, etc. There's even a way that it will create bootable CD's that will allow you to become a client, and restore your machine from bare-iron.
I understand that there are cheaper solutions out there, including the one that you mentioned, The problem with trying to copy your own files manually to an external drive is that there is no easy and foolproof way to do it. If you try to copy the entire C: drive to the external drive using Windows Explorer, the copying will stop when the system encounters a file it cannot copy (for example, a user.dmp file), plus for those who don't turn on the option to display hidden and system files, often things like Outlook and Outlook Express e-mail folders and files, as well as the Windows Address Book, do not get copied. Plus, it will take a long time to do the copy every time you do a backup, since it will copy each file every time. The Dantz Retrospect software takes care of all of this and also will not copy files that have not changed since the last backup, a very important time saver, and one that makes the Maxtor OneTouch External Hard Drive system relatively painless to use.
These are the good old days you'll be telling your children about. Make them worthwhile.
You should check out DAR. It does exactly what you want. It's free under the GPL.
It's command line based and you will need to read the documentation before using it, but it does what you want.
We don't see the world as it is, we see it as we are.
-- Anais Nin
It still must take a while to back up 3TB to another box.
Do you use iSCSI on Gb Ethernet, or external point-to-point SAS for something like that?
Does the box need high IO through put as well?
Forget thrust, drag, lift and weight. Airplanes fly because of money.
Now that I have agreed with you about backing up being hard, I will tell you what I know. A guy in my lab said that there is a utility called Backup that ships with Windows XP Professional. I haven't used it, but he says that it works well enough for backing up files and even has an incremental feature. So, I would suggest looking into that utility.
Slightly off topic, but in line with some of the above posts, I would mention that there are very easy ways to synchronize the iTunes music folder between multiple machines running OS X (this works for other folders, but I use it for this and the OP mentioned music particularly). This makes a good backup and it syncs both ways. You just have to be careful to run it the right way if you use both machines.
copy music to the current machine:copy music to the other machine:
Otherwise, buy a separate HDD and just periodically run a script that recursively copies all files on one drive to the other.
This is exactly how I do backups at work. I have four active file servers and one server with a big damn hard drive on another floor that updates a copy of everything on all the other servers twice a day. I'm using XXCopy, http://www.xxcopy.com/ and it works pretty well - even generates log files similar to BackupExec. Then on the weekends it runs a PowerArchiver script and dumps everything with a modification date less than 7 days old into a zip file and shoots it across the internet to another computer that extracts the zip file onto its own dupe of all the servers, keeping the zip for incremental purposes.
All this for under $100 in software and two 300GB drives each thrown in their ownn old ass desktop. And it's completely automated - no room for human error.
We all have friends (surely?) - we could be doing the same thing across the internet to eachother's houses. Two guys buy big hard drives to be hosted in eachother's desktop (or extra computer) and a script on each computer that dump changed files to a zip and shoots them across the net.
No sig for you. YOU GET NO SIG!
What, you don't want to shell out several thousand dollars for Netbackup and a tape library??? What's your problem!? (OK, so I'm a little biased... supporting Netbackup is what keeps a roof over my head).
The number 1 mistake people make when doing backups: They write far, far too much data to their tapes. If I had a nickle for every time I saw a user backing up their swap partition and wondering why they where running out of tapes... well, it'd maybe get me a free meal a month. At a fast food joint. From the dollar menu. I digress. Make sure all of that 30Gb is stuff you genuinely can't get anywhere else. Oh, and RAR works great with all those important documents.
Seriously, though. Why not use a tape drive? DDS tape drives sell for next to nothing on Ebay (my DDS-3 6 tape autochanger was less than $20). NTBackup is free and spans quite nicely. DDS4 tapes hold 20-30Gb of data and cost about the same as a high quality audio tape. Incidentally, Microsoft: Please modify NT backup to work with CD/DVD-RWs (or even DVD-RAMs). I wait for the feature with every new version of windows, it sounds like such a simple idea to me but they've never done it.
Small business:
Nero bundle a fairly decent backup product with their Burning ROM software. It's very reasonably priced. It comes free with many burners.
Backup Exec isn't much more expensive and works VERY well. Tapes only, though.
You're really, really cheap? Buy another hard drive and mirror your primary. 30Gb drives costs next to nothing.
I think this is more due to a lack of decent backup software than anything else. Backups in windows always seem to be a pain.
I prefer backup by disk image. This is easier on the mac:
1) Plug in external firewire drive (or USB if you like)
2) User SuperDuper to do a differential backup clone my hard drive to the firewire drive.
Should my HD fail:
- I can boot off the external drive and use it exactly as if it were the internal one.
- I can clone the external drive back to the new laptop drive when I get it
Should the laptop die or be stolen
- I can obtain a new mac and immediately boot off the backup and work from there
- I can clone the image to the new drive when I have time.
"I don't understand why so many home users are against using a good, old fashioned tape backup."
Well, it is very simple really. People are scared of things they know nothing about. Think about it, CD's and DVD's are pretty much ubiquitous and probably the only storage device many are framiliar with outside a hard drive. The obvious choice for most average users.
I have a 2.04TB RAID-5 array (4x750GB drives) attached to a 3ware 9590SE controller. I back this up to a RAID-0 array every so often on my other computer (a bunch of random disks using LVM... I'm not so worried about using raid 0, because it's a backup, and I doubt both boxen will die simultaneously.) I have a crossover cable running between the built-in gigabit ethernet ports on each (Intel Pro/1000), and the backup speed is actually acceptable. (I find that unless you are really willing to lay out for a very good gigabit switch (as opposed to hub), the crossover cable adds a lot for not very much money. I use the second built-in gigabit port on one box, and an add-in 3com card on the other, for normal network activity.)
My backup strategy is basically to NFS mount the other volume and create a giant tar file. Simple, and it works. YMMV
Say it with me now: RAID is not a backup solution.
In my life, I've managed to blow two RAID arrays. The first was in our departmental webserver at work, where a fan ate through a bundle of drive power wires over the weekend, shorting +12 to +5 and really f@#$ing up the entire 9 disk RAID-5 array. Every drive controller board was dead. The better part of that day was when we found the backup group had kept all of our backups on the same DLT tape, because they fit so nicely. Too bad the drive ate the only backup tape when it was put in for restore... Wound up buying an identical drive on eBay, placing it on each disk, and pulling an image. With all that done, I got nine new drives and pushed the images back onto them, and recovered most of it...
The second time was due to a screwy driver upgrade on my desktop machine. Long story short, it mangled large disk transfers. Since I was running software RAID-1 at the time, it mangled both disks in identical ways. I had growing corruption across the array and didn't know it until too late...
That said, I do run RAID-1 at home as a short-term strategy to protect against individual disk failure. That doesn't take the place of my weekly full backup, however. I did cut out the incrementals every night, though. They don't buy me much for my particular style of usage - YMMV.
That's what he gets for giving administrative privileges on his box to someone (his son, in this case) who doesn't need it and thus shouldn't have it.
His son should have had his own restricted account on the box, if even that much (since there was accounting data on the box, I'd argue that nobody else should have had access to that system). Admittedly, once you have local access to the box you can be compromised by someone determined enough, but at least it would take some effort.
Backups are no substitute for proper system administration techniques -- they are a part of them.
Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
Nero does exactly what you described. In the backup wizard, you simply select the files/directories you want backed up. It will then tell you how many CDs/DVDs it will take to store all of it. And you hit, go. it really is just as simple as you describe.
It means it's not a consumer feature on the level of Time Machine.
Vista builds are available for download. You can see how they implemented it yourself.
No, it's easy to use and intuitive because it's a representative interface nobody's ever done before for file backups, especially not Microsoft, who will, as I said, rely on plain calendar and item list controls. Vista's system also does not expose public APIs for application integration the way Time Machine does, letting you recover deleted address book contacts, mail, photos, and more. Vista's only works on the filesystem level.
"Sufferin' succotash."
I use the following script to backup my Window box.
/D/BACKUP
/C/Documents\ and\ Settings/nwk
.tar, tar is universally available, and you have a single file that can be easily burned to disc.
It is a shell script that I run under Cygwin that creates a tar backup of a given directory. It also creates a log file of the backup and an error file that you can inspect for files that were not successfully backed up.
Put the mybackup.sh script somewhere accessible to your PATH variable
Set the BACKUPDIR variable to your backup directory.
I backup to my second hardrive at D:\BACKUP so I set BACKUPDIR to
I backup my directory with the following command from the the Cygwin command line:
mybackup.sh
with tab completions it's something more like:
myb[TAB]/C/D[TAB]nwk
If you want you can set up a cron task to schedule automated backups at regular intervals. You could also modify this script for incremental backups.
This script can also be used on Linux and UNIX systems (just change the BACKUPDIR). What I like about this script is I can see what didn't get backed up , restored backups preserve the file and directory timestamps, the backups all have a unique name based on the directory name + date + timestamp
mybackup.sh:
#!/bin/sh
#2005-08-11
PATH=/bin
#for testing
#SRC="/C/temp"
#first argument is path to backup
SRC=$1
echo $1
BACKUPDIR="/D/BACKUP"
DIRNAME=`dirname "${SRC}"`
BASENAME=`basename "${SRC}"`
TIMESTAMP=`date +%Y%m%d-%H%M`
ARCHIVENAME="$BASENAME-$TIMESTAMP"
TARFILE="${BACKUPDIR}/${ARCHIVENAME}.tar"
LOGFILE="${ARCHIVENAME}_out.txt"
LOGFILE="$BACKUPDIR/$LOGFILE"
ERRORFILE="${ARCHIVENAME}_err.txt"
ERRORFILE="$BACKUPDIR/$ERRORFILE"
echo $TARFILE
echo $LOGFILE
echo $ERRORFILE
#change directory to create relative path tape archive
cd "$DIRNAME"
tar cvf "${TARFILE}" "${BASENAME}" 1>"${LOGFILE}" 2>"${ERRORFILE}"
I have a crossover cable running between the built-in gigabit ethernet ports on each (Intel Pro/1000)
With modern auto-sensing NICs you don't need a crossover cable; straight-through is just fine. They'll work out how to use the send and receive lines.
90 percent of these posts are jackasses talking about what they can do in Linux, MacOS, or saying that he should buy other hardware. That's what it seems like. Thanks Slashdot for the -5, I'm-a-fracking-embicile-who-doesn't-understand-the -question answers.
If the OP sees this, this is exactly what you need:
http://www.backup4all.com/
They have a fully functional 30 day trial, it's easy as hell to use, and if you decide to buy it, it isn't too bad price wise. It should meet all your requirements. To all you jackasses bringing up Linux(I'm a Linux user myself), MacOS(fucking weenies), or telling him to invest in new hardware, I'd like to buy you all a round of STFU. Morons.
I haven't had a tape or other backup device in what seems like an eternity.. I've lost 2 drives total over the past 5 years, and both incidents went off without a hitch.
my i recommed buying a copy of WinRar and telling it to do a store (no compression) and to ignor locked files and put a recovery reocrd and to split the archive to the size of the dvd's and then just burn them your self.. sure it requires you to do stuff but it solves all the problems you stated (WinRar is cheap)
if you want a one stop shop you are going to have to deal with dat,bak, archives that are propritory, and you will end up paying for it too
'...if only "Jumping to a Conclusion" was an event in the Olympics.'
Wow, did you even read my post? I'm backing up onto a RAID-0 array because I'm fairly confident that the source of the data (my RAID-5 hardware array) is not going anywhere. Yes, a huge RAID-0 array is highly fault-prone, but if my backup server dies, so what, I've still got the original (except in the rare case of both computers being killed simultaneously, which is an acceptable risk to me. I can't afford 2.5TB of offsite storage.) The RAID-0 allows me to throw together my miscellaneous drives and create a volume large enough to hold my backups.
If the raid-0 array fails, I haven't really lost anything, because I still have all the data on my RAID-5 array (the original data source).
...and the DVDs have a far shorter shelf life than hard drives...
Where did this myth that CD-R's and DVD (+/-)R's have a short shelf life come from? I can assert that it IS a myth, since I have a stack of about 30 CD-R's that I burned in 1999 that are still perfectly readable. Since then I've been burning about 100 CDs a year and last fall I backed up the whole collection onto hard drives. Every single one was readable.
Are you people storing your discs in sandpaper-envelopes or using them as frisbees in the meantime? What's the failure mode of these discs with the "short shelf life"?
Have you ever wondered How to Take Over