Slashdot Mirror
It's 2006 and Backups For Home User Still Tricky?
CranberryKing asks: "What is it about backups that always seems so difficult? I am trying to do a simple backup on my home XP system/s (about 30GB of files) that will write to my DVD burner. I don't want compression (most of it is MP3s, which don't compress well). I want a routine to simply write my selection to the DVD writer and spread it across however many discs are required (rather than me manually approximating and copying to each disc). I want the files on the disc readable from any system, so no proprietary backup wrapper or DAT files, please. My last attempt was using a free program that looked good called Simply Safe Backup, but it created two coasters before crashing with an unknown error. If I can just get a full backup to work smoothly, then I'll worry about scheduling, incremental, and encryption. This seems like a very common scenario for home & small offices. Is there an elegant, reliable & cheap (free) solution to this?"
Backups for the home or small business user do not need to be tricky, difficult, inconvenient or time consuming. But you do need to have the right equipment and software for the job.
I would say that the method that you chose, which is using a DVD-Writer drive, is not the best solution to your problem. I have found a product that does work well, and that is the Maxtor OneTouch External Hard Drive solution. I have one of the newer models, the Maxtor OneTouch II and with the bundled Dantz Restrospect software, it works great. You can schedule the hard drive backup at a certain time or (and this is where the OneTouch gets its name) you can hook up the external hard drive anytime and press the button on the front, and the software will take care of the rest of the backup procedure. It is quite easy and even users who have in the past been put off by other backup solutions (like backup tapes and recordable CDs) have embraced it. You can add other features like incremental backups easily as well through the software as well, and it stores the files in the Maxtor OneTouch drive in a regular file system, so it can be accessed even on machines without the Dantz Retrospect software loaded.
The issue I have found is that for most home or small business users, if the backup procedure is tedious or cumbersome, the user will not do the backups and data loss will occur. After using this device and recommending it to others, I have found it has gone a long way to solve this problem... it's truly a twenty-first century method of system backup.
The last Maxtor OneTouch II I bought was under $200 Canadian and had a 100GB capacity and includes all the software and cables that you need to get connected and working right away.
P.S. I do not work for Maxtor or Dantz, but I am a happy customer and I have sold this device to others in the past.
These are the good old days you'll be telling your children about. Make them worthwhile.
What you seem to want is a full disk copy, not necessarily a backup.
You don't want compression. You don't want everything packed together. You want all the files and directory structure to be preserved as-is.
That's a copy, not a backup.
Try Ghost or something from Partition Magic, if you've got the money. Otherwise, buy a separate HDD and just periodically run a script that recursively copies all files on one drive to the other.
"I want a routine to simply write my selection to the DVD writer and spread it across however many discs are required"
A 250 gig hard disk is under $100.00. How long are you going to take to back up 250 gigs to dvds (It takes time. I did it once - never again).
I use it and it's prevented some real heartaches caused by deleted/corrupt files.
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
But to the point, Backup lets you create plans based on what to back up, where to back it up to and how often. Then it pops up a window when automatic backups are going to start telling you that one is going to begin and do you want to cancel. I think it's great and 9/10 of the time I never have to think about it.
Not since Marie-Antoinette played milkmaid has looking simple and honest been so fake and complicated.
http://www.apple.com/macosx/leopard/timemachine.ht ml
not for Windows, but arguably (will soon be) the greatest step forward for "home user" backups.
Mac OS X comes with Disk Utility. Using that, and Automator, you can set up a script to image your drive to a bootable drive image every night.
Problem solved.
The real litigious bastards...
I don't understand why so many home users are against using a good, old fashioned tape backup. Look, you can get a DDS-4 tape drive from eBay for less than $100. In fact, I'm about to sell my Sun external DDS-4 drive there soon. You can then get a compatible SCSI card for about $20 if not less. Then you just have to get the tapes. A new box of ten DDS-4 tapes -- equivalent to about 480GB compressed -- can be found for around $50 on eBay.
... whoops), I use a six-month rotation.
Because Windows Backup recognizes most tape drives, you can always use that to do you full and incremental backups. It's certainly not going to be anywhere close to something like Veritas NetBackup, but it still allows media management, is compatible from system to system (as long as it's the same version of Windows or newer), and you don't really need to do anything. Mark what you want to backup, make sure the tape is in the drive and ready to go, then back the stuff up. If you have a completelsystem crash, Backup can read the contents of the tape and rebuild the index.
I know, I know, the Slashdot crowd doesn't seem to like tapes. Whatever. They work fine for me. I use a three month rotation with a full backup at the beginning of every month and incrementals every Sunday. For the infrequently-changed directories (almost called them file systems
And don't complain about the slow speed of tape drives because that's what overnight backups are for. Let the system back up your files while you're asleep. Besides, DDS-4 goes at about 15-20GB/hour. Even if you just need to go out and run some errands, you can set it to backup as you're about to walk out the door.
The Overrated mod is for reversing inappropriate, positive mods, not for voicing disagreement with a post.
Chances are even a generic OEM DVD Writer comes with write software that is able to do the simple backups you are requesting. (Although I am with many users, just add a new hard drive for backups, even a USB external is going to be about 50-100 bucks and will be virtually instant in comparison to DVD and just as reliable if not more.)
WindowsXP pre-dates DVD Writing as the norm of the time (2001), so it doesn't inherently support it (which draws out the OSX and Linux crowds of telling you to get a real OS and then they list 20 command line tools that are fairly cumbersome.)
Since it appears you are using Windows, when you can, move to Vista, Backups are easy, able to use DVDs, and can do full system bit by bit as well as file/folder backups, all with a couple of clicks.
The _easiest_ way I can think of to do this without either spending a lot of money or switching operating systems, is probably just get an archiver, possibly http://www.rarlab.com/ winrar, and on the create archive screen, select to only store the data (no compression, runs faster and mp3's are already compressed anyway, why compress them again?), and tell it to split the archives into whatever size you want (depending on what kind of dvd's you use. single or double layer, etc, just input the number of bytes and it'll auto-split them into appropriate segments). Then just burn the archives to disc one at a time, which is a bit time consuming but by no means difficult to do.
This isn't the prettiest solution in the world, and it can't save a lot of things (like all your settings and such) but it works for personal files quite well. And you just need the appropriate unarchive program installed after you reinstall, so either winrar for windows, or unrar for *nix, should you ever decide to cross over. That is, assuming you choose the rar format. IMHO, as far as the windows-friendly archives go, it seems to work the best out of the common formats for this sort of backup work.
I don't use Windows, but for my friends (the ones who can actually be made to care about backups), I recommend this:
http://www.xs4all.nl/~edienske/abakt/
Support both 'traditional' (compress/split) type backups and a file copy method (good for a USB hard drive, for less savvy users who want to be able to just plug the thing in and retrieve the file they just borked).
Open source. Feel the love.
Not the easiest thing to setup, so I set it up for them, save the profile, and tell them how to do a backup (plug in drive, start program, press go).
Command attempted to use minibuffer while in minibuffer
Nope. No way.
Crumb's Corollary: Never bring a knife to a bun fight.
SizeMe is a very simple, free-as-in-beer GUI program for Windows. You drag'n'drop a mess of files into the window, and it rearranges them (but doesn't modify them) so that you can burn them to the minimum number of discs possible. It even lets you drag the images into Nero et al to burn them. Worth a look.
RAID is really mostly about availablity, not backup. And, I'd say recommending RAID as a replacement for backups is selling a false sense of security.
just plain rsync.
local to local or local to remote.
works well, its free and I believe its multiplatform.
copy disk to disk. tape is useless now - its too error prone compared to disks. disks are the new 'swappable carts'.
--
"It is now safe to switch off your computer."
I usually use synctoy powertoy for windows xp to do the backing up for me. It can run in a variety of modes and is usually good enough for most backups. http://www.microsoft.com/windowsxp/using/digitalph otography/prophoto/synctoy.mspx
Get a second HDD. Internal or external. Add a "Scheduled Task" that will run "backup.bat" periodically. backup.bat needs one line for an xcopy:
/d /e /h /o
xcopy C:\ D:\
The first run will take a while, since it's copying everything. Subsequent runs will only copy what's been modified since the last backup. It really doesn't get much easier than that, if you ask me.
Who doesn't like free music?
There's a little utility called XCOPY that comes with Windowss that copies files without stopping, skipping inaccessible ones and continuing. XCOPY /C /D /E /F /H usually does the trick for me.
If you don't know where you are going, you will wind up somewhere else.
Acronis True Image is the only program so far that doesn't suck at backups. I use it constantly to backup to a USB harddrive I bought at discount. Perfect. Takes just minutes thanks to incremental backups.
I have a Windows box at home I use for some work-related stuff. I have about 200Gb of changing data that I need backed up regularly. After messin' around with some backup software and hardware, I settled on an external 300Gb USB 2 drive and this simple command saved as a *.bat file and executed from Task Scheduler every night:
/CEDY "C:\" "U:\"
xcopy
Works just fine at no additional software cost.
Before that I tried an old version of Backup Exec and the latest version of Acronis TrueImage and a few other programs. They all have some drawbacks: too complicated, not automated enough, only work with removable media, too CPU or memory intensive, or some other nonsense.
When it comes to backups, simpler is better. HD to HD backups is the way to go. Compressing data is a bad idea. Creating multi-volume backups that span removable media is an even worse idea. I am talking about backups for your home PC, of course. I doubt too many of us can afford a robotic tape library and a NetBackup server.
first, create softlinks of all the files and put them into a directory called backup or something.
/tmp/somerandomdir /tmp/somerandomdir /backup > files.tar
then, just use a simple script, something like
mkdir
cd
tar -c
split files.tar -b DVDSIZE
opendir(DIR, ".") || die "can't opendir $directory: $!";
while ($current_file = readdir(DIR))
{
#print "file is $current_file";
mkdir $current_file+"dir"
mv $current_file $current_file+"dir/"+$current_file
mkisofs -o $current_file+".iso" $current_file+"dir"
(can't remember how to burn isos on the commandline)
}
of course, use a real language for the script, pretty it up etc, but it shouldn't be too hard.
Phil
Look into XCOPY's bigger brother Robocopy. (The Robust File Copier)
It's available in the windows resource kits, which you can download the tools for Windows Server 2003 direct from MS. Just extract robocopy.exe (and robocopy.txt or doc) from it.
If you're not processing transactions 24/7 this is pretty simple. I took an old machine, threw in a big ATA drive, and installed Ubuntu Linux and Backuppc, which Ubuntu has packaged. It automatically backs up every machine on my network, both Windows (via SMB) and Linux (via rsync). It has a Web browser interface with the manual permanently on line in the browser. While it doesn't do true "snapshots" it does give you a series of backup points going back in time, It shares redundant files to avoid needless duplication. My backup drive is less than half full. without compression. If space gets tight I can keep adding cheap drives to the backup box (and put them under LVM if I want to see them as a single large storage space). Any one of my drives (including the backup) could of course fail at any time, However, as long as I don't have multiple simultaneous failures, I should be pretty much covered. Barring a fire, I'm pretty safe. I check it every couple of weeks to make sure it is still alive (it always is). You could back up the backup drive to a portable drive every so often and keep those off site. if you were really paranoid,
I have had the occasion to restore from the backup when I did something stupid to a production directory. I found the most recent valid version of the directory among the backups, and just scp'd the entire directory to my production machine.
I've had much more trouble with unreadable tapes than with unreadable drives. I don't know where you are finding sub-$100 tape drives with any capacity. The ones I see are more like $2,000.
After years of frustration with a dozen backup systems from Unison, rsync, Windows backup (yikes), Retrospect, etc., and I finally found an elegant, simple, and secure method.
My requirements were:
1. I want to keep the files in their original format -- no proprietary compression/monolithic files
2. It needs to be automated as possible
3. I need to have an offsite rotation
4. It needs to be encrypted
I chose SyncBack as the backup software... freeware is available, but $25 gets you the latest major rev. It supports a ton of features including backup to an FTP site. I also picked up PGP Desktop Pro which includes Whole Disk encryption. That way I can just encrypt my entire 160GB external drive.
The process:
1. Plug in external USB drive (which has been encrypted via PGP Whole Disk encryption)
2. Type in passphrase to unlock drive for the Windows session
3. SyncBack runs 3 scheduled jobs to backup to the USB drive
4. At the end of the backup job, SyncBack automatically pops up an HTML report of what was copied and any errors
5. Once I verify everything looks good, I unplug the USB drive take it to work and give it to a co-worker. If the co-worker tries to plug in the drive to read the data, it just looks like an unformated partition unless he has PGP installed. In that case, it asks for the passphrase (which he doesn't have of course)
6. I take the 2nd USB drive at work home and go to Step 1
Use a hard drive, using burnt discs is just silly,you need to verify the burn, they get scratched, crack ect...
Microsoft has a free backup utility called Synctoy v1.2 It works better than any other sync/backup software I have ever used for home use. Bust out the wallet and get an external harddrive (usually comes with backup software) or build one yourself even cheaper. Using Disks for backup is just a bad idea.
Backup was the first thing that came to my mind when reading the posters requirements, because it backs up whatever you like across as many DVD's (or CD's) as needed, all as plain files you can pull off by hand later if for some reason you do not have Backup.
I'm surprised there are not more solutions that provide this very simply ability that really is a lifesafer when you just want to recover a little data.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Many people solved that problem by downloading the freeware version of XXCOPY which actually works right. At least it always has for me and I've never seen any complaints from any others.
I'm not sure that you still need to worry about that. But I'm not sure that you don't.
You can't see ANYTHING from a car, You've got to get out of the goddamned contraption and walk...Edward Abbey
But as I found out the other day it won't backup locked files in Windows such as System Volume Information. So doing a full backup of your partition without these is pretty useless.
Yeah, Nero Backup is great for doing a backup of a none Windows drive with no locked files, just not your Boot drive.
And when somebody steals the computer, how do you restore the data?
Well, let's start with a few things like causes:
1. User error, in particular:
1a. I-don't-know-what-I'm-doing error, aka luser error
1b. I-know-what-I'm-doing error, aka admin/poweruser error
2. Software issues
2a. Corrupted files
2b. Viruses
3. Hardware breakdown
3a. Disk failure
3b. Short circuit, controller failure, leaky water cooling taking out multiple disks
4. Crisis
4a. Your house burning down
4b. Break-in
Then there's the importance of data, at least three:
1. Personal/Important things
2. A-lot-of-work things (like a ripped CD collection, recreatable but much work)
3. Bulk data
Back-up methods:
1. In-machine backup (RAID)
2. Near-line backup (DVD/external disk)
3. Offsite backup (DVD/external disk)
4. Network backup
5. Internet backup
The thing is, you don't manage to serve every need at once. Many here talk about external disks. I remember a slashdot post from a previous discussion, where the burglar had kindly taken the PC as well as the external disk lying nearby. Or if the house burns, it all burns. Yes, it sucks bigtime in any case, but at least now your digicam photos can survive. One of the hardest things about it, from what I've understood is that your past is pretty much erased. Clothes, furniture, souvenirs and trinkets.
Another issue is the time between you discover the problem and the error occurred. Suddenly notice you must have deleted that important folder by accident, or it's been eaten by filesystem corruption, or bad sectors (yes, they get remapped, no they don't always manage to rescue the data). Or you want to return your system to a virus-free state. Good luck doing that with your daily sync'ing backup to an external HDD.
Part of it is also the effort just actually doing it, even if it's just "One push" if you're going to hide it/put it in a fire safe/take it offsite. I would prefer having an automated network backup run, but my network stretches like 5 meters and my Internet connection is too slow. Some of the really important stuff(tm) could go over the Internet, but not all my bulk data. Plus, these should have more versions too. Overall, I find making a good backup solution is far from trivial.
Live today, because you never know what tomorrow brings
For windows users it can be a pain to backup on to a single file (or fileset). Here are a few approaches that worked for me on Windows: Norton Ghost Network or Enterprise Edition, Restrospect Workstation(this is my best recommendation for user friendly methods), Acronis Trueimage Workstation(with Universal Restore component), Windows 2000 Backup Utility included in Windows 2000(a little awkward but works well), xcopy, Nero. Acronis TruImage Workstation(with Universal Restore Component) is one of my top choices for "users" because it has a easy to use wizard wrapper around the MS syspart tool in Windows to create a "bare metal" backup which can be transfered to a new computer easily as well as make file backups to a number of medias while the OS is running. Retrospect also works well but I didn't see any tools for imaging an OS, bare-metal or otherwise.
Now to be clear I do not consider a second drive or RAID a backup solution, only protection against defective/worn hardware. The reason is that if an electrical problem, virus or accidental file destruction occurs the file would not be protected. A true backup needs to be entirely separate from the system being backed up. DVDs fit this criteria (slow to write, fast to retrieve) but are not ideal because contrary to popular belief, their shelf life is 2-5 years (according to an article by IBM) max depending on quality of DVD in use. Convenient, but not ideal. A Hard drive in my experience lasts between 2-10 years. USB enclosures make this very easy and notebook drives/usb enclosures make this very portable. Tapes have a shelf life of up to 10 years, although its slow.(I've heard some say 30 but what home user has a controlled environment). USB Sticks (Flsh memory) are fast to write and retrieve but I believe they degrade per write more rapidly than Tapes or Hard drives. But all of these would qualify as a true backup because they can be contained separately. For home users I suggest either a USB Enclosure for a hard drive(replace every 3-4 years recommended) or Tape (DAT 72 is the best for the money these days and backwards compatible with DDS-4). There are SATA DAT Tape drives available so SCSI controllers are less of a concern now. I don't recommend DVD's as 2 years later you may be in for a nasty surprise on an attempt to retrieve.
The other thing for all users (business and home) to keep in mind: They should have TWO kinds of backups: File backups (incremental) and OS image backups. The reason for the OS image backup (or drive image) is that even if your essential files are saved, your application installation files/dependencies may not. Many files cannot be backed up properly because an active process has ownership of them while the OS is running. So for a TRUE OS backup, a separate boot CD or USB disk should be creaetd (Norton Ghost, Acronis TrueImage (really friendly for this) or "Recovery Is Possible" (RIP, a free open source linux based recovery tool you burn to a separate CD) all have tools to do this. A image backup (again done with a separate boot disk) should be done once all applications you require are installed, or you add a new application. This way should data on your hard drive be destroyed, you can easily restore your OS and all its application installs, logs and user dat files in one step. Otherwise it can take HOURS to do a new setup (OS install, drives, applications, tweaking settings...)
Hope this helps everyone.
PS: I'll check out Bacula, its not for general home users, but it looks good for techies.
PPS: RIP (Recovery is possible) has been great for rescuing data from dying hard drives when windows couldn't read the data. Just in case a users didn't do their backups in time and wants to avoid paying hundreds of dollars to recover the data. Just do a new image of the bad drive using the RIP tools, and read the files off the new drive.
'Imagination is more important than knowledge' - Einstien
Here's the point I think the other poster meant:
RAID0 is striping. So every other chunk of your data is on each disk - so every file is on every disk. So you are essentially guaranteed that any time 1 disk goes bad your entire array is useless - COMPLETELY useless. Lose 1/4 of the drives, lose 100% of the data.
Unless you have an extreme need for contiguous, single file read/write speed, RAID0 is a poor choice. (For many asychronous reads a bunch of drives with your data randomly split between them is much more efficient in the average case, because seeks are much more costly than most reads. Many RAID1 implementations will choose to read from whichever drive is less utilized, so for pure asynch read-speed RAID1 is often best.)
HOWEVER, if all you have is concatenated drives - where the first part of your virtual drive is on the first physical drive, and the second part on the second one - then you skip the giant reliability disadvantage of RAID0. Essentially you then have a scenario where a single drive failure will most likely only take out its fraction of your data. (After some fun with fsck and some luck on the fs level) Lose 1/4 of your drives, lose 1/4 of your data. I can see being comfortable with this scenario.
Now, if you concat'd your drives using LVM, that's what you've got. Based on your original post, I think you did this. That is not RAID0.
So I think you're ok with having only one copy of the data, you probably have LVM concat, and that's fine.
The OP was trying to point out that for any data you even slightly care about RAID0 is a poor choice compared to meerly concating it.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot