Slashdot Mirror
Ask Slashdot: It's World Backup Day; How Do You Back Up?
MrSeb writes "Today is World Backup Day! The premise is that you back up your computers on March 31, so that you're not an April Fool if your hard drive crashes tomorrow. How do Slashdot users back up? RAID? Multiple RAIDs? If you're in LA, on a fault line, do you keep a redundant copy of your data in another geographic region?"
Simple. Redundancy backup.
It's a raid.
Apple hate aside, time machine is an amazingly excellent backup system.
It backs up to a Netgear Readynas configured in RAID 5. Hourly, daily, weekly backups. I've never lost anything thanks to this great system.
In linux I try to approximate this with BackupPC.
http://backuppc.sourceforge.net/
It is really an excellent piece of software, though no where near as refined of course. You pretty much only get daily backups though since the kernel in linux does not track filesystem changes so hourly backups would be very prohibitive.
It's easier to fight for one's principles than to live up to them.
With a loud beeping noise.
William of Ockham had no beard. The most likely explanation is that it was chewed off by squirrels every morning.
Can we moderate this article flamebait?
Change is certain; progress is not obligatory.
My weekly backups: something like:
0 0 * * 0 /home/me/backup.sh
#### backup.sh ##### /dev/null
cp -r home/me/*
I haven't missed a backup yet :-)
Amateur. I take polaroids of my platters and store them in a safe deposit box.
Live on the edge guys...
/does/ boot or you /don't/ get toasted by bolts of electricity then the sense of relief is wonderful !
;-)
When you boot up in the morning and it takes a little longer than usual, the heart beats a little faster and you think "OMG is the machine going to fail? My data will be gone". Or perhaps there's an electrical storm to liven your day up - "If that thunder gets any closer I might have to shut down the PC, but if lightning hits then everything's toast !".
These scenarios, and many others, all get the blood pumping in fear. If the computer
Try it - it's fun
while (true != false) process_more_stupid_code();
I use a secure distributed grid. The software is an open source tool, Tahoe LAFS (http://tahoe-lafs.org). The grid is composed of ~15 servers contributed by different people all over the world. There are a half dozen servers in various locations in the US, about the same number in Europe, and the remainder in Russia and the Ukraine.
My files are AES256-encrypted on my machine, split into 13 pieces using Solomon-Reed coding, any five of which are sufficient to reconstruct my files, and then those 13 pieces are distributed to the servers in the grid. I run daily backups, but since uploads to the grid are idempotent, only the changed or new files are stored. I also run a bi-weekly "repair" operation which checks all of my files (all versions, from all backup runs) to see if any of their pieces are lost. If so, it reconstructs the missing pieces and deploys them to servers in the grid. The individual servers in the grid are fairly reliable, but problems do happen, so repair is important.
I get about 100 KBps net upload rate, so this isn't a good solution for backing up terabytes, and the occasional "surge" in my data generation (usually caused by a day of heavy photo-taking) often causes my "daily" backup to take a few days to run, but all in all it works very well.
Should my server ever die, I only need two pieces of information to get all of my data back: The grid "introducer" URL, which will allow me to set up a new node connected to the grid, and my root "dircap", which is a ~100-byte string containing the identifier and decryption key for the root directory of my archive. That directory contains the decryption keys for the files and directories it references.
Since this grid is all volunteer-based, the only cost to me for this backup solution is the hardware and bandwidth I provide to my grid (I provide 1 TB of disk and grid usage consumes a fairly small fraction of my Comcast connection), plus the time I spend administering my server and checking to see that my backup and repair processes are running. Oh, and I also contribute (a little) to the Tahoe LAFS project, but that's due to interest, not a requirement.
I'm very, very happy with this solution.
BTW, the grid could use another 20 nodes or so, if anyone is interested. There's a fair amount of trust required of new members to the grid, though, so it might take us a while to vet new members. The trust is required not because other members of the grid might have access to files that are not their own, but we need to verify that new members will behave appropriately -- providing their fair share of storage and bandwidth, and not consuming too much.
Anyone interested should check out the grid's policies and philosophy at: http://bigpig.org/twiki/bin/view/Main/WebHome. If all of that looks good, join the mailing list, introduce yourself and we'll consider allowing you to join the grid.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
All of our important files (even the kids' files) are on the server. It backs itself up automatically 3 times per week to external USB drives. I rotate the USB backup drives every few weeks. So we need do nothing special today, as the backup works fine.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
World backup day? How about world test your restore day? All the backups in the world don't mean anything unless you test your restores and know your data.
(1st sig) If this were a snappy sig, you'd be reading it right now. (2nd sig) I'm a karma whore. >Insert FUD here
I just noticed I needed quotes around the bpaths variable assignment. Furthermore, my backup script has been broken since January!
Thanks, Slashdot, for making my look at my script!
(T>t && O(n)--) == sqrt(666)
rsnapshot seems to work pretty well for incremental rsync'd backups for me. It uses symlinks to maintain the older snapshots, to save on total filesystem usage. It can do rsync over ssh for backing up remote servers/pushing local vital data to a safe remote location.
Local backup server uses Linux software RAID for good measure (5x1TB RAID 5 + 10x2TB RAID 6).
Backup is only half the problem. Restore is the other half. And indeed that's where I've usually had the most problems. The third problem is validating the restore. You always worry that you are either going to overwrite something on the restore target or miss something on the restore source and end up in an inconsistent state.
Time machine is revolutionary because it is so simple and seems to be almost flawless. I've had lots of backup systems over the years including dump 0 but everyone has been plagued with issues that arose when things were off normal. I've cobbled all sorts of things like rsync and cpio but the only thing that comes close to working as flawlessly as time machine is a NetApp.
At work where I can control the remote servers securley on a closed network I am able to use time machine for a remote backup. But at home I don't have a remote server I can target for the remote backup.
TO do a remote bakcup at home I use Crashplan. I looked a lot of competitors like Mosy but settled on crashplan for two killer reasons. The giant problem with all these commercial backups is that while the incremental backups are simple over the net, the restore of a whole hard disk cannot be done over the net. You have to pay them to burn DVDs and send them to you. ANd that assumes you know what time period you want to recover.
UNlike all the other methods crashplan lets you pick a buddy who runs crash plan and then you can back up your disks to each others computer. If you need to to a massive restore you just drive over to your buddy's house and pick up the drive, bring it home, and restore locally. This also solves the problem of the first dump being too large to send over the net as well. You do it locally then drop the drive off to your buddy.
Brilliant!! plus with crash plan you pay for the app once not monthly.
I've used it for years now and it works very well and it very easy to set up. All your files are encrypted so buddies can't read each other's drives.
The only flaw with crashplan is that it runs in java so you have this instance of java running 24/7 and not to put to fine a point on it: java sucks. I don't know if it is crashplan or other things that run in the JAVA VM but over the week it bloats up to 600MB to 800MB. THe workaround solution is to kill the java VM every few days. Empirically crashplan is robust enough to survive this and restart. But that's a really awful solution.
Some drink at the fountain of knowledge. Others just gargle.
Indeed it is not as efficient as it could be. However, using it is only slightly more complicated than "buy a usb hard drive and plug into computer"
An efficient, totally ideal process that no one actually bothers to use because it's either too complicated, or because it isn't actually licensed for your platform or whatever, is no backup system at all.
Also, ZFS is a filesystem that can be set up to preserve version information. It's not a backup while it's on the same disk....
Can you be Even More Awesome?!
Just a suggestion; you shouldn't delete any backups prior to writing (and possibly verifying) your new backup. Imagine what would happen if your disks failed during your backup. It's more likely than you think; it's a period of intense I/O. I've personally had it happen during raid reconstruction.
You might consider timestamping your backups, and deleting all but the most recent 3 after a successful backup.
Something like:
.. should clean up them nicely.
A government is a body of people notably ungoverned - AC
I use both. Time Machine back up to a deduplicated RAID-Z volume. When Time Machine backs up a file (e.g. a VM disk image or an 8MB stripe from a sparse image) with only a few small changes, the decuplication kicks in and means it only takes up a couple of blocks.
I am TheRaven on Soylent News
That's nothing. You should see my butterfly collection...