Slashdot Mirror
Affordable Home Backups for 10-100G Systems?
MichaelJames asks: "Ok, I have my MP3's streaming, all our digital pictures up, and a file server running on one machine in the basement. What would be the best way to do simple backups of the system and data? Get a tape drive Get a CDRW or DVDRW to backup the MP3 and pics, but use the old Zip drive for the file server data?" With drives in the 10-20 gig range only getting smaller and less expensive, what are we to do for backups, that have yet to scale well in the same range. For home systems with up to 100G of storage, what do you use to back up that much data, with a solution that's affordable to the average computer user? Have DVD writers become cheap enough for serious consideration as a backup media?
Given that a 100G hard drive is cheaper than any removable media solution, why not just buy another hard drive and install it in a removable (not hot-swappable, just removable) rack?
Racks are $20 at my local Fry's, and inserts for other hard drives are $10.
I don't really think CDs and DVDs really aren't big enough for regular backup of large sets of files. It's just too inconvenient to have to setup a bunch of different 5GB backups, one per DVD (or swap DVDs). The only convenient solutions are to do what the first poster said: use a second harddrive, they're relatively cheap. Or buy a tape drive to store the backups.
Personally, I backup to a second harddrive.
I have to say that this is coming from someone with a total of around 280gig at the house, but...
Out of 100gig, how much do you really NEED to back up?
The vast majority of my space is taken up by MP3s (where I converted my CD collection), but that could easily be replaced. To tell you the truth, of the things that I would need (documents, pictures, etc), I could easily fit it all onto a CDR. Well, maybe two. (I take lots of pictures)
Basically it boils down to, do you really need to shell out the money for that extra drive?
:^)
'Life is like a spoonful of Drain-O, it feels good on the way down but leaves you feeling hollow inside'
I once worked at a place where we had a lightning storm. Within a week, about half of the hard drives had failed, out of about a dozen. RAID won't save you then. And how fast can you get replacement hard drives installed, anyway?
All the affected machines were plugged into good UPSes, too.
Moral of the story: Always use offline backups.
Why are you bothering to back up your data?
/` or a virus)? If so, then a RAID solution is useless.
That may seem like a stupid question, but you need to consider the reasons you want to have a backup before you settle on a method.
Are you afraid of your drive failing? If so, then using a RAID solution should cover you.
Are you afraid of losing your whole system (perhaps due to lightning or theft)? If so, then your backup must be kept physically isolated from your system.
Are you afraid of accidentally deleting files (such as `rm -rf
Are you afraid of having your system down for an hour or two while you replace a drive? If so, then regardless of other issues, you need a RAID setup.
Do you want to use your MP3s with some other device? If so, you probably want CD-R copies.
Of course, there are other considerations that I haven't mentioned or thought of.
Raid doesn't address one of the other common modes of data loss: catastrophic failure/natural disaster. If your raid gets set on fire, shocked with 1MV lightning, or doused in water, it will probably be completely gone. Offline backups (such as removable, or offsite backups) are much more reliable (it isn't likely your house is going to burn down at the same time your bank is subsumed by a tidal wave).
Another thing that some of us are looking towards is finding a trustworthy friend to share capacity with. If you each buy the extra hard drive (or have space to spare), and rsync nightly with each other, you can get reasonable coverage, and offsite backup. Just pick one reasonably geographically far from you so your data doesn't get sucked up in the same tornado.
Or if you don't trust the "friend", use an encrypted filesystem, or crypt the files first.
What do you do when the building burns down?
Not to distro-bait, but Debian in particular shines here because apt makes it so damn easy to bring a system back to the state you wanted. For myself I have created a meta-package (.deb) which does nothing but depend on the applications I want installed on every desktop system: galeon, gnucash, xchat, gaim, xmms, vim-gtk, and a handful of others. Then I back up my meta-package, all of 10k including a few shell scripts I wrote for myself. Install my meta-package on a new system, and voilá, apt fetches and installs every app, that I need to continue working, dependencies included.
It's rare that you're presented with a knob whose only two positions are Make History and Flee Your Glorious Destiny.
Most of you are missing this problem (fire/theft). What are your various solutions to account for and protect against this?
h ol d=1&commentsort=0&mode=thread&cid=2689576
I use a removable drive that can be taken to another building or put in a fire safe. Any other options out there? I'm sure we're creative enough to have some decent options.
More info on mine (don't want to re-type)...go to
http://slashdot.org/comments.pl?sid=24768&thres
(sorry, you have to cut and paste cause I'm lazy)
What about the possible security issues raised by this scheme, do you have anything protecting the integrity of other people's data from the prying eyes of the host computer?
RAID is to provide either additional speed and/or hotswappable capability. RAID really stinks as a backup, since RAID doesn't care when some program deletes most of the hard drive, when some user removes too many files, or when the OS barfs. Sure, RAID will save your DATA if one HDD fails, as long as whatever caused it to fail didn't affect the other drive, but for the reasons already listed, this doesn't mean RAID is a valid method of backup.
However, a HDD in an external enclosure could be considered a valid backup, however, for true redundancy, you better have two drives you swap, and you better be doing surface tests regularly. A drive, properly treated, should last many, many years. Also, you could combine a drive with monthly or quad-yearly backups to CD-R, just make sure you do your research on the inks used in CD-R disks, some don't last as long as others.
Just my $.02
What happens when your intern scratches your delicate DVD?
I forget...are we at war with Eurasia or East Asia?
That worked really well for backing up our 80MB drives onto stacks of 1.44MB floppies, since you would really only need to insert about 5-10 floppies during your weekly backup, just to get the files that changed.
So why not just do incremental backup onto CD-Rs? Even with 100GB of archives, most of those are static. You probably won't need to use more than one CDR per week (maybe two) to track the changes. It's cheap, relatively painless if you've got the right software (and it wouldn't be hard to throw together incremental backup/recovery scripts in Perl if you're into that sort of thing.) and you've probably already got a CD burner.
If less than 650MB of files change in a week, the rest of the CDR can be filled up with files that were on earlier CDRs (this way your backup set can remain finite and you can throw out the earlier CDRs as they become obsolete. Or if you keep them all, you can reconstruct that state of your hard drive at *any* time, not just at the last backup.) This seems ideal to me--why is everyone else talking about expensive solutions like tape drives, DVD-RWs, and second hard drives?
I have a positive modifier on Troll. When I mod someone Troll their karma should go UP!
I have to wonder whether (first of all) why in the heck anyone would need to have 100GB of disk space on a home system. But then I have five systems networked together and have more storage than I would have thought sane a few years a go though I have a bit of a ways to go before I will run into the poster's backup problem. It wasn't too long ago that, if you could afford 100GB, you could probably afford a SCSI array controller that would let you do a lot of RAID, hot swapping, automatic drive replacement, etc. With today's cheap disk prices you don't have to be wealthy to have an ocean of disk space. (I can remember the days when we thought having 900MB on a MicroVAX II was extravagant.)
You could always do it the traditional way and get some tape drives. Unfortunately, they're much more expensive than you might think when you have to backup that much disk space. You certainly wouldn't want to go cheap and be feeding 90m DAT cartridges into a drive all night (it'll start feeling like you're backing up to floppies before long). A good high capacity tape drive can get, what, 20GB onto a single cartridge? Not bad. And I think that at this point in time, tape is more cost effective than DVD-R. (Something tells me that the MPAA, and maybe the RIAA, will try to keep it that way too.)
Mirroring disks can be helpful. Hard disks are getting cheaper and cheaper. Heck it's almost scary mow much disk space you get in a typical PC sold at Best Buy nowadays (and without a backup device; it's almost criminal). If you're running mirrored disks you'll forestall the inevitable disk crash that takes all your data with it. Question for the Linux folks using the `md' driver: Does it allow adding a third member to a mirrorset? And, if so, can it be done while the system is `live'? (The third member gets removed and taken offsite in case there's a disaster.)
One final thought: The poster wasn't actually running a 100GB filesystem were they? I'm thinking that a power glitch could cause a world record to be set for the longest fsck-on-reboot run. Plus I'd think that backing up such a beast would be a challenge. I tend to keep my filesystem sizes no larger than what I can fit on a single tape cartridge... just to make life simple. (I'm used to having to pipe `df' commands through `more' at work so I don't mind lots of mount points. :-) )
CUR ALLOC 20195.....5804M
There are lots of good suggestions about RAID, copying to another drive, etc. But as long as the backed-up data is stored in the same physical location (home) as the original, you are not safe from natural disasters or theft. Maybe this is paranoia but it depends how important your data is to you.
I have isolated the data that changes on a regular basis that I want to back up. Mainly this consists of mail and financial data, in the 10-20MB range. I have 2 boxes so each box backs up data to the other box. But just to be safe I back up to a trusted friend's machine also. It's a large ftp transfer but not bad at all with cable modem.
For other data that I want to save that doesn't change regularly (photos, mp3s, etc) I use CDRs.
i trade some usa food aid packages for DLT2000XT Internet is GREAT!
Internet is Great!!! junis
True -- but given the article's "affordable", "home", "10-100GB" parameters, I'd be quite happy regarding hard drives as a real solution.
Don't expect one hard drive to last you 10 years, because 10 years from now, systems with 40-pin IDE won't exist. (And likewise, neither will readers for the tapes you purchase today. When was the last time you saw a DC600 cartridge tape drive available?)
If you're talking longterm storage, leave your "backup" drive somewhere secure, and expect to replace it every 3-5 years. (That'll probably be a 500G serial IDE drive 5 years from now, a terabyte-range solid-state device 10 years from now, and a petabyte-range holocube 20 years from now.)
sure, if you are just backing up your own crap and it's under 20,000 files, i'm sure everyone here has that under control. everyone knows how to get a second drive and back up their stuff.
the problem is when you have hundreds of THOUSANDS OF FILES or you are responsible for backing up hundreds of people's stuff...without tape rotations your dead meat. becoming aware of WHEN things are deleted can take months.
case in point: i had a researcher delete 10 critical files for a 5 million dollar grant. The lady has in the vicinity of 100,000 files. she did not know she was missing them for 8 months.
i pulled out my archived monthly DLT, and restored her files.
the last 6 months tapes did not have her files on them...but the 7th did.
For my money, it's hard to beat the new ATA RAID cards that are out. Most can be had for less than $100.
Couple that with two or four 80GB drives, for less than $150 each, and you've got yourself a pretty nice array that will keep your data safe against all but the most horrendous problems.
Even with this, you're probably wise to have some offline backup solution to go along with it.
What data would you really want back if your house was swallowed by a hole in the ground? In that situation, do you really need access to your 30GB of MP3 files?
If the anwers is that you really only need access to your Quicken files, then arranging to have those backed up online should be pretty cheap and easy.
Summary: cheap ATA RAID for hardware redundancy, online backup for truly life-critical files.
Err, am I the only one on Slashdot that realizes that this guy's post was meant to be funny?
He admits to sharing his bandwidth with all his neighbors using 802.11. (Some people do that but don't admit to it in a /. post.) If that didn't tip you off, everyone agreeing to back up their personal data to all their neighbors' hard drives should have.
If I can be modded down for being a troll, can I be modded up for being an orc, or a balrog?
I currently have about 90 gig of total home storage...soon going to about 130 gig. But, there's only about 40 GB actually being used for data...MP3's, digital photo's, photo editing stuff, old book reports...you know what I mean. A lot of space is OS files because my total file storage is across 5 machines. Screw backing up the programs! For the most part, I'm going to need to dig out the CD's/disks anyway.
I spread the backups around. Run a script to handle all the machines sequentially. 10 gig goes to a machine down the hall....runs across the fast ethernet wire sometime at night and gets compressed at it's destination. Another 15GB comes off of that machine and is dropped two floors below on a Samba share. Gets compressed too. So on and so forth. One log file gets written...PGP'd and SMTP'd to greet me when I get to work at 6:00 AM.
Yup...it's a pain sometimes. But I more efficiently use the storage without dedicating any one unit. I always leave enough space for other work. I increase tolerance so that if a box dies for good I only lose a piece of the backup scheme. The whole shebang runs while I'm snoozing and can afford network traffic and CPU cycles to compress. And they're all full backups to boot.
I've been nailed a couple of times, but not fatally with this setup. Oh yeah...all the boxes are on UPS's. That's important. I've lost more to the power company than to ghosts in the machine.....
Unless you're creating 10 - 100 GBs of *new* data between every backup, it might be simpler (and cheaper) to use incremental backups.
Instead of dumping all 100GB of files every time, 95% of which haven't changed since the last backup, use an incremental backup program to write only the 5% that actually changed. After the initial archive, the backup files will be significantly smaller, and could potentially saved on CDs.
ShoutingMan.com
So 100GB drives are getting more and more common. So what? Are backups really any more a problem than when 200MB or less drives were common? We didn't have CD-R back then, much less DVD-R... Only smaller capacity tapes and floppies. And, people with 100GB drives... is what you have on that drive really of more substantial value than what you would have been able to store on a 200MB drive? In my opinion, I don't value what I have on my 30GB drive any more than I valued what was on my 200MB drive "way back when." I think the ratio of data I cared about to data I don't care about was about the same then as it is now. And the backup technology available today can hit that percentage (which I think is about 20%) pretty easily (CD-R) with about as much relative difficulty as floppy-swapping could have back then. Everything else can be reinstalled... no big deal.