Best Home Backup Strategy Now?

jollyreaper writes "Technology moves quickly and what was conventional wisdom last year can be folly this year. But the one thing that's remained constant is hard drives are far too large to backup via conventional means. Tape is expensive and can be unreliable, though it certainly has its proponents. DVDs are just too small. There are prosumer devices like the Drobo, but it's still just a giant box of hard drives, basically RAID. And as we've all had drilled into our heads, 'RAID is not backup.' When last this topic came up on Slashdot, the consensus was that hard drives were the best way to backup hard drives. Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week. Is there any better advice these days?"

Comments

SSD

[shentino]

Back up to a honking flash drive?

Re:SSD

[Zurk]

This is a bad idea. Other than the ludicrous cost of the SSD, flash drives tend to fail all at once. bam! and all your data is gone. This is also why i do not use a USB key for backups.
On my system everything is dumped on a 2TB mirrored system (simple 2 x 2TB HDDs running debian software encryption + RAID lvm) and periodically backed up to blu ray DLs in duplicate. At $10/disk from japan (see ebay) two verbatims back up 50GB in duplicate for $20.
Typically it takes 2-3 months to generate that amount which means its cost effective. DVDRs (Taiyo Yudens) fill the gap if there is not enough data to justify a bunch of blu rays.
 

Re:SSD

[commodore64_love]

A flash drive is probably the most stable technology. The drawback is the high expense. My strategy is several fold:

- Nearly all my home movies are recorded on Super VHS tape. Being analog if the tape gets damaged, it will still be watchable (wrinkles appear as momentary scanlines).

- My downloaded porn is backed-up on an external USB drive. If the c: drives fails, I can just copy the stuff over (and vice-versa).

- Stuff that I can buy on DVD or CD like Babylon 5 or Star Trek, I buy. These discs are physically pressed with pits so they won't self-erase themselves like DVD-Rs or CD-Rs tend to do. They should last the rest of my life.

Unfortunately none of these strategies protect my from a major accident like a house fire. I just need to make sure not to do something stupid like fall asleep with a cigarette in my mouth.

Re:SSD

[mdwh2]

Flash drives have the advantage of their small size. No one wants to lug an external hard disk around with them, and discs don't fit in one's pocket either. So what good is your hard disk back up system when your house burns down, is flooded, or burgled?

The answer is to use a range of devices - just because I have a 500GB drive doesn't mean that all data on that disk is equally important. I can use a second hard disk for most data (e.g., videos, mp3s), and then an 8GB drive (which costs, ooh, about £5 - hardly a "ludicrous" cost) is more than ample to store the things that I couldn't lose, such as my source code, personal photos etc.

Re:SSD

[router]

Discs can and will bitrot, I have had commercial pressed CDs go bad, even though they are in storage and I only used them to rip to mp3. Laserdisc folks noticed it a long time ago. I personally use the async mirrored external method. Turn on mirror discs, mount, rsync each active external to its mirror, unmount external mirror and turn off. This way I can't accidentally delete and they are direct replacements. Seems to work so far.

Re:SSD

[MaskedSlacker]

an 8GB drive (which costs, ooh, about £5 - hardly a "ludicrous" cost)

A 2TB hard drive costs $230. A 16GB flash drive costs $42. That is 200 times the per GB cost. If you are trying to store large volumes of data (which is what was being referred to) SSDs ARE ludicrously expensive. 200 times as expensive as hard drives. Given that an external 2.5" drive DOES fit in my pocket, and costs FAR less per GB, there is little justification for the thumb drive.

Re:SSD

[tchuladdiass]

DVDs are slightly better than CDs. A CD has the physical pits pressed on the top of the disk, then a thin silver backing is placed on top. When that backing gets damaged, goodbye data. A DVD, on the other hand, has the silver reflective part sandwiched in the middle of the disk. Therefore it is more resistant to physical damage.

Re:SSD

[pantherace]

Indeed, the first CD I purchased, Civilization 2, is now useless, as anything other than a conversation piece/example to pull up online of why CDs aren't backup/example to pull up online of why I 'pirated' something that I 'own'.

Lots of mileage out of that disk, both for games, and for smacking down online arguments.

Re:SSD

[Divebus]

Depends on how much you're backing up. If it's a few files, use some DVD-Rs. If it's a desktop, just get another hard drive and don't drop it. If it's lots and lots of stuff, the first 100TB would be cheapest on LTO4 data tape. Yeah, the drives are a few thousand but the 1.6TB tapes are only about $50 and will more than make up for the drive cost in capacity and speed after 100TB. The second 100TB would be even cheaper.

Re:SSD

[Aazzkkimm]

In my experience, DVDs fail much more frequently than CDs. I used to back everything up on DVDs until one day I tried to use one that was about a year and a half old and it was bad. After checking others I found that I had a failure rate of about 50% of anything older than one year. This is with name-brand discs (multiple brands), both DVD-R and DVD+R.

On the other hand, I've got CDs that are well over 10 years old, and still read and write just fine....

Re:SSD

[MobileTatsu-NJG]

an 8GB drive (which costs, ooh, about £5 - hardly a "ludicrous" cost)

A 2TB hard drive costs $230. A 16GB flash drive costs $42. That is 200 times the per GB cost. If you are trying to store large volumes of data (which is what was being referred to) SSDs ARE ludicrously expensive. 200 times as expensive as hard drives. Given that an external 2.5" drive DOES fit in my pocket, and costs FAR less per GB, there is little justification for the thumb drive.

Your math only works if you have two full terabytes to fill up. If you only have 16 gigs of stuff, the cost per gig isn't that useful of measurement.

Believe it or not, I agree with your assessment, at lesat in the context of my own needs. I'm just not convinced on the 'one size fits all' argument.

Re:SSD

[jonbryce]

You can get 2.5" 500GB external hard drives which will fit in most pockets.

Re:SSD

[Falcon4]

Au contraire. I'd recommend backing up a Flash drive to your hard drive, for the various reasons stated elsewhere in this thread...

Re:SSD

[commodore64_love]

Your comment is off-topic. We weren't discussing DVD-Rs or CD-Rs (which fade and are a lousy backup system), but commercial DVDs/CDs which have the pits physically pressed into them. They are permanent. If you buy Star Trek today you will still be able to play it when you're 80.

As for laserrot, when the silver oxide peels off the disc, I've never experienced it. I've heard that if you live in a high humidity environment like the tropics then you might have a problem, but I live in the relatively-cool northeast. Ssome of my discs are twenty years old and everything still plays just fine.

Re:SSD

[commodore64_love]

It seems clear to me you abused the disc in some fashion. I have music CDs that predate Civ 2 by nearly ten years, and they all play flawlessly. If you treat your discs like old-fashioned records, they'll last. If you hand the CD to a kid, they'll die within a month (even those supposedly-industructible carts I've seen get destroyed by kids). As with everything, it's ultimately upto the user to protect what he/she owns.

Re:SSD

[FlyingBishop]

That's a fairly naive cost assessment. The lifespan of that SSD is much shorter than the 2.5" drive. I'd suspect that it is at least half as long, making it in fact 400 times as expensive, and (most importantly) requiring more frequent backups.

Re:SSD

[davester666]

No. He want's a "Home Backup Strategy".

So, in the event his home fails, where will he live and with what possessions.

Common strategies are:
-become wealthy and purchase multiple homes, duplicate possessions, have good fire and burglar systems
-a cheaper but less effective method is to purchase home insurance

Re:SSD

[Rockoon]

That's a fairly naive cost assessment. The lifespan of that SSD is much shorter than the 2.5" drive.

Citation? This is going against prevailing wisdom (and so is the disasterous failure arguement, since they fail gracefully)

Re:SSD

[Anonymous Coward]

Sounds like you are referring to pressed discs, (ie: bought in a store) this does not apply to burnt media.

Re:SSD

[spire3661]

Blu-Rays are even better. They have a very hard, thick coating of plastic on the data side, one of the few reasons im glad BR won. http://en.wikipedia.org/wiki/Blu-ray_Disc#Hard-coating_technology

Re:SSD

[Anonymous Coward]

Your comment is more off topic than his is. This thread is about data backup, and this branch is about backup to CD and DVD. (in other words CD-R and DVD-R, etc)

Re:SSD

[Antique Geekmeister]

Yes, he probably 'abused' it by playing the game. Music CD's have relative simple behavior: they read a track at a time, in a linear fashion, and are unlikely to do hunt&peck data searches. Game CD's are begging to suffer the worst abuses of start-stop usages, for many continuous hours, in a way no music or video media will suffer unless used by a rapper stringing together 3-second segments in strange orders.

Re:SSD

[commodore64_love]

>>>Your comment is more off topic...this branch is about backup to CD and DVD (in other words CD-R and DVD-R, etc)

Yes Mr. Coward, but if you go back to my original post, I stated one method of backing-up data is to simply BUY the music or video on commercial CD or DVD. Since you're apparently too lazy to read the whole thread, I'll quote my original post that this reply links to:

My strategy is several fold:
- Nearly all my home movies are recorded on Super VHS...
- My downloaded porn is backed-up on an external USB...
- Stuff that I can buy on DVD or CD like Babylon 5 or Star Trek, I buy. These discs are physically pressed with pits so they won't self-erase themselves like DVD-Rs or CD-Rs tend to do.

Replacing downloaded songs or TV shows with storebought versions IS a method of backup. And I clarified I was discussing *pressed* discs not the writable kind.

Re:SSD

[MobileTatsu-NJG]

That's a fairly naive cost assessment. The lifespan of that SSD is much shorter than the 2.5" drive. I'd suspect that it is at least half as long, making it in fact 400 times as expensive, and (most importantly) requiring more frequent backups.

Assuming what you say is true (due to wear levelling and the utter lack of moving parts, I'm not inclined to agree), the issue I brought up is still there. It's only more expensive if you reach the threshold.

Everybody has different things to consider. Cost-per-gig is not a one-size-fits-all answer.

Re:SSD

[chriseh]

Wow, you really have no understanding of how this technology works. Even if a rapper were to have 3 second segments on an audio track, it sill has to be pressed in a linear fashion. Also, stop/start has no impact on the data of disks, since there is no physical contact with the data portion of the disk in any way. It's simply a laser that scans the surface.

Re:SSD

[westlake]

Unfortunately none of these strategies protect my from a major accident like a house fire. I just need to make sure not to do something stupid like fall asleep with a cigarette in my mouth.

You could buy a media rated fire safe.

You'll pay a stiff premium over the price of a safe that only has to protect paper. "Fahrenheit 451" and all that.

You might also consider renting a safety deposit box at your bank.

For porn I'd want instant immolation. Media no more durable than flash paper.

Re:SSD

[adolf]

- Stuff that I can buy on DVD or CD like Babylon 5 or Star Trek, I buy. These discs are physically pressed with pits so they won't self-erase themselves like DVD-Rs or CD-Rs tend to do. They should last the rest of my life.

Waitafrigginsecond -- you mean to say that your backup strategy for store-bought discs is: (drumroll) NOTHING?

Re:SSD

[fractoid]

For porn I'd want instant immolation.

Immolation porn? Sicko.

Re:SSD

[Moryath]

Bullshit.

I've got a pretty large collection of old games. One of the things I've noticed about them is that, even for factory-pressed discs, you have to deal with the following factors:

- Unsealing. If the disc wasn't sealed properly, it will begin to oxidize. Several of mine have done this. It moves from the edge inwards.

- Plastic clouding. Even if it's stored in a cool, dry place and kept out of sunlight, clouding can and will occur. The cheaper the disc was produced, the sooner this happens. Many of my older titles (Civ 2 among them) suffer from precisely this problem and that's one reason I began taking them all and making ISO backups just in case as well.

- Physical usage. If it's something you use often, wear and tear occurs. Games have access patterns that are clearly different from music CD's. For one, music CD's tend to spin at the standard 1X rate (unless you're seeking), which doesn't cause the disc to deform much (as opposed to full-speed data access... see the high-speed footage from the Mythbusters ep on this if you want). Two, data CD's access a lot more erratically - and the more "back and forth" you have, the more likely you are to scratch the disc from ordinary use. If you put an older CD (not physically designed for higher-speed drives) into a modern higher-speed drive, you can make the problem worse.

Of course, sure, you could abuse them. I saw a Rock Band disc once that looked like an LP - someone had put it in at a convention 15 feet away from the DDR setup, and the room had been bouncing so violently that the disc read head had literally put grooves into the disc. But I doubt that's what happened to the gpp's Civ 2 disc.

Re:SSD

[edack]

How about a Teletype ASR 33 with mylar tape? Sure it takes while, but I can still read my tapes from 1977 (DG Nova 820 16 MB Core) How about you?

Re:SSD

[richardellisjr]

Sorry, I haven't read all the threads in this article and someone may have proposed a similar resultion. My proposal is thus (word for the cautious I'm still working out the details of this): first off find some one who also wishes to backup their data (in my case it's my father). First etup an rsync going between the two computers (this will need two rsync cron jobs on each side). Second setup either a versioning file system or have the files somehow check out and in the files to version control as it's doing it's thing (this is the part I haven't worked out yet). This theoretically will work in my situation since my father is buying the same NAS I have (a DLink DNS-323 which I can't praise enough for it's hackability) since only have to work out the logistics for one OS and one piece of hardware and only have to wory about firewalls.

Re:SSD

[_merlin]

It's actually a very thin coating, which is the very reason it has to be so hard - because of the lase wavelength used, a thicker coating causes problems.

Re:SSD

[Antique Geekmeister]

_Yes_, I understand the technology. No, a rapper does not have to "press" it in a linear fashion: half the fun for a skilled disc jockey at a rap session is tweaking the effects, and that requires dynamic control, not a "pressed" recording. It's the difference between a live concert and a live concert tape.

Moreover, while ideally there is no physical contact, in real live, heads crash. Disks flex, and deform, as they're spun and stopped and spun and stopped. So please do not assume that the wear of a music recording (with several minutes of consecutively recorded music) are handled or physically wear the same as randomly accessed data. That kind of simplification is why people thought CD's and DVD's would last for decades, and why in normal use in real life they last far less.

Re:SSD

[Nutria]

Unfortunately none of these strategies protect my from a major accident like a house fire.

Have you considered making copies of those home movies and mailing/driving them to your mother's house? There's a very small likelihood of both being destroyed in the same month.

Unless you both live in the same disaster zone. That's why my uncle mailed backups from L.A. to LA.

Re:SSD

[Magic5Ball]

Your statements are logically consistent, but I respectfully agree with Mr. Coward for the following reasons.

1) Main features of backups non-exhaustively include the ability to restore files and to restore functionality. You might be able to restore functionality by ripping from a commercial disc, but the resulting byte stream will almost certainly be different from the original. Even with respect to restoring functionality, ripping from a commercial disc will almost certainly take longer than copying an encoded file with the same human level content. And that does not include important meta-data such as ratings, access times, album organization/collections, album covers, etc.

2) Clearly the intent of this entire story is to discuss best practices with respect to backing up user generated data, of which one special case is audio-visual content, of which one special case is content which is available commercially as pressed discs.

In short, you have presented one relatively expensive solution for one special case of a special case, which as you have acknowledged, is not generalizable as you've described it to the problem being discussed in the story. Pressed discs as a backup solution fit neither the spirit nor the letter of the problem described.

About the only way to discuss pressed discs with relevance to a home backup method would be to point out the costs of getting a disc pressed for really irreplaceable data (currently around $10^3 to $10^4 per disc), or the relative merits of the various optical disc formats with respect to the probable availability of devices to read them at various points in the future (signals from a CD-* can be read with a flatbed scanner, but blue laser devices which can record at a higher density are more likely to be relevant for data storage in two decades than red laser devices).

Re:SSD

[Architect_sasyr]

No problem.

suspect: imagine to be the case or true or probable; "I suspect he is a fugitive"; "I surmised that the butler did it"

Dictionary Words for When you are trying to be smart

Re:SSD

[ls671]

Well here I go again:

1) You need to have a backup strategy first:. The strategy will be elaborated according to your needs and the amount of risk that you are willing to take.

2) Incremental backups should be part of your strategy.

3) You should backup on a second computer to prevent against a single point of failure (e.g. you controller). It is best that this computer is physically hosted at a different site than your production computer to prevent against disasters. If not, you should at least take full images once in a while that you store somewhere else. Again, this all depends on the amount of risk that you are willing to take, no backup solution is 100% proof. A fire at my bank, a fire at my prod site and a fire at my backup site occurring at about the same time would leave me totally screwed.

http://slashdot.org/comments.pl?sid=1292323&cid=28588177&art_pos=40

http://slashdot.org/comments.pl?sid=1292323&cid=28588079&art_pos=41

http://slashdot.org/comments.pl?sid=1277921&cid=28430691&art_pos=76

http://slashdot.org/comments.pl?sid=1277921&cid=28429713&art_pos=78

Re:SSD

[Anonymous Coward]

You should backup on a second computer

"back up". The word "backup" is a verb, not a noun. (more info)

Also, based on the posts referenced in your links, you should learn the difference between "lose" and loose".

Re:SSD

[ls671]

> "back up". The word "backup" is a verb, not a noun.

Actually, it is the other way around according to your link ;-)))

Re:SSD

[hab136]

Moreover, while ideally there is no physical contact, in real live, heads crash.

Hard drives heads can crash. At rest, hard drive heads sit on the magnetic disc itself. Once the disc in the drive is spinning, hard drive heads float over the magnetic disc on a cushion of air. If that cushion fails somehow, the head hits the spinning disc and "crashes". CD lasers however sit a fixed height below the CD; there's no air cushion to fail. Since the CD laser doesn't (can't) change height relative to the disc, there's no way for it to touch the disc, short of applying a hammer or other blunt object.

Disks flex, and deform, as they're spun and stopped and spun and stopped.

Indeed - and data discs are spun at 40x the speed of an audio CD (depending on the drive), thus more stress on the disc itself.

So please do not assume that the wear of a music recording (with several minutes of consecutively recorded music) are handled or physically wear the same as randomly accessed data.

Newer CD drives are constant angular velocity (CAV) - they spin the same speed regardless of where you are reading on the disc, so access pattern (random vs sequential) won't change the stress on the disc. The extra stress comes from when the drive spins up and down because there are pauses between accesses, whereas a music CD will be started, spin continuously, then stop.

Re:SSD

[Anonymous Coward]

Knackers. You have a disk with angular momentum. Accelerating or decelerating it requires torque. Do that often enough and it will start to crack.

Re:SSD

[Hognoxious]

Have you considered making copies of those home movies and mailing/driving them to your mother's house?

It's further to the post office. What's more it's quite difficult to drive up stairs.

Re:SSD

[slim]

From common usage (rather than one pedant), it's a verb, a noun, and an adjective.

Verb: You should backup your data
Noun: All my data is on a backup
Adjective: You need a backup disk

Re:SSD

[Rockoon]

umm, still trying to figure out how this was a reply to me.

Re:SSD

[Antique Geekmeister]

You've got me on the "head crash" point somewhat. You'd think that a CD laser couldn't head crash, but you've got spinning objects inside boxes with a head that has to move radially, and a lot of them are made amazingly poorly. Mechanical absurdities happen, and the insertion and removal of CD's and their physically open nature tends to allow a lot of debris to accumulate. The result is dust and hair and grit causing mechanical damage: I'm not saying this is a large source of the common damage, but I've certainly seen it in household machines and laptops and poorly maintained server environments. (Company servers shoved in poorly maintained closets, for example.)

And yes, CAV is helpful in many ways: but the random access introduces some fascinating lateral stresses on the mounted reader and potentially writer components, and those darn things _fail_ and do serious physical abuses to discs in the process.

Re:SSD

[Quothz]

From common usage (rather than one pedant), it's a verb, a noun, and an adjective.

Common usage doesn't mean what I think you think it means. Merriam-Webster, the source that actually matters, says "backup" is a noun and "back up" is a verb. The usage you posted as adjectival is a noun form.

Re:SSD

[commodore64_love]

>>>Music CD's have relative simple behavior...Game CD's are begging to suffer the worst abuses of start-stop usages

No because if I recall correctly, the Civilization 2 CD is not used but instead copied to the hard drive. But even if it were used directly, I don't see how jumping the laser from track-to-track will damage the underlying silver oxide. It's not a needle on a record player that slowly-but-surely erodes the track.

I have PS1 games that are nearly fifteen years old, and they still work just fine, despite being my favorites and played frequently in that less-than-friendly console. It all goes back to the user - if you treat your CDs with care like records, then they won't get scratched and will last the long haul. If you toss them into the tray and scratch them, or drop them, then of course they're going to fail.

Speaking of records, one of the most fragile mediums ever produced, I have some dating back to the 1950s and they play just fine.

Re:SSD

[commodore64_love]

In the VERY unlikely event your CD or DVD died, you could always go buy it again. The reason I say unlikely is because I've never had any of my storebought discs fail. That's over twentyfive years of collecting.

Re:SSD

[Killjoy_NL]

I think you mean "Whoah that's so hot!" :D

Re:SSD

[router]

Tell that to xbox360 owners; or try it yourself. Take your vertical xbox360, and make it horizontal while the disc is spinning. That Call Of Duty disc you _were_ playing? It now has a nice ring scratched in it and requires a trip to place of purchase to swap.

Re:SSD

[Hafnia]

True ... only it was Tiger Woods 09 , and somebody lifting the console from the floor. The disk was seriously damaged !

Re:SSD

[Hafnia]

It's not very scientific to take your own personal experience and expect the rest of the world to get the exact same results ! There is probably a very big variation in the quality of the CD's and DVD's out there !

Re:SSD

[Hyppy]

Quit with the "Wahhhh, you're offtopic!" bullshit. This is a discussion about backup solutions, so factory-pressed CDs and DVDs don't exactly belong, either.

Re:SSD

[iamhassi]

"umm, still trying to figure out how this was a reply to me."

he said "I'd suspect that it is at least half as long", meaning he imagines it to be true without any facts to back up his claim.

I suspect he doesn't know the answer and is incapable of googling it.

Re:SSD

[adolf]

But you ignore the human aspect of it all. I once had a nicely-packaged, limited edition 24kt gold Sony release of Pink Floyd's Wish You Were Here, which I accidentally dropped a screwdriver on. Went right through the top layer of the disc.

I can't just buy it again -- it's unavailable outside of collectors circles. I've got other limited-edition works which can't just be bought new, too.

And then there's the aspect of young children handling media (or friends with young children who like to borrow discs).

A dozen years ago when that Floyd CD got destroyed, I couldn't pull this off due to costs, but nowadays, I rip everything, and just play the backup. Costs almost nothing, and keeps things secure.

It's not even very time-consuming -- blank goes into one $25 drive, source goes into another $25 drive, I sing a short incantation with the mouse, and usable media gets spit out sometime later. And on a quad-core machine, I don't see any performance hit at all while this is going on.

Re:SSD

[arkane1234]

Funny, since we all seem to agree that when we store our data for archiving we are performing a backup.
This is a technically based word, so webster is by far not the place to look.
Next thing you will say is that "type" is a description not a verb.... geez
It's all semantics.

Do we have to bring this up over and over again???

[Anonymous Coward]

rsync.

External and Online

[basementman]

Switching off-site backups every week is an unnecessary hassle. Back up to an external hard drive and an online backup service. Anything more than that is overkill unless you have really important data.

Re:External and Online

[eggman9713]

That's fine if your ISP doesn't have draconian caps. I have over 2TB of stuff (legal, mind you, lets not get a redundant "You must be pirating" theme going). Mostly photos and video content. My ISP caps at 100GB per month. Online backup is not a viable option except for my most important stuff. I use the offsite backup drive method, however I don't have two sets that I swap, I just have one offsite backup that I bring home from work ever other week.

Re:External and Online

[InsertWittyNameHere]

It's true, being realistic my life wouldnt fall apart if I lost all of my files so I don't bother with multiple extra backups. (Work is a different story)

Most of my files (like documents, spreadsheets, PDFs, etc) I backup onto an 8GB USB key.

I email some personal stuff (like my resume) to myself so I have a copy of it on Gmail plus I can access it from anywhere.

Music and movies go on an external drive.

For a simple solution if you have a Mac, just get an external drive and use Time Machine.

Re:External and Online

[Pentium100]

Not only caps, but some people (like me) have connection with a bad upload speed. I can do 150KB/s if I load balance over two connections but a single connection can only do 80KB/s, way slower than my DDS4 tape drive (not to mention LTO1).

Re:External and Online

[JustOK]

so, you're pirating space at work for your off-site storage, or do you pay them?

Re:External and Online

[Anonymous Coward]

You have 2TB of data, but how much are you actively adding/changing on a weekly basis? Sure, it'll take a while to upload your initial 2TB, but incremental backups should not too much bandwidth.

The problem of course is finding online backup solutions that do incremental backups reliably and efficiently.

Re:External and Online

[Anonymous Coward]

2TB is 40 blu ray DLs. at 10 bux apiece you can back up your files for $400.

Re:External and Online

[Anonymous Coward]

One of the reasons to have two external drives is that the one you are using to make a backup is no longer external. If you get hit by lightning, or you computers power supply decides to EOL itself, while you are doing the backup to your one and only "external" drive, you will see my point.
Another benefit of having two external drives is that if you screw up a restore from backup, you have another one to try it on. Stupid things like these happen way more often that you would think, often more often than simple drive failures.

Re:External and Online

[rvw]

That's fine if your ISP doesn't have draconian caps. I have over 2TB of stuff (legal, mind you, lets not get a redundant "You must be pirating" theme going). Mostly photos and video content. My ISP caps at 100GB per month. Online backup is not a viable option except for my most important stuff. I use the offsite backup drive method, however I don't have two sets that I swap, I just have one offsite backup that I bring home from work ever other week.

Some of those online backup services offer the option to send in harddrives or tapes to make the start. If you stick with offsite backups, you can leave one big basis backup at work, and only swap the incremental backups. Then two simple 2.5" usb drives are big enough to handle that.

Re:External and Online

[rvw]

For a simple solution if you have a Mac, just get an external drive and use Time Machine.

Get two external 2.5" usb drives, and always leave one of them offsite.

Re:External and Online

[Darkness404]

You are assuming the average computer has a Blu-Ray drive, most don't.

Re:External and Online

[Anonymous Coward]

average computer from when ? 1970 ?
fine. add $150 for a blu ray burner. gee...wasnt that expensive ?
http://www.newegg.com/Product/Product.aspx?Item=N82E16827106254

Re:External and Online

[mdwh2]

For a simple solution if you have a Mac, just get an external drive and use Time Machine.

Completely useless if your computer is nicked, or damaged through fire, flood etc.

Re:External and Online

[YrWrstNtmr]

fine. add $150 for a blu ray burner. gee...wasnt that expensive ?

Now you're up to $550. For that, you could get a whole nother PC w/ 2 ea 1TB drives. Doesn't have to be even vaguely fast, just enough to host the drives. Access time to the entire backup is as fast as booting it, instead of digging through a stack of 40/50/60 disks.

Re:External and Online

[Anonymous Coward]

And all you have to do is swap out 40 disks. A 4x burner would take 30 hours.

Re:External and Online

[sadler121]

Your average slashdotter is not going to get a cheapy $550 computer. Your average Joe maybe, and then they will complain to us that their computer is so slow...

Re:External and Online

[Sorthum]

Yes, because work is going to complain about the hard drive in my desk drawer.

I also keep a pair of shoes there as well; my manager's never complained about that either.

Unless you work for a fast-food type company, I'd imagine most places are cool about this.

Re:External and Online

[Darkness404]

...An Intel Atom CPU would do the job for storage. The point is to store the data, not so you can play games on max settings.

Re:External and Online

[Anonymous Coward]

I can see the external hard drive if you have more data than will fit on 2 or 3 DVD-R or DVD-RW disks. I would NEVER recommend backing up to any online service. Too many security and privacy issues. I do an off-sit backup about once a month, and local backups to DVD-RW disk once a week. But then I don't have terabytes of data to back up.

Re:External and Online

[fruitbane]

That's certainly preferable to a number of other solutions. What we really need is something very information dense and relatively stable over time. Speed is less an issue. It's great to backup quickly, but speed should take second place to the other two issues.

Re:External and Online

[InsertWittyNameHere]

If the time machine drive gets damaged and you only have that one time machine then, ya, youre toast. However if you lose just your computer you can still get your data from the time machine. You can even restore from it to a new computer/drive and continue pretty much where you left off.

Re:External and Online

[westlake]

Sure, it'll take a while to upload your initial 2TB

His service is capped at 100 GB a month.

Uploading 2 TB would take the better part of two years - assuming 100% of his traffic was dedicated to the process.

It would be simpler and cheaper to use a courier service.

Re:External and Online

[Anonymous Coward]

Buy a fireproof safe you can bolt to the closet floor and you can have two backups and alternate between them.

Re:External and Online

[godrik]

Actually 550$ for a computer is quite expensive... My regular computer is a eeebox. I paid it 300$. It makes alsmost no noise, is small and is power efficient. It won't run far cry, but I do not want to run it anyway...

Re:External and Online

[Dare nMc]

Sure, it'll take a while to upload your initial 2TB, but

This also means it will take a while to recover your initial 2TB? I would think online backup is a great idea for being able to get that last days or hours worth of changes back with "minimal repetitive manual intervention required for backup". But that would only be after you have used the primary recovery plan to get all but the latest data/applications running. Also since I wouldn't trust any backup system that I don't test occasionally, that initial transfer time seams to rule this out as the only off-site solution.

Re:External and Online

[drinkypoo]

Or for that, you could just buy three external drives.

One thing I do when I'm running Linux (or most anything but Linux) is make bootable backups. It's easy as pie these days — the latest releases of grub will let you use the device's UUID even for groot, which eliminates most of the problem cases with grub on removable devices. Then you can make a backup with tar, and update it thereafter with rsync (in single user mode.)

It more or less goes without saying that you'll be able to get a better used computer when you need to restore your backup, so unless you plan to need it on horribly short notice, you're better off just buying disks. I happen to like external disks, because you can always turn an external into an internal, but you can't always lay your hands on an enclosure. But I guess that's really no different from buying a PC and stuffing it with disks, except the convenience factor.

Re:External and Online

[Tobenisstinky]

Why would you buy a computer to do backup. You already have the computer, and I doubt you're using so many cycles it can't do a backup at some point. Buy 2 desktop 4 bay enclosures, slap in 4 1 TB drives, swap 'em around every week or month, when you run out of space slap in some more drives. I know it will run you a bit more than $550, but not much; and it's (semi)-portable! It will also be a hell of a lot faster than burning disks or writing tapes for Ch%^*@'s sake!

Re:External and Online

[MattSausage]

Your average slashdotter is not going to get a cheapy $550 computer. Your average Joe maybe, and then they will complain to us that their computer is so slow...

I would suggest your average slashdotter will be very distressed if they have to pay $550 for a simple storage system. To suggest someone is going to buy a gaming pc for storage and backup is exactly what your 'Average joe' would do.. not a slashdotter.

Re:External and Online

[Chris Mattern]

However if you lose just your computer you can still get your data from the time machine.

It's a pretty selective fire or flood that gets your computer but not the time machine drive attached to it. And if it gets stolen, odds are that the thief will decide he won't mind owning a nice external drive to go with it.

Re:External and Online

[lymond01]

Your average slashdotter WILL get a cheapy $550 computer if it suits the needs and place it in his media closet away from his $1200 gaming PC. Doing disk to disk backups is probably the best solution right now if you have terabytes of data. Just keep adding disk and as the price goes down and it's time to renew your hardware, get another $550 computer that has twice the storage.

Re:External and Online

[KillerBob]

Intel Atom? Geez. You're spending too much...

My network fileserver is a Via C7 1.5GHz with 1GB of RAM, running Linux. It's connected directly to the router via gigabit ethernet (the router shares with the network via 802.11a/b/g/n). The fileserver has a 500GB hard drive on which the operating system itself is installed, and a 1TB drive connected via eSata. Its contents are served up to the network via Samba and FTP. Anything important gets put on the network hard drive, as well as on the computer that generated the content. As all of the workstations are laptops, they're easy to grab in the event that I need to get out, and they're usually already with their user if they're not home to grab it. Anything *really* important also gets shared around the family, so that there's always at least 2 or 3 people who have stuff like family photos and important documents.

There's really no point in spending large amounts of money setting up a network fileserver, or a mirroring/striping RAID, or having two external hard drives where you keep one onsite, one offsite. Save yourself the trouble. The total cost for my network file server was about $200, including the case, and under full load it uses less electricity than a compact fluorescent light bulb. Adding extra storage to it is as simple as buying a new hard drive to connect.

Re:External and Online

[Anonymous Coward]

I never spend $400 from scratch computer. Where the hell are you guys getting these numbers? By 6 month old parts, newegg.com ..

Re:External and Online

[RichardJenkins]

Actually most online backup software would find a whole lot of duplicated blocks of data in that - meaning he'd probably end up uploading a whole bunch less than 2TB. The point stands though, if he wants to use remote backup for that much data it's be nigh impossible with most providers - and likely cost a whole bunch too.

Re:External and Online

[Darkness404]

Where did you get the motherboard used in the build? Because the only cheap ones I can ever find on Newegg with decent reviews that said they worked with Linux are Intel Atom CPUs.

Re:External and Online

[JayTech]

Most, if not all, of the boards I've ever purchased from Newegg work perfectly with Linux. For instance, I know that the PC CHIPS A15G AMD board works with Fedora, Red Hat Linux, and Ubuntu. IMHO, almost any board that's more than a few months old will be supported by almost any Linux distribution... Linux has far greater hardware compatibility than, say, M$ Vista.

One good strategy I use is to pick out the board I want, then pick out the Linux distribution, and Google both of those together to see if there's any info available. If you don't find any results, then post in the distribution's forum and find out if there is support for the motherboard you picked out. Easy as that!

Re:External and Online

[chammy]

Your average slashdotter is not going to get a cheapy $550 computer.

A $550 computer may be cheap but with the right parts it's not going to be slow. At work we just installed a $600 quad core system with 4GB of ram and 2TB of disk. Explain to me how this is a "slow" computer?

Re:External and Online

[hedwards]

Or you can house a RAIDZ in an external enclosure. It's even faster as you don't necessarily have to reboot unless it's really necessary and you've got a way of easily checking for and correcting bit rot.

The main downside is that you're going to have to find a way to get the disks offsite or you're not really that backed up. Just from user error, and power supply problems. The one thing that I do like about DVDs and other WORM mediums is that you just have to worry about the media getting corrupted, not user error after it's been successfully written.

Re:External and Online

[statusbar]

Interestingly, amazon web services now has a import/export service, where they will accept your USB drive via courier and import it into their "Simple Storage Service" aka S3.

      http://aws.amazon.com/importexport/

--jeffk++

Re:External and Online

[WhatAmIDoingHere]

He's not talking about building a gaming rig, he's talking about throwing together something to store data.

Re:External and Online

[antijava]

I have a pair of external FW800 drives in a software RAID-1 configuration. Every month I bring one of them home from the office, remirror, break the mirror again and take it back to the office.

If I have a total system loss (fire, theft, etc.) I'm worst case 1 month out of date.

My very critical data that I can't tolerate a 1 month loss of, I use online backup (http://www.tarsnap.com)

Re:External and Online

[ProKras]

Your average slashdotter is not going to get a cheapy $550 computer. Your average Joe maybe, and then they will complain to us that their computer is so slow...

Your average slashdotter probably has enough old hardware lying around to build a file server that's good enough, or knows where to scrounge up the hardware for nil. The only thing to buy would be the hard drives. You can find a 500 GB EIDE hard drive for around $60. Put 4 of those in and you've got plenty of storage. Or if you want SATA, I've seen SATA cards for under $10.

Re:External and Online

[jhol13]

Switching off-site backups every week is an unnecessary hassle

Depends. I do that with two (500 G) USB hard disks, I take one to work. Not every week, but still far easier than pushing it over 1Mbit/s ADSL.

Re:External and Online

[KillerBob]

I got mine here:

http://pccyber.com/?v=product&i=MB-VIA-PC2500E

But they happen to be a vendor that's just up the street from my home. And I can assure you it works with Linux. It's not 64-bit, but it works fine with 32-bit Linux. :)

Pretty much *any* Via C7-based mobo will work with Linux. The C7 is an x86 chip, and supports the full x86 instruction set, as well as some very nice enhancements and optimizations for things like crypto that other chips don't have.

Re:External and Online

[Eivind]

Online backup is great, but only if your bandwith is reasonable in relation to your backup-needs.

I've got 10Mbps symetrical, and currently backup around 50GB. (around half/half pictures and music, ignorable amounts of code and documents)
10Mbps in practice tends to work out to slightly above 1MB/s, around 4GB/hour.

So a full backup takes 12 hours, and a restore would too. I use rsync for backup though, so a normal (nightly) backup only transfers what has been changed, that is rarely more than a small fraction of the data, thus 9 times out of 10 the nigthly backup is done in less than half an hour, which is more than fast enough to be practical.

I would say, if your bandwith is sufficient to upload the data-amount you want to backup in less than 24 hours, then it's practical, otherwise not.
It's *really* hassle-free though, zero-admin normally. And it brings additional advantages, like getting access to my complete data anywhere with ssh-client.

Re:External and Online

[klashn]

SSDs are more abundant these days. At least they're shockproof ;-) I had an external drive that I used for backup, and I forgot it was connected to my laptop when I yanked my laptop away from the desk :-(

Re:External and Online

[Anonymous Coward]

CrashPlan (Free) provides software to backup locally to disk and offsite to another computer for free.
Online storage is extra - but given costs of a 1TB drive these days.. and given it fully automates the process of offsite - no longer a hassle.

http://www.crashplan.com

Re:External and Online

[mrboyd]

Banks, some gov agency and military aren't that cool with you bringing external storage to and from home on a regular basis. :)

Re:External and Online

[cgenman]

Your average slashdotter really ought to have five or six usable motherboards, chipsets, and cases sitting around in their work room. Maybe a dual-chip P4 server, a couple of old laptops with broken CD drives and dead batteries, a project motherboard from when Micro ITX was hot, and that one computer that you were "definitely going to fix some day." Add into that spare, perfectly geometrically distributed, and completely questionable 160, 80, 40, 20, 8, and 4 GB hard drives.

Really, if we're talking about an idealized slashdot user who doesn't already have a spare server sitting around, you're still looking at building from scratch. A $60 motherboard + chip combo, a $20 PSU, $200 in hard drives, a $15 metal case, a $5 novelty case badge, and BSD = a perfectly serviceable $300 file dump.

I wouldn't be surprised if the average Slashdotter has a slower computer than the average Joe. The average Joe doesn't know when to upgrade his computer, and thinks he needs to do it every two years to get the latest version of Word and Outlook. Your run-of-the-mill computer professional, on the other hand, sees their computer as a tool to get certain things done. Tools have weight, especially when they're still working fine. And as such, it might take a lot longer to switch out that working component... especially once you know it by heart.

Re:External and Online

[RockDoctor]

Your average slashdotter is not going to get a cheapy $550 computer.

Your average slashdotter is going to understand the concept of "fit for purpose".

I wrote the above. Then I thought "please dear FSM, let it be true that the average slashdotter is going to understand the concept of "fit for purpose" ; now that I pose the question, I'm less confident of the answer.

Re:External and Online

[goosesensor]

Okay, who doesn't have "really important data"? External backup is a must in case of physical disaster or theft. And I donno about anyone else, but backing up an absolute minimum of a few gigs to an online server just isn't practical.

Re:External and Online

[Morty]

Depending on where you work, bringing media to and from work might be a problem. Even if you work someplace where this is not an issue, you might have problems getting access to your media if you are laid off or fired. Not to mention potential legal issues with ownership of your personal data if it's physically located at work. [Might want to make sure the media is clearly labelled "personal property of YOURNAME".]

Also, if you live close enough to work, a natural disaster (hurricane or earthquake) could cause problems at both places.

Re:External and Online

[Anonymous Coward]

Way to miss the point entirely and still get modded insightful!

Re:External and Online

[laejoh]

I love your username!

i don't have hard drives. i just keep 30 chinese teenagers in my basement and force them to memorize numbers ...

Re:External and Online

[Anonymous Coward]

Actually most online backup software would find a whole lot of duplicated blocks of data in that

It's (already efficently compressed) videos and photos. Unless there is lots of high level redundancy *between* files (e.g. multiple copies of same video in the same format with a few edits) there's not going to be much duplication.

Re:External and Online

[Anonymous Coward]

Yup, that is exactly what I bought last week. The JetWay atom boards have the option for daughter boards, such as a 4 port SATA II board.

This board . .

http://linitx.com/viewproduct.php?prodid=12295

with this addon

http://linitx.com/viewproduct.php?prodid=12427

2gb Ram and Ubuntu server. Just debating over drives at the moment. 4 or 6 x 750Gb seems to be the way to go at the moment.

Admitedly I want more than a backup, my plan is to use PXE and PartImage to backup/image freinds and family's laptops. Im also interested if it will run myth backend for playback of vidoes.

Re:External and Online

[Ilgaz]

What about using a gigabit or even 100mbit drive at your neighbor granting him/her same right explaining the benefits?

Every backup tool has encryption support too.

Re:External and Online

[Z4mp4n0]

Why?
You can buy a new mac, boot from the OS X disk and choose "Recover From Time Machine".

Voila!

Re:External and Online

[Anonymous Coward]

take a while? are you kidding?
2TB = 2048GB
100GB per month means roughly 20.5 months of uploading to get all of the data.
and that's if nothing changes. there's all the incremental changes that will need to be added as well, so it's more like 2 years straight of uploading.

yeah. good luck with that.

Re:External and Online

[jbarr]

This really is a good solution. Just share the drives and you are good to go.

If you are using Windows as the OS, for even more "protection", just use something like GoodSync to automatically backup one drive to the other, and you get instant redundancy. Sure, you need to double your disk space, but that goes with the territory.

Re:External and Online

[AmazingChicken]

Your average slashdotter is not going to get a cheapy $550 computer. Your average Joe maybe, [snip]

Between Darkness404 and KillerBob (and my own scots ethic) I have to disagree. I think the average slashdotter is going to purchase only the equipment he or she can't scam from the office, and when they do the niceities of convenience is not going to attract them over lower price. I snipped the broad generalization about complaints because folk will complain about everything. Doesn't matter the price; consider the high number of complaints which followed the first-year sales of Hummers. The vehicles, I mean.

Re:External and Online

[monkeySauce]

No, I think $550 sounds about right. When I read "get a PC" on slashdot, I assume that means go to newegg and buy the parts, not go to best buy bent over with your pants down.

Skimp on everything but the drives and you'll probably even come in under budget and have much more processing capability than you need for a backup server.

Re:External and Online

[Anonymous Coward]

Unless you work for a fast-food type company, I'd imagine most places are cool about this.

Places that handle classified and sensitive data won't be. Can you spell HIPAA?

Re:External and Online

[Capt.DrumkenBum]

I don't know about the 'average' Slashdotter, if there is such an animal. But I built a new and improved gaming machine, and turned my old gaming rig into my backup server.

Re:External and Online

[Capt.DrumkenBum]

A bank safety deposit box is a cheep and simple solution to all of the problems listed above.

Re:External and Online

[Anonymous Coward]

For that, you could get a whole nother PC w/ 2 ea 1TB drives.

LOL @ "a whole nother"

That sounds funny to me when people say it, I about died laughing to see someone type it.

What exactly is this "nother" anyway? =P

Re:External and Online

[adpads]

I don't know who your average slashdotter is, but I guess compared to most of you all I'm a regular joe, and my backup method of choice is to have a whole second computer. Who doesn't have one kicking around? And if not, even so the price of a second drive is what it is, and then you can go to the pawn shop and choose between a USB bridge for it, or some ancient, probably hot laptop with a standard drive bay in it?

Re:External and Online

[hebertpa]

thats not true. I am sure that there are plenty of slashdotters out there that have one , two, maybe even three $600 computers. its not the price of the computer but what your using it for.

Re:External and Online

[MeatBag PussRocket]

these sorts of places usually have designated areas for personal belongings such as lockers, not always, but often.

Re:External and Online

[Hurricane78]

Online? That is the most stupid thing I heard all year!

It's the SUV of backup strategies. It has nothing you want, but is makes you look oh so cool... oh wait, it makes you look like an idiot.

It's slow, it is the most insecure thing you can do, data could get stolen, deleted, the company can die, it can cancel your account, oh and did I say that it is freaking *slow*?
How slow? Well, your home DSL line might have a realistic upload speed of 50 kB per second. And assuming your disk is 1 TB big, then it would take:

(1TB / 50kB)/60/60/24 = 231.48148... DAYS! That's nearly 8 months!

Good luck with that!

Re:External and Online

[Anonymous Coward]

No company I ever worked for ever presumed to look in my desk drawers.

say what?

[Hognoxious]

But the one thing that's remained constant is hard drives are far too large to backup via conventional means.

Maybe it's unconventional to use, I dunno, another hard drive?

Re:say what?

[Anonymous Coward]

dude. Seriously?

  Not only did you not read the article. You didn't read the summary.

Re:say what?

[Dachannien]

I think the OP's post arose from a misunderstanding of what "RAID is not backup" means.

The adage isn't an admonition not to use hard drives as a means of backing up data. Rather, it is concerned with the fact that any change to your data is committed to each duplicate volume in a RAID, so if you delete an important file, for example, it's just as gone as if you weren't running a RAID.

That's completely different from mirroring your drive onto an external hard drive and putting it on a shelf somewhere. If you delete a file on your live system, you can restore from that backup.

Re:say what?

[Hognoxious]

Actually I read the summary and decided it was stupid. Sort of like, "I want to make a ham sandwich. Conventionally these contain bread and ham, but I'm an idiot so I want to make it from dog hair and epoxy resin".

Re:say what?

[Anonymous Coward]

Your summary of the summary is fucking awesome.

Re:say what?

[Anonymous Coward]

Protip:
Use hard drives from old boxes. Hard drives typically fail with in the first three months or after about seven years of use. If you are anything like me you have at least a couple of 40 and 60 gig ide drives around that have served for a few years but are small and slow by todays standards.

These drives are actually in their prime in terms of reliability. Buy one of those USB devices that you can slot both ide and sata drives into for writing your backups. Store the drives in a reasonably dry place and they will be readable for a long long time.

Re:say what?

[Anonymous Coward]

Agreed. Unfortunaly there is no +1 awesome.

Re:say what?

[Machtyn]

Or running an external RAID 1 setup. If one external backup drive fails, the other will have all the data. But, certainly, if you are going to have two drives of the same size, then alternating backups to each drive is the best. If you are really concerned, get 4 drives. Have two sets of RAID 1 drives, alternate each set every week. Don't forget to test your backups and your restoration procedure.

Re:say what?

[Phroggy]

If you've got four hard drives, I would use two in a RAID-1 for the live system, and rotate the other two, using rsync to copy the changes. Don't mirror your backups; it's a waste (compared to the benefits of having twice as many snapshots available).

Re:say what?

[Akzo]

Not if you rotate 3+ drives in a RAID 1 setup. That way one drive will always be an older exact replica of the other two. Of course this is still vulnerable to being destroyed unless you store it offsite, but at least you don't have to worry about a hard drive failing.

Re:say what?

[Anonymous Coward]

I backup to 2 identical TB drives, then put them in a firesafe. (Primarily family pics/videos/projects)

Re:say what?

[SharpFang]

The adage isn't an admonition not to use

stop making up words.

Re:say what?

[Anonymous Coward]

Actually I read the summary and decided it was stupid. Sort of like, "I want to make a ham sandwich. Conventionally these contain bread and ham, but I'm an idiot so I want to make it from dog hair and epoxy resin".

I don't care WHO you are, that is some funny stuff right there.

Re:say what?

[crispytwo]

I'm completely confused with the question too.

To me, get an external drive to do backups. Also, make sure it is a RAID -- 0 or 5 or better 6. If you can afford it, get 2 just like this and rotate them daily, weekly or however often you do backups. The more history you have the better.

I use rsync on 2 Synology 407s each with 3 Terrabytes storage, RAID 5, rotated weekly, backups daily.

The justification is that we can afford to lose a maximum of a week worth of work, i.e. if there's a fire and somebody axes the boxes... more than that is way too costly... but the reality is that it is less than 2 days of work, and probably less, unless everyone's machines are on fire and are axed just right.

How much are you willing to lose?

Re:say what?

[Jonathan A]

Actually I read the summary and decided it was stupid. Sort of like, "I want to make a ham sandwich. Conventionally these contain bread and ham, but I'm an idiot so I want to make it from dog hair and epoxy resin".

I think the poster posed a perfectly valid question. The way I read the summary was (using your analogy) more like,

I want to make a ham sandwich. The last time I checked, these were made with Wonder bread and Oscar Meyer ham. Since then, has anyone discovered a better recipe?

Of course, while your analogy is somewhat amusing, it is not necessarily the most appropriate one since computer tech changes significantly faster than ham sandwich tech. :)

Re:say what?

[jnork]

RAID is instant protection against data loss due to hard drive failure. That's about it.

Backups are longer-term protection against data loss by any means.

RAID can actually be part of an effective backup strategy. If you keep mirrored drives, periodically swap a blank drive in for one of your mirrors. Mark the mirror and store it. The system will replicate the data to the new drive. Presto, instant backup! Sort of. Doesn't require special backup software, though of course you have to invest in the RAID system. But you also get a RAID system out of it.

Re:say what?

[Anonymous Coward]

Sure it is.

But after 25 years in this business, I can say with calm assurance that "spinning magnetic storage" is not "backup"... even if it isn't always spinning.

Sure, tape backup is more expensive than disks or optical, but it's also *backup*.

LTO4 is still really pricey, but it's 1.6TB per cartridge. LTO1 and 2 are quite a bit cheaper now, especially on the used market. If you can't afford LTO1 or 2, for your business, then you're not really in business; you're playing.

And I've recovered data off backup tapes that were 10 years old with no problems at all.

Course, they were *tar* tapes; not some stupid proprietary Windows format that's been obsolete for 9 years...

Re:say what?

[PiSkyHi]

hmmmmmmm.... resin....

Re:say what?

[cas2000]

If you've got four hard drives, I would use two in a RAID-1 for the live system, and rotate the other two, using rsync to copy the changes. Don't mirror your backups; it's a waste (compared to the benefits of having twice as many snapshots available).

that's similar to what i've been doing on my home network since my tape drive died (and disk drives are so much bigger than affordable tape drives that it's not worth the cost of replacing it). my main machine (combined workstation/server), and all other machines (workstations, mythbox, etc) all have RAID-1 as their boot drives. the server also has two hot-swappable drives which are used for backup, labelled backup1 and backup2. all the machines on the network are backed up by rsync every 4 hours to backup1 which is, in turn, rsynced up once per week to backup2. they are umounted when not in use, and can be hot-swapped easily.

it's kind of ugly, but it works OK....I'm not entirely convinced that hard disks make a GOOD backup medium (as opposed to a servicable hack), but this is a hell of a lot cheaper than buying an LTO-3 or LTO-4 tape drive plus auto-changer robot.

it has occurred to me that it might be worthwhile to add a backup3 drive, so i can have 4-hourly, daily, and weekly backups. and i probably should use rdiff-backup rather than plain rsync as that would give me versioned backups. i'm planning the next reorganisation/upgrade of my disk/storage space now, so i'll probably make those changes at the same time.

Re:say what?

[dwater]

Why not just use RAID1 as a method for copying the data - it happens continuously, so it takes no time to make the copy. When you're ready to take the backup, shut the volume down/unmount/whever you need to do to make sure the volume is 'good', then fail that drive and switch it for another drive.

You can keep sets of drives just like you would set of tapes; and take them off-site/etc.

Re:say what?

[jollyreaper]

The adage isn't an admonition not to use hard drives as a means of backing up data. Rather, it is concerned with the fact that any change to your data is committed to each duplicate volume in a RAID, so if you delete an important file, for example, it's just as gone as if you weren't running a RAID.

That's completely different from mirroring your drive onto an external hard drive and putting it on a shelf somewhere. If you delete a file on your live system, you can restore from that backup.

OP here. The way I meant it was "You don't stick your files on a RAID and consider that a backup." Theoretically, a RAID in your office and a RAID across town that are syched would be good backup. But in a single RAID, you could have two drive failures at once, a power surge, card failure, gremlins, whatever. It appears many people use a Drobo as a file server and so the question becomes "How do you backup the Drobo?" Because this goes back to point 1, RAID is not a backup.

Raid 1 + version control

[oldhack]

Not the same as external backup, but it provides redundancy against a single drive failure and provides history. Otherwise, run backup overnight every now and then to an external drive and store it away.

Re:Raid 1 + version control

[Anonymous Coward]

Excellent. To get free/easy versioning of backups with differential and gz compression, use rdiff-backup to that external drive.

A 20GB VM backup (Full-span) turns into about 4GB of rdiff-backup data plus about 1GB as 30days of daily differential backups are rotated for a total of 5GB. I've recovered from this backup 2 times in the last 8 months. Works perfectly.

Re:Raid 1 + version control

[arndawg]

Not a bad suggestion. But you are vulnerable to filesystem-corruption or errors in the version control system. But it seam like a good enough setup for home-users actually (for media backup atleast). If you got really important pictures and documents they are often small enough to just upload to an online service or even burn to dvd.

Network Backups

[Anonymous Coward]

You should try Mozy - free for 2GB, $5/month for unlimited. I've been using them for a year and its seamless.
https://mozy.com/?code=WAQ9DM

Re:Network Backups

[anotheregomaniac]

Carbonite works well to. I've got over 100GB backed up with them for $50/year. Comes out to about $0.04/GB/Month.

Re:Network Backups

[1310nm]

They have some funny testimonials:

"I spent one week and $1,500 recovering critical files after my son crashed my computer playing video games. I learned my lesson the hard way and now back up everything with Mozy. If my hard drive crashes again, Iâ(TM)m ready."

I use...

[haifastudent]

... Amazon S3 mounted on FUSE. USD $0.17 per gigabyte to store offsite, and with FUSE I can browse it like it's local. I am very happy, and saving money too.

Re:I use...

[Anonymous Coward]

Are you joking? S3 is perhaps the most overpriced way to backup data.

You're paying at least $0.15/GB/month for the space, and then paying $0.10/GB transferred in and $0.17/GB transferred out.

So if you were to use 1TB of storage over 5 years filling it perhaps 3 times over that period and reading it 10x, it would cost $1800 for the space alone, $300 bw in, $1700 bw out, for a total price of $3800.

Meanwhile, you can get 1TB hard drives for $80 everyday (you could almost buy 50 of them for the price of your online service). I'd love to hear how you can twist the math around so badly that it looks like you're actually saving money! Ever considered a career in politics?

Re:I use...

[Anonymous Coward]

Amazon S3 would cost me over $300/month. I don't know where you got 0.17 pricing. but that does not include sending or retreiving the data. Browsing also has costs...

http://aws.amazon.com/s3/#pricing

Re:I use...

[rvw]

Mozy is much cheaper, although S3 can be cheaper when you want to backup multiple machines.

Re:I use...

[nametaken]

Is there any particular reason you had to be an ass here?

Re:I use...

[labnet]

Have you considered a career in politics?, as your figures are totally unrealistic for home use.

We use S3 for our critical stuff like family photos, videos, docs. Thats around 5GB.
(75c/month)
Music and DVD's that take up most of the file server space don't go to S3 but are on a mirrored server with very occasional offsite backup.

Re:I use...

[farnsworth]

Are you joking? S3 is perhaps the most overpriced way to backup data.

If you are generating 3/5 of a terabyte of important data anually, ~$900 per year for even a 99.99% chance of a backup being available is cheap. I suspect that s3, in practice, provides greater than 99.99% success rates. Pairing s3 with a local fw drive with time machine or rsync is essentially 100% availability for the data on your computer.

Re:I use...

[Larryish]

Well, you are what you eat.

Just call him Mr. Rimjob

Re:I use...

[bkpark]

Have you considered a career in politics?, as your figures are totally unrealistic for home use.

Don't encourage him! We already have too many terrible politicians far, far removed from reality as it is.

Re:I use...

[Anonymous Coward]

Comparing what you get from S3 to a raw 1Tb hard drive is so utterly ridiculous that it is extremely difficult to take you seriously. You are comparing apples to oranges here and are surprised.

Re:I use...

[Anonymous Coward]

Use Rackspace instead of Amazon S3... way cheaper ($0.15 / GB via JungleDisk.com with no up/down fees).

Re:I use...

[Anonymous Coward]

Price out power consumption for an always-on 1TB RAID for five years before dismissing S3. Not $3800 but it'll make a dent. Don't forget to put a price on your own time and patience, too.

It doesn't make sense for everything, but for irreplaceable data (home pics and vids, mostly) I'm quite happy keeping copies in the cloud. If you only have tens of GB of irreplaceable data, S3 makes way more sense than rolling your own. An additional advantage is being able to stick home movies in a public bucket and send URLs around to the grandparents rather than DVDs.

Re:I use...

[Anonymous Coward]

I'll bite.

With that $80 hard drive, you're going to need:

1. a computer ($1000)
2. at least one redundant drive (another $80)
3. electricity for 5 years (10 cents per kWh @ 120 W * 5 years = $526 )
4. If off-site, you'll need an Internet connection. Just say Cable/DSL @ $50/mo = $3000

So we're at $4606 for a home built remote backup solution and that's if none of the hardware dies before 5 years is up. You'd probably be better off using a colocation center or you could save the headache of worrying about hardware altogether and use S3.

Re:I use...

[crunzh]

Umm, maybe is is considered cheap compared to other offsite solution.

Re:I use...

[nmg196]

Then you must be rich. I did some cost comparisions and found S3 to be about the most expensive way of getting data stored online that I could find. Saving money compared to what?

Re:I use...

[jabuzz]

Wrong, you are going to need

  1. A cheap single disk NAS enclosure ($100 tops)
  2. It does not need to be redundant, its a backup your redundancy is your primary copy. ($0)
  3. It can spin down the disk for 90% of the time. Power consumption averaged over a day will be say 5W (on your figures $22)
  4. I have a family member with Cable/DSL and no capacity monitoring 00:00 to 08:00 ($0)

Total cost ~ $200. I can replace it with brand new hardware every year and still save $2800, and probably up the capacity at the same time.

Amazon S3 is hugely expensive, there are far cheaper options for most people.

Re:I use...

[Anonymous Coward]

You're not including the cost of your own time though - buying HDDs, the RAID setup, the backup setup and testing, rotating the disks, taking an offsite copy etc. Plus as someone else mentioned in a few years you'll get new HDD tech and you'll have 50 1TB drives that don't plug in to anything so you'll have to buy new drives, copy it all over, buy new RAID equip etc.

Re:I use...

[aclarke]

I use Amazon S3 via JungleDisk for my off-site backups, and I'm very happy with the solution. For now. I pay about $45/month, so yes it is expensive. On the other hand, I have > 100GB backed up there. I have done the cost comparison with several other services, but for various reasons JungleDisk/S3 continues to be the best online backup service I have found for my particular needs.

I share my DSL with my neighbour via a wifi linkup. I do plan on putting an external drive at his house for my off-site backup. However, in 6 months or so that I've been thinking of it, I just haven't gotten around to it. So I ask you which is cheaper: an offsite backup plan that is in place and available should I need it, or a cheaper solution that I haven't set up yet so if I need it, it won't be available?

Simplicity needs to be factored into the cost of a backup solution. If it's too complicated for someone to set up and use, then it won't be there and up-to-date when needed and is useless. I can build a computer as easily as the next person on here, but I also have my business to run, and would rather spend my spare time with my family. Having a backup is much much cheaper than not having one, if it's needed.

This is also why Time Machine is so awesome. Basically, It Just Works. This is what powers my on-site backups.

Re:I use...

[aclarke]

Mozy is theoretically cheaper, but in actual practice I found it unusable. As I mentioned in another post, I have well over 100GB backed up using JungleDisk/S3. I thought I'd try Mozy's "unlimited" plan. First of all, Mozy's software would take up well over 1GB of RAM and spike a core for hours trying to process my upload. Even on an 8 core Mac Pro with 14GB of RAM that starts to get annoying after a few weeks.

I say "a few weeks" because Mozy would start uploading my data, usually using less than my upload speed, and then would invariably crash out or stop after a few hours or a day or so of uploading. Then the process would start all over again. It was because of this that after a couple months of trying, I realized that Mozy would never ever actually back up my data. Mozy may appear to be cheaper, but that's because, for me at least, it just wouldn't work.

I'll stick for now with JungleDisk/S3, because they have incentive to make their service work for heavy users. I pay for what I use, and they provide me with the service I need.

Re:I use...

[Anonymous Coward]

Sorry for pointing out the obvious, I'll try not to do it again, Mr. Oversensitive OOPS I did it again.

Re:I use...

[haifastudent]

I'd love to hear how you can twist the math around so badly that it looks like you're actually saving money! Ever considered a career in politics?

I'm not using a terabyte. Assuming 50 GB (I'm at about half that):
[(0.17 * 12 * 5) + 0.10 ] $/GB * 50 GB = $17.24

How twisted is that?

Re:I use...

[Anonymous Coward]

There is something to be said for offsite highly redundant backups. Not to mention, if you're pulling a large file down from S3, you can get really fast transfer rates using bittorrent (since S3 is distributed).

Backup?

[Anonymous Coward]

What do you mean Take a Backup?
My Hard disk is nowhere near full.
Hmmmm, I must download some more Music.

Ok, Only joking.
Seriously, how many Windows users know that lots of important stuff is in hidden direcetories that just copying your files is not going to work?

BD-R DL discs aren't too bad

[Brian Stretch]

Almost 50GB per disc and brand name blanks aren't too expensive if you know where to look. (Hey Newegg: surely y'all could save us some nuisance if you'd import a shipping container or two of blanks direct from Japan...) Nero Linux supports Blu-ray drives. RAID1 for primary storage with BD-R DL backup, with the backups ideally stored securely off-site should be sufficiently paranoid for most home users though Blu-ray is too new to have real-world long-term integrity statistics.

Remote backup to a rented dedicated server is also a possibility though not terribly practical in America due to certain monopoly carriers (<cough>AT&T</cough>) being too cheap to build FTTH, at least until they run out of duct tape and bailing wire to keep their WWII-era copper plant patched together, and even then.

Re:BD-R DL discs aren't too bad

[Anonymous Coward]

Almost 50GB per disc and brand name blanks aren't too expensive if you know where to look

That's over 10x what I'd be willing to spend on storage, and about 20x more expensive than using hard disks. So I think the answer to the question in the summary of whether "there's a better way" than backing up to hard disks is still solidly, "no."

Re:BD-R DL discs aren't too bad

[Anonymous Coward]

So if I buy $250 BD writers for all my machines, I can pay "only" almost 3x as much than hard drives cost per GB? And that's assuming I actually import the discs myself too.

Then I only have to find backup software that's compatible with my BD writers, that will use every disc 'till the last byte (else you're wasting space, and the cost goes up even further).

Backing up my 10TB array (pretty big, I know) means at least one $250 writer, then 200 discs to swap by hand (tedious, long, boring) which will also cost $2280.

Or I can get ten 1TB drives for $800 total. And I can actually reuse them, just like I could with tapes before. Your BD discs aren't rewritable -- use them once, then you're done. If I start to get rid of older backups to make place for newer backups, then your solution's price skyrockets even further (keep buying more blanks). More drives means more $ again.

Hard drives are far better all-around, at 1/3 of the cost or less, and can be plugged in any old computer, no $250 drive required, works with pretty much any software (and OS) you could imagine, much faster backup/restore speeds too, can be used with a variety of great filesystems (don't have much of a choice for BD discs) and so on. I could add decent SATA port multipliers & enclosures to my solution (10+ drives on at once) and still be lots cheaper in the end.

Re:BD-R DL discs aren't too bad

[Anonymous Coward]

oh yeah, until lightning takes out your 10TB array AND your 10TB backup and you start cursing spinning disks as backup solutions.

Re:BD-R DL discs aren't too bad

[Anonymous Coward]

The guy could build 10 TB array, store offsite, then pay for bandwidth for incrementals and it would STILL run out cheaper than BD.

Differential + hard drive - online

[junglebeast]

All of the online backup strategies are a joke. Due to bandwidth restrictions, it would take years just to make a backup of a typical user's hard drive, and they don't offer enough space (seriously). The cheapest form of medium currently is hard drives, so my current backup system is to have 2 equally sized 500 GB drives and I use Acronis on a schedule to do a differential backup of one drive to the other once a week during early morning hours. If the differentials start to get too large, I'll do a new full backup and start doing differentials from there again. I haven't found any backup solution that is "totally" automatic in this regard, but since it only requires manual intervention once every several months it's not a huge deal.

Re:Differential + hard drive - online

[Skylinux]

Yes Acronis is great, I used to install it for our clients all the time.
It was configured to backup every time when the computer was turned off (Startmenu => Shutdown). This was very reliable and I don't recall any issues with this setup. I do recall restoring entire systems from backups a few times.

1) Install new drive
2) Boot with recovery CD, select restore ... wait a few minutes
3) Done

Acronis is a nice little program, well worth the money.

Re:Differential + hard drive - online

[rainwalker]

Really? I don't think you've looked at this very carefully...personally I use Mozy, it's a couple bucks a month, the initial upload took a week or so, but it was all backgrounded and I never even noticed (yes, you can turn your computer off, etc.). Daily incremental backups take just a few seconds. Retrieval is via downloading, if you just want a few files, or for some money ($50? I think?) they'll overnight you a couple of DVD's with your whole backup on it. So, it's cheap, requires absolutely no thinking on my part, is fire/meteor proof, and has unlimited storage. The choice was obvious, from my point of view.

Re:Differential + hard drive - online

[Blakey Rat]

Mozy is dirt cheap and has unlimited storage. I wouldn't call it a "joke", unless you have more than, say, a hundred gigs to back up.

Re:Differential + hard drive - online

[Anonymous Coward]

All of the online backup strategies are a joke. Due to bandwidth restrictions, it would take years just to make a backup of a typical user's hard drive, and they don't offer enough space (seriously).

S3 lets you mail in your data (HDD, tape, etc) and they'll upload it to the bucket you specify. From then on, all you would need to do is upload diffs.

Re:Differential + hard drive - online

[prophecyvi]

How did you get modded up? Mozy Home works perfectly well at a dirt-cheap rate. The statement "all of the online backup strategies are a joke" is a clear indication that you haven't researched this topic even a little bit.

Re:Differential + hard drive - online

[macintard]

All of the online backup strategies are a joke. Due to bandwidth restrictions, it would take years just to make a backup of a typical user's hard drive, and they don't offer enough space (seriously). The cheapest form of medium currently is hard drives, so my current backup system is to have 2 equally sized 500 GB drives and I use Acronis on a schedule to do a differential backup of one drive to the other once a week during early morning hours. If the differentials start to get too large, I'll do a new full backup and start doing differentials from there again. I haven't found any backup solution that is "totally" automatic in this regard, but since it only requires manual intervention once every several months it's not a huge deal.

A joke, huh? Have you heard of seeding, compression, or de-duplication?

Re:Differential + hard drive - online

[PNutts]

All of the online backup strategies are a joke...and they don't offer enough space (seriously).

Carbonite offers unlimited storage which is enough for me. I'm not, however, familiar with your requirements.

Re:Differential + hard drive - online

[mcrbids]

I use Mozy, it's a couple bucks a month, the initial upload took a week or so, but it was all backgrounded and I never even noticed (yes, you can turn your computer off, etc.).

That's the point that most people miss - it's not going to download all your data every time it does a backup, just the changed files. And that's a much lower number than many people believe, and uses much less bandwidth than most anybody would expect, especially when you factor in compression!

Even on busy servers with over 250,000 file transfer operations per day, I see backups done remotely at 1 Mb taking just a few hours using rsync - I can't see Mozy being any hassle at all.

Re:Differential + hard drive - online

[dhardisty]

Agreed. It's also easy to use.

Re:Differential + hard drive - online

[junglebeast]

I have about 1 TB of data, that may be a little above average but not much. Now look at Mozy -- $3.95 + $0.5/GB per month. That means there's going to be a minimum charge of about $500 just to make the initial backup, on top of the regular monthly charge. After the original backup is made, probably looking at 100-200 GB/mo, in deltas, due to things like movies being downloaded, then deleted, etc, which would be another additional $75/mo. None of that really matters when you consider that the upload speed is about 768Kbps, which may be subject to further throttling. I'll let you calculate how many years it will take just to make the initial backup. Oh and don't even think about doing anything like browsing the internet while you wait all those years, because your download speed will also be crippled due to the acks and whatnot you cannot quickly respond to. In other words, I stand by my original claim of mozy backup being a joke.

Re:Differential + hard drive - online

[goofy183]

Not sure where you are getting your Mozy prices but for home use it is $5/month with unlimited storage space. I currently have 3 home accounts, one of which has over 1TB of data and I'm paying a total of $15/month for all three.

Re:Differential + hard drive - online

[lancejjj]

All of the online backup strategies are a joke. Due to bandwidth restrictions, it would take years just to make a backup of a typical user's hard drive, and they don't offer enough space (seriously).

This is, in a word, wrong.

I have been using an online backup service for my brother's computer. I used to run rsync via cron and retain deleted files, but it was a pain in my butt, so last year I just had him spend the $60/year for an on-line backup service.

This past Thursday his PC's drive died - completely unrecoverable, unrepairable, toast.

The backup service in use had his 200 GB of user data. We popped in a new drive, installed the OS, downloaded the back-up files over the weekend, and now he's back in business. None of his documents were lost.

Was bandwidth too low to recover his data? No. Did it take years to backup and/or restore? No, but it did take a couple days to download it all. Was there space limitations on the service? No.

Did it save all his family photos, documents, email archive and music library? Yes.

Re:Differential + hard drive - online

[dixonpete]

My experience with online back-up has been good as well. Considering most office people only generate at most a GB or two of unique data per month it's economical. Ghost a back-up drive for the OS and programs and you're set.

Re:Differential + hard drive - online

[lsco]

Mozy sounds good until you consider that the amount of data being backed up could be pushing 750Gb, at least in my situation it is, and that would cost me $375 per month data storage. Even 100Gb @ 0.50c per month is a decent outlay. How many extra hard drives could I buy for that? /rhetorical

Re:Differential + hard drive - online

[prometheus123abc]

use rsync. It only uploads the changes between folders or directories, to use the least amount of bandwidth. You can also tell it to preserve user privs, etc. It was designed to be able to deal with an intermittent or broken connection. Anyway, why copy your ENTIRE drive? Doesn't make much sense to me. A great deal of it should be replaceable.

Re:Differential + hard drive - online

[goofy183]

Except mozy only charges $5/month for unlimited storage for a personal account so it should only cost you $60/year for as much as you want.

Re:Differential + hard drive - online

[Anonymous Coward]

Mozy is fast because it relies on the idea that most people are backing up the same files. eg all your DLLs that come with Windows. There is nothing to upload, it just remembers that you have that file.

But it won't be fast if you are backing up photos, images, Adobe Illustrator files, original documents, conference proceedings, baby videos, and other unique files that noone else would have. If you generate a lot of documents, then its gonna take a while to upload.

Re:Differential + hard drive - online

[Anonymous Coward]

I use Mozy for the same reasons, I started the day I thought my "unbreakable" RAID1 array was lost with all the photos/vids of my kids. I'll never be disciplinate enough to follow a backup strategy, average Joe neither.

First backup was slow but I added files/folders to the backup set according to "my" priority: kids photos, kids video, source files, word excel... and each completed backup was a true relief.
Of course this solution is suitable for me because I don't have an upload cap. I now backup around 90 GB. For good measure (and download time...) I also have a "local backup" on another PC at home, just plain folder sync with no fancy feature. Crude but efficient.

I'm a happy Mozy user, but there are similar unlimited services around like Carbonite.
I don't know for others services, but Mozy store the last 10 versions of each files, saved me once too...

Anything can happed: your house can burn, you can be robbed... and no on-site solution will survive this.

Regards
Anonymous Nhiko

Re:Differential + hard drive - online

[Anonymous Coward]

Look at Mozy. I have a >80GB backup online that wirks well.
The initial backup session can take ages, in my situation I gradually added folders to back up, avoiding to upload 70GB from the beginning.

In the days of paper and books, backing up meand "pick up a specimen and lock it in a closet for 1000 years". Data was tied to the medium (the book, paper...). In the numeric days, data is free from the medium. Backing up media to preserve data is not the most appropriate. It's better to preserve data by copying it on recent (=compatible) medium.
Preservation by continuity.

I use a combination of 2 Time Machine disks (on 2 different location) as an immediate backup for my Mac Laptop, also backed up on Mozy.
Then, a 500GB HD serves a a primay Library. All old data is on this HD. It is replicated on a similar HD in my basement (concrete walls, fireproof doors, etc...) via RJ45.
Finally, This will be also backed up on Mozy.
It seems to me like a viable way of preserving the data for some time.

Re:Differential + hard drive - online

[Anonymous Coward]

Years? You need a better internet connection (or to be a bit more selective about what you backup). Mine's nothing special but I can still upload about 300Mb an hour. I use JungleDisk which sticks it all on Amazon S3. Hardly costs anything, have got probably 30Gb on there.

Re:Differential + hard drive - online

[Anonymous Coward]

Mozy.com offers unlimited storage for 4.95 a month..... bandwidth is a real problem. The initial backup is a long process after that it is pretty decent.

Re:Differential + hard drive - online

[QuantumRiff]

My inital Sync with the Amazon S3 storage cloud (I'm a very happy JungleDisk owner.. Only Linux app I've happily paid for so far) took about a week, to upload my 10GB of photos. Now, its almost nothing, a few docs here and there, maybe an hour or two when I dump a bunch of pictures from my memory card.. Cable modem's have limited upload, but most of your data is usually pretty static.

Re:Differential + hard drive - online

[Anonymous Coward]

is it still called a slashvertisement if it's in a comment?

Re:Differential + hard drive - online

[Anonymous Coward]

Not sure what service you think is a joke. I've used Carbonite for online storage and it works fine. My initial upload was 80GB of data and it took a day to get it uploaded. Now only changed files are backed up. Downloading has not been a problem either. I think having external storage attached to the computer as well as online is the best way to go now days. If the data is that important to you, it needs be saved in several places.

Re:Differential + hard drive - online

[Anonymous Coward]

Will you please quantify your online backup experience please?
How fast is your upload? How much do you pay for your internet access? How much data was actually backed up?

Re:Differential + hard drive - online

[dkuhn]

I found a product CrashPlan (run on Mac and Windows) and love it. It allows me to backup locally as well off site (i.e. family member or friend.). They also provide a service to host your backup but you pay per MB. Hosting at a family member or friend means I get off site backup and control without a monthly cost. If I have a serious problem and need my off site data I make a phone call to someone I know and trust and they ship me my data. I have now ~ 300GB of data with this backup solution. I tried Mozy for about 5 months. The price seems right, until you have a problem. They could not handle the amount of data I had (at the time ~ 200GB) as their software would just churn for 24 hours trying to determine what to even upload to the service.. When I had a hard drive failure it took me almost 4 months or constant back and forth until I got them to send me my data back. They had some "back end issue". Their support just kept stringing along until I think they finally fixed their problem. Their customer service sucked and it can be a huge pain to get your data if you have a major failure. For small amounts of data they might be fine, but for a lot of techies I don't think they can handle the volume of data.

Re:Differential + hard drive - online

[Anonymous Coward]

I thought Mozy looked pretty good until I tried it. After a few weeks of happiness Mozy downloaded an update (for my Mac) that trashed everything. Can't access the status page any more, and after dealing with tech support I find that all of the backups I made are lost! Sorry, this service is not reliable.

Re:Differential + hard drive - online

[jchernia]

I tried Mozy and the client kept hanging (Mac). I kept shrinking the size of my backup set, but it just didn't work. It may have been Comcast's fault given that it would get a few percent through and then stop. I worked through this for about two weeks before giving up. I will give Mozy credit for making good on their refund policy - they did not make it difficult.

Re:Differential + hard drive - online

[Anonymous Coward]

Yeah, Mozy's fine for personal stuff... but if you're running a small business, be prepared to pay $0.50 per gig per month. Upload that 750GB drive of client photos, and you're looking at paying some serious cash per month. You'd be better off buying a new 1 TB HD from NewEgg every couple of weeks, backing up to it, writing the date on it, then stashing it somewhere safe.

Re:Differential + hard drive - online

[Hurricane78]

Well, you either got a crazy fast connection, or no big data at all.

Let's calculate that for something realistic, assuming 50 kilobytes per second of upload speed, and a 1 TB disc:

(1 TB / 50 kB) /60/60/24 = 231.48148... DAYS!

For it to take a week, you would have to have a line of over 1.6 MB/s. That is, in DSL speak, a ~13 Mb/s *upload* speed. Which would usually result in a ~420 Mb/s download speed. (Extrapolated from my "official" 192 kb/s (up) for a 6000 kb/s (down) line.)
Or your disk would have to be only 30 GB big.
Or something in between.

Backups are a software problem

[Anonymous Coward]

Aside from the choice of backup medium (home users back up to flash if they have less than 8GB of data and to hard disks if they need more space), the big reason why people don't backup is that doing it right is ridiculously complicated, due to lack of software support. There's basically no way to produce a reliable full backup from within a running system. If people are supposed to make backups that they can rely on, this must change.

Re:Backups are a software problem

[dltaylor]

For a "FULL" backup, boot from external storage (FireWire, for example, on a Mac), or optical media, then gzip the entire disk image to a file on external storage. ALL of the state is saved. No log or event files are "in progress" on the main disk.

Booted from external storage, I think something like "ghost" would work similarly, but I don't have a copy to try.

If you can partition your data from the "operating system", then image just the OS when you make changes, such as after an update or program install. It will be smaller and faster to back up.

"user data" partitions can be backed up with the system running during idle time.

We're talking home computers, not 24/7 production servers.

Re:Backups are a software problem

[jonbryce]

I rsync my Mac across to a bootable firewire drive. The, if the hard drive fails, which it has unfortunately done so twice in the last year, I can boot from the firewire drive and rsync in the opposite direction to my internal drive.

I like my layered approach..

[Anrego]

I decided that I have three main "categories of data":

- easily replaceable: This is stuff that is fairly easy to replace.. for instance I have ripped a huge portion of my DVD collection (for my own use). If I lost this data, it would not be a tragedy .. just a pain in the ass.
- hard to replace: This is stuff that does exist "out there".. but would not be easy to replace. This includes old TV shows that you can't buy or if you can are very hard to find.
- irreplaceable: Self explanatory.. this is my documents, code, photos, etc that could not be replaced if lost

I keep everything besides OS files on a file server. Raid 6 (two parity stripes).. this is the first layer..
to me this is adequate to protect "easily replaceable" stuff (which in my case constitutes a huge chunk of file space).

I backup everything in the "hard to replace" and "irreplaceable" categories to a seperate (removable but stays in the system) hard disk (so far 1TB has been enough to hold all this data). I make a
secondary backup to a second removable drive and store this "off site". This secondary backup does not get updated very often.. which is the trade off I guess... but it provides a "last hope" if something
crazy ever happened.. like my house burning down.

Oh.. and backups are encrypted!

Re:I like my layered approach..

[fluffernutter]

I second this.. I do this as well.

I have three repositories:
- A 2TB internal raid 10 composed of four 1 Tb drives
- An external USB drive that acts as a repository for BackupPC
- A plain external USB drive at my parent's house

-Almost everything (ie. easly replacable) goes on my 2Tb RAID 10 except for the stuff that is on the solo primary drive, which is backed up with BackupPC on an external USB drive
-The hard to replace stuff goes on the RAID and has incremental backups via BackupPC
-The impossible to replace stuff gets all three. on the RAID, backup with BackupPC, rsync every four hours to the drive at my parent's house.

Re:I like my layered approach..

[Anonymous Coward]

that's a lot of work to keep your porn safe

Re:I like my layered approach..

[Like2Byte]

Your post reminded me of this discussion on "Security Now!".

original transcript: http://www.grc.com/sn/sn-198.htm

(emphasis mine)
[[snip]]
Steve: MacBreak Weekly, just as we were getting ready to do this. And he made a comment about - you were talking about ripping DVDs. And he said, yeah, you know, you can get a terabyte drive now for 90 bucks.

Leo: Exactly.

Steve: And I'm thinking, yeah, and that's what SpinRite costs. And he said so, you know, there's really no need to burn all those. Just rip them all onto that terabyte drive. And I'm thinking, yes, please do. Because, please.

Leo: Why is that, Steve?

Steve: Good. Put your whole movie collection on there because I will have your money. When that $89 terabyte drive craps out on you...

Leo: We're buying - are you saying people should buy fancier drives, or just this is inevitable?

Steve: Put all the crown jewels, put everything you have on hard disk.

Leo: Well, don't throw away the DVDs. Keep them. But it really is true that, if there's data on there, it's worth more than 89 bucks. It's not a question of buying another drive, it's a question of getting that data back.

Steve: Yes. I mean, people, for a while people were saying, well, gee, Steve, $89, that's pretty steep. And I'd say, yes, I understand. And then they'd say, well, we can buy a new drive for that. Yes, but it doesn't - it's not all of the data that you've got. It's not everything that's been installed in your system before. It's not, I mean, what's your time worth to, like, recreate everything from scratch? And in some cases these are irreplaceable. These are people's entire photo libraries that have never been backed up, never put somewhere else.
[[snip]]

The point is, Terabyte drives fail, too. Keep that in mind for your data retention policy. One might even be so inclined to purchase SpinRite ahead of time to validate the drive's integrity before being placed into use and occasionally validating the drive's integrity from time to time.

Re:I like my layered approach..

[Anonymous Coward]

your dad must be fascinated by that USB drive on his computer...the one that magically fills up with fresh porn all the time, no matter how much he deletes, the porn just keeps coming and coming, new stuff all the time

I hope you at least include some from his favorite categories, maybe some retro poon

Re:I like my layered approach..

[kabloom]

There's also a fourth category to consider: embarassing or legally problematic, for example, something like a diary. This is data that, though important, is less important than the damage it can cause if it gets out, and you may prefer to lose it over letting it out of your one primary computer). Though you seem to have that covered by encrypting your backups, there may be data that's too sensitive for that.

Re:I like my layered approach..

[Anrego]

The point is, Terabyte drives fail, too. Keep that in mind for your data retention policy. One might even be so inclined to purchase SpinRite ahead of time to validate the drive's integrity before being placed into use and occasionally validating the drive's integrity from time to time.

Indeed, which is why I use 2 of them. Obviously this isn't fool proof.. but I figure it's reasonable enough.

In my approach.. if raid fails.. I can likely get a very recent backup of the hard to replace and irreplaceable stuff from the first (always attached) 1TB drive. If that fails (which as you said, can happen), I have one more change.. the off-site drive.. which is probably a little out of date.. .. and if that fails.. then I just go cry in a corner...

Re:I like my layered approach..

[timeOday]

I backup everything in the "hard to replace" and "irreplaceable" categories to a seperate (removable but stays in the system) hard disk (so far 1TB has been enough to hold all this data).

1 TB is so much. I am curious how everybody uses so much space? My system partition is 2.7 gig gzipped. My home partition includes my family photos (my most precious data), which is accumulating at 100-150 MB per year and approaching 1 gig. I have 20 gig of mp3's, and 2.5 gigs of other random junk I never look at like old tax returns and data from my dissertation. My drive is "only" 160 GB, and the majority of it is pvr recordings, it does fill up but I've always been able to find shows at least a couple months old I still haven't watched, and clear some out. I don't bother to back those up so my 60 GB backup drive is sufficient. I feel if I had more space I'd probably just lose track of what all I have.

So I'm just curious how people use all the space? Shooting lots of photos and feeling like you need to keep them all in raw mode? Home videos? Yeah, pornography, ha ha, but seriously what takes most of your space?

Re:I like my layered approach..

[Anrego]

Well, as I said, I include both the hard to replace (which may not be stuff I made.. but stuff like old TV shows and such that I found after years of searching).

My irreplaceable stuff probably fits in under 200GB.. a lot of that is converted home movies .. and virtual machines (some of my hobby projects involve multiple virtual machines and virtual network environments.. which ads up).

Documents and source code and photos take up very little space.

That being said.. my "replaceable" stuff probably constitutes about 3.5TB .. I have a massive DVD collection (well over 600) of which I am gradually ripping (surprisingly just for personal use).

Re:I like my layered approach..

[Animaether]

Wouldn't that be reasonably solved by using two raids?

i.e. say you're using a (mirrored) RAID setup as your 'live' setup in order to prevent any data loss from e.g. surface degradation or even total failure of a drive's electronics. This doesn't guard against user error (zap a file and it's gone on the mirror just as well) and viruses and such, of course.

So you also make backups -of- this live system to a backup drive. Now if that's a single drive.. sure, that could die, and that would suck. But if that is -also- a mirrored raid (with proper controllers and whatnot).. wouldn't you decrease the odds of that backup being dead quite a bit? especially if you backup regularly, as any problems identified with one of the drives in the raid setup should be alerted so you can go slot in a spare?

I realize that there's always a chance that both drives die, or the raid controller itself dies.. but it still seems much better than using a single drive backup... or worse, blu-rays, DVDs, etc.
( I don't really trust online backup services enough to use them for backup of important things; but I suppose they -should- be a reasonable alternative for those willing to pay for them )

Re:I like my layered approach..

[bdc0]

I have a similar set of data. I use Truecrypt (Windows XP, cygwin) for the irreplaceable and sensitive files, e.g. tax records. Then use a file, not whole disk and rsync the files to it. Unmount and rsync as suggested by other posters

Re:I like my layered approach..

[jonbryce]

Videos of one description or another take the vast majority of my space. Music takes up quite a bit as I keep it in lossless formats wherever possible.

Re:I like my layered approach..

[Anonymous Coward]

I'm a hobbyist photographer. I shoot in continuous drive mode, so I can fill a 4 GB memory card in just a few hours. The idea is that even if most of the shots are terrible, I'll get a couple of good ones. RAW is way to space-hungry for the marginal increase in possible quality, so I haven't shot anything in it for months.

My photo library is currently well past 100 GB. Thousands and thousands of photos from dozens of events.

Re:I like my layered approach..

[Kizeh]

Lots of photos in RAW format. Also, scans of family albums and photos. Doesn't really much matter if they're RAW or DNG, as long as they're lossless sensor images, they're going to be big. Even being pretty picky with the ones I keep it adds up to 5-20 GB a year. (Then again, I do a lot of photography.)

Re:I like my layered approach..

[timeOday]

RAW is way to space-hungry for the marginal increase in possible quality, so I haven't shot anything in it for months.

I was excited to get a RAW-capable camera, and now I don't use that mode. I haven't found that postprocessing can turn poor photos into good ones, nor have I found that starting with RAW instead of jpeg helps the quality all that much when making white balance corrections etc. Admittedly this may be partly due to my toolchain. I like the jpegs out of my camera (Panasonic LX3) more than what I have accomplished with UFRaw/DCRaw (which produces blocky grain), and I use gimp which is still limited to 24 bit depth. But when I look at a lot of the high-rated stuff on flickr most of it is too over-processed for my liking.

I also delete all but my favorite photos. I only want to remember the highlights, not how mundane my life really was :)

Re:I like my layered approach..

[Jack Greenbaum]

Sure would suck if your DVD's and your computer burn down in the same house fire. Do you have insurance to cover them all, and a list of titles so you can replace them? You can fit alot of DVDs on a 1TB hard drive ...

Re:I like my layered approach..

[Anonymous Coward]

Agreed.

When you lose a hard drive, no matter how big, you realise one thing.

It's always the smallest files you miss the most.
That txt you wrote while drunk that was amusing.
The resume, the photos.

Basically those files that are unique only to you.
These are the only files you really need.
You can most likely fit all of these comfortably on a 4gb flash drive, but with 16 and 32 so cheap, why not get one of those?

Re:I like my layered approach..

[failedlogic]

I think this is where file 'metadata' can be REALLY useful. You could tag a file 'Irreplacable' (instead of storing it in an 'Irreplaceable Folder' and when you run a backup script you can download data according to the metadata.

One thing that saved my butt a few times - especially with DVD backups - is PAR (Parity Archive) files. I made a 100% parity archive of some 3 GB of split zip archive. One of the archives became corrupted and I was able to recover it.

I'm moving and am in the midst of trying to organize my computer at the same time. Having gone from Windows -> Mac -> Windows, its become increasingly frustrating to keep stuff backed up. You get the .DS_Store files littered all over the place too. I have to categorize and sort a lot of documents that I never bothered to do.

I'm really contemplating ditching DVDs all together. They are a pain, with all the cases risk of scratch, etc. A couple of hard drvies seems to be a good solution. Two 1 TB mirrored (non-RAID) easy to put 'on-line' and another where I just backup the data and turn-off and put the drive away until needed.

There's been some /. topics about it before but I'm giving some more serious thoughts of putting my personal work in a Versioning System (Git, CVS, Subversion, etc) and everything else is backed up with RSync.

Re:I like my layered approach..

[ZosX]

Since you asked....

Yes. I store just about all of my pictures in raw format at 12 megs per shot. Seeing as how I have thousands of saved shots, it adds up to many gigs. I also have about 40,000 mp3s/oggs/flacs, which eats up around 300 gigs or so. I have every episode of star trek TOS and TNG. A bunch of HBO shows and a whole bunch of movies. Oh I have like 100 gigs of classic PC games and about 50 gigs of PSX games and like every single NES/SNES/Genesis/GBA/GB/NEO-GEO rom known to man. I probably have like 50 gigs of comic books too and a whole crapload of e-books. Also add in backups of isos for various commercial software and it all really starts to add up quick. I need to get at least another TB drive just to back up my backups. :)

I wish there was somewhere where I could just throw it all up on ftp for you fellow geeks. If anyone's really interested...my e-mail is public on slashdot.

Re:I like my layered approach..

[DarthVain]

I think this is important to do. Classify your data, and keep it separate for easy backup. Back up in order and frequency by importance.

External USB HD are fine for this. HD are big enough that it shouldn't be an issue.

I have Terabyte+ system but really when you get down to it, most of it is not critical data. I would also go with the 3 tiered system, Critical, Important, and Annoying. Stuff like personal photos, tax information, personal documents etc.. would all be critical and would all take up the least amount of space, likely less than 10GB anyway. Maybe your Music collection falls into the next category, and your torrent video into the Annoying category.

Using online storage to back up everything would be overkill, but using it to only back up "critical" files might be worthwhile.

Re:I like my layered approach..

[greed]

The thing about RAID is....

RAID will correctly propagate an incorrect command across all copies in the mirror.

Any sort of incremental backup system will allow you to go back "in time", like with source control, to before that error and pull the data back.

So, I've got RAID for hardware errors and Retrospect for operator errors and MAJOR hardware errors. (And have hand-rolled a crude equivalent with dump(8) in the past.) Retrospect or SuperDuper is handy for system changes, too; back everything up, swap disks, restore.

(Windows users may find 'restore' is a much more complicated step than us UNIX/Linux/OSX users. Especially if the bloody thing deactivates before you get a chance to restore the system with the activation code in it.)

Re:Do we have to bring this up over and over again

[haifastudent]

rsync.

That's the protocol. Now what media do you recommend? Another hard drive?

If you're paranoid...

[MukiMuki]

OpenSolaris and 8 drive RAIDZ-2. PHYSICALLY disconnect that fileserver (and turn it off) and sync up to it once a month.

Use GlusterFS or RSync to sync that up to your main computer. If you can figure it out, make incrimental backups to DVD once a week (or day, if it's that important). Take those DVDs off-site into a vacuum sealed (not expensive, you can make one that uses a hand pump and a box). If everything goes to hell, restoring from DVDs takes forever but you have that option, and that's what's important.

Windows Home Server + Jungle Disk

[nadamucho]

Windows Home Server actually has very good backup options. a)It allows for folder duplication on shared folders, protecting your shared files against a single hard drive failure. b)It allows you to add a hard drive as a backup drive, basically to dump all the shared folders, which can then be taken offsite. c)Jungle Disk has a WHS plugin, and there's an alternate Jungle Disk plugin which is allegedly better on whsplus.com, which provides your online protection. Automated daily backups mated with Volume Shadowing means that not only is your data safe, but previous versions are available too.

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

Second Windows Home Server; works very well for me.

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

Any be "reliable" you actually mean "used to have a known, unpatched for more than 1/2 yead data loss bug?"

Re:Windows Home Server + Jungle Disk

[tbhall77]

I use windows home server as well and its great. You can install the windows home connector software on your desktops and it'll do backups according to the schedule you set and keep them for however long you want. Default schedule is to run between 12am - 6am and keep 3 daily, 3 weekly, and 3 monthly. With the WHS BDBB, you can make a backup of those backups to another drive to take offsite if you wish. Also, these desktop backups are fast after the first initial backup. Only the changed clusters are backed up aftet that. here is a doc that goes into it more. http://download.microsoft.com/download/1/8/0/18096c95-4850-4176-9821-970691b98aaf/Windows_Home_Server_Technical_Brief_-_Home_Computer_Backup_and_Restore.docx

Re:Windows Home Server + Jungle Disk

[1310nm]

WHS is basically just JBOD; I don't feel safe with it. When I was in the beta, they didn't explain exactly where or how files were stored, just that if you weren't using the full capacity of the array, that they were redundantly stored across drives. I like to be able to mount drives under linux to recover files if the OS crashes.

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

Windows Home Server is awesome.

+1

Re:Windows Home Server + Jungle Disk

[Falcon4]

Okay, well let's start here. First of all, the most impressive feature of WHS is that each disk operates completely independently, with their own file systems, meaning that you can take a drive out of the "array", plug it into a PC, and you can browse the folder structure to access that resides on that particular drive (in its original folder/file hierarchy as well). I'd certainly love to see you do that with a RAID, or even a JBOD.

Backup, on the other hand, is a proprietary clusterf*ck. But it really doesn't matter that much, since backups are only a copy of existing data, and if it fails, you just delete the backups and make fresh ones (since it's statistically very unlikely that both your computer and server would fail at once - and in that case you're boned anyway). If your PC fails, just do a bare-metal restore off your last image (which is why it's so proprietary).

True, MS hides a lot of stuff from the operations of WHS, but it doesn't take a rocket scientist to uncover how it works under the hood...

Re:Windows Home Server + Jungle Disk

[Falcon4]

Using a homebuilt WHS here myself. I never backed up until I got WHS, because it was simply too much a pain, and I didn't want to dedicate a hard drive to tediously backing everything up. I knew it needed to be done, but I never did. Finally, I found that WHS employs bare-metal image restoration off a bootable CD. THAT was the feature that got me hooked.

Basically, WHS does nightly (or daily) incremental backups over the network (or VPN if you use Hamachi), taking about 5 minutes average for each backup. Trash your hard drive by accident (i.e. beating the snot out of it when it keeps crashing...)? Drop in a new hard drive, break out the restore CD, connect to the server, restore last night's backup. It also automatically manages cropping out outdated backups and cleaning up the backup database.

Additionally, if you have multiple computers (up to 10), it'll cross-reference matching parts between different computers (so-called "single instancing") - that is, if two computers have a copy of a movie, it'll only occupy the space on the server once. Even if two files are different but have matching parts... it'll do that as well.

Run out of space on the server? Plug in any USB drive, add it to the pool, get extra storage. Build your own WHS and put it in the living room as a sort of "media center" PC (SageTV works with WHS). Make it your torrent box. Honestly, I can't see anyone living without a WHS in their home, at least anyone that's got more than one computer...

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

And WHS supports Time Machine :)

Re:Windows Home Server + Jungle Disk

[metallurge]

So, how does the licensing work? My understanding is that you get 10 slots for all eternity, and each unique client permanently uses a slot. If I take a WinXP machine and upgrade it in place to Vista, will I have just permanently burned a slot? How about if I take a given set of hardware and do a clean reinstall of Windows, have I permanently burned a slot?

Re:Windows Home Server + Jungle Disk

[Falcon4]

No, they're not permanently "burned in"... you can only have up to 10 systems "in place" at one time. You can "remove" a system at any time (and free its slot), but doing so deletes any backups that system had. I've done this countless times with my network with the number of times I reformat and repurpose systems.

Re:Windows Home Server + Jungle Disk

[metallurge]

Oh good. I had pretty much ruled out WHS out of suspicion on this score. I am glad to know it was unfounded. Thanks for the information!

Re:Windows Home Server + Jungle Disk

[PiSkyHi]

Your boot drive has failed. Some people want to access their data using the same OS install. Most of those people will not use Windows at all to achieve this.

Re:Windows Home Server + Jungle Disk

[FirstTimeCaller]

Wow. Suggesting a Windows solution on Slashdot. Brave man (excuse the assumption that you are male, but I like my odds). You're also right on the money. It's pretty much configure and forget. The best part is that the restore is pretty fast and painless. Backing up is easy... backing up in such a way that you can easily and reliably restore is the trick and WHS succeeds at this.

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

Second that - WHS is an easy answer since you it will duplicate files across HDD's for you.

For the $100 US you can get it for, repurposing old hardware makes this easier for creative types (like me) who don't know scripting or linux/bsd.

Re:Windows Home Server + Jungle Disk

[Anonymous Coward]

Just don't try WHS on a network with more than one or two computers. The backups are single threaded so they take all day to run.

My system

[Pathwalker]

I bought a pair of Infrant ReadyNas NV+ systems a couple of years ago; I kept one for myself, and gave one to my parents.
My computers back up to my nas box, my parent's computer backs up to their nas box.
I keep a ssh tunnel open between both of our networks, and each nas box uses rsync to back up to the other one.

The only problem I've run into so far is Comcast's 250 gig cap; but so far I've been edging in slightly under the limit.

Re:My system

[BlackCreek]

I have a ReadyNas Duo. I find it extremely slow for incremental backups. Sometimes I think I should have just bought a big external disk.

Do you perform full backups or incremental?

[...]

My backup scheme:

Every two or three weeks, I'll do a incremental backup of my whole home directory to my ReadyNas (through faubackup).

The actually important stuff is under either Git or Mercurial, I push them either to the ReadyNas or to my G1 SD card after a significant commit. I have been considering using only one of these, and using private repositories with either Bitbucket or Github.

What I'm doing this fall...

[JimXugle]

I've re-purposed a computer as a backup server, which lives at my parents house. It runs Ubuntu, with ZFS running over FUSE. Each night, a scripted CRON event will run zpool scrub on my storage pool, and if there is a problem, it will send me a text.

My MacBook Pro will use Time Machine over NFS over SSH to make the actual backups from my dorm/wherever I happen to be.

Commence CDDL/GPL/BSD Flamewar.

Re:What I'm doing this fall...

[coaxial]

How did you setup time machine to do that, since it doesn't work out of the box like that.

Re:What I'm doing this fall...

[BlackCreek]

The problem I have such a setup is the energy cost of having a (old?) PC on at all times. Did you ever try to measure the cost of running it per month?

(Or my assumptions are wrong? Wake-on-lan?)

Re:What I'm doing this fall...

[HuguesT]

Hello, I'm not the parent,

I'm assuming you know how to connect to a remote machine over a ssh tunnel. The NFS ports are well-known. Then this is a simple matter of letting time-machine accept NFS mounts as valid backup drives : you can have a look at this site for instance.

Cheers.

Re:What I'm doing this fall...

[ZorkZero]

Great, you'll have a frankenstein pony, which doesn't exist yet. You know what a flamewar is though, so you must have some sort of credibility. You're certainly buzzword compliant. Nice score on that post.

Time machine over NFS over SSH? I assume you know what that entails. Wait, no I don't. ZFS over FUSE? Sounds spiffy, maybe you could put another layer of abstraction in there somewhere to make things more reliable.

For real backup needs, tape is still king. LTO4 can write 120MB/s, twice that if the data is compressible, and has a 30 year shelf life. We're talking about cheap and easy though.. Well guess what? Nothing's changed since this came up last week. Choose your cheap storage option and take your backups offline if you want them to be safer.

Re:What I'm doing this fall...

[sydney094]

Just to let you know... Time Machine over the network is still a little flaky. I wouldn't necessarily trust this as a mechanism for automated backups. You still will probably want to stick to good old fashioned rsync for your backups. You can still setup a cron job (or Launch{Agent|Daemon}) to handle this automatically.

Alternatively, you could patch into the FSEvents framework and roll your own TimeMachine-ish like service.

Re:What I'm doing this fall...

[Samah]

I've re-purposed a computer as a backup server, which lives at my parents house. It runs Ubuntu, with ZFS running over FUSE. Each night, a scripted CRON event will run zpool scrub on my storage pool, and if there is a problem, it will send me a text.

Very close to what I'm running, except I don't have an automatic script at the moment since I only copy data to the filesystem occasionally. It's running on two raptor drives as a mirrored vdev.

Re:What I'm doing this fall...

[Shamenaught]

Personally, I'd have to agree that ZFS is probably the way forwards for storage at the moment. I've not tried FUSE yet, although I have looked into Solaris and Nexenta, installing them on virtual machines for testing (I want to ensure my backup is stable, and want to put it through its paces before I setup what will be my next server).

Once you realize that ZFS can run Raid-Z in a system on chronically faulty hardware, and it still loses no data, then you understand that any other storage methods (bar possibly online backup) are second-class in comparison.

Re:What I'm doing this fall...

[dstar]

>For real backup needs, tape is still king.

No, it's not.

As it happens, I do storage administration for a living. My previous employer had 4.5PB, and was actively evaluating moving from tape to a disk backup (although, since they're a major financial company and have so much data, it won't be a fast transition). My _current_ employer has perhaps half that, and our primary backup system is snapvaulted to a remote site, and then backing that up to a virtual tape library. It will be our *only* backup system as soon as we have enough capacity there to handle what little remains on tape.

Tape is expensive, time consuming, and problematic. The media costs are higher than for disk, the labor is *far* higher, and we have more trouble with our tape backups than our disk backups.

And the media cost for disks becomes even greater when you consider that tape compression is streaming, but data dedupe gets to take into account everything on the virtual library (we're aiming at a compression ratio of ~11 for libraries which have a full backup cycle. I have no reason to believe we won't do better than that).

Tape is obsolete.

Re:What I'm doing this fall...

[Anonymous Coward]

Time machine IS NOT backup.

Time Machine is usefull for everyday work. It does not replace backup and will never replace backup. TM is like looking in your paper basket for previous document. It is far from microfilm stored in a secure location.

Keep using Time Machine and get also a good backup software.
Use Time machine for regular use.
Set incremental backup on external or remote disk once a week. If you need extra safety, back it up again incrementally on another disk once a month.

At home I have A LOT of pictures. About a century worth of pictures. Totally irreplacable. They get the full backup treatment (BKP, and BKP for BKP). last backup on DVD I did was last year: 30 DL DVD... A pain!

I also have a lot of music. Whether I converted it from CD or I purchased it on iTune, I use Time Machine and Backup. No Backup of backup.

One last thing, get a software you like. Not the best, or the cheapest but one that you will use. Personally, mine is home-built.

Re:What I'm doing this fall...

[jvin248]

If you tune your system, those old boxes can be quite low power... like energy star rated Pentium-II's from 1998+/-. These work great as file servers. Install something like FreeNAS or build a Debian server and set it up headless & remove all unnecessary cards (sound, video, etc). I've had them idle less than 20watts before.

those old pc's often didn't need auxiliary cpu fans (I have a P2-266Mhz box that's 'fanless'). Compared with newer P4's that all have blazing fans because they throw off so much heat - maybe you can dial them back...

You can check your system with a kill-a-watt meter. Don't go by the powersupply 'rated' number - that's at peak load not squeezed down energy sipping for infrequent backup server use.

Offsite and full backups

[stoffell]

I recently started with an off-site rsync copy (daily) to a remote linux box. The linux box has an encrypted partition for added security.

Advantages: automatic, offsite, encrypted (rsync over ssh, over an Openvpn tunnel), rsync algorithm (only copy differences, etc..)... An extra step in the future will be adding BackupPC to the remote linux box.

Why all the hassle? I need to safe-guard approx. 25 GB (mostly pictures) and online backup services. If they're free, no thanks.

I use Limewire to backup my multimedia

[Anonymous Coward]

I store all of my porn videos and ripped music in the Limewire cloud, and let other people back it up for me. Works great, and I often realize I have backed up songs that I don't even remember ripping!

Re:I use Limewire to backup my multimedia

[JustOK]

Hey, some of your files are corrupted, they don't really work anymore...so I've heard

not really.

[ZERO1ZERO]

"it there any better advice these days"

Not really, keep doing it like that. for how to do that read this: http://jwz.livejournal.com/801607.html

I'm kinda a 'option 1' guy, but stuff that's really important, I just burn on to DVD every so often.

The other option, now that most folk now have halfdecent connections is to set up an rsync to a buddies machine, (and reciprocate) , using encryption, you now have an automatic off site back up.

Same old

[Anonymous Coward]

HDD are fast, cheap, and sort of reliable. Keep two separate (physically different locations) double sets of them and its relatively safe.... and probably still the best way to do it for large data collections.

If the data only has small changes to it day by day and you have fast upload access consider adding over the internet off site back up (two locations).

Anyone know the long term stability of SSD drives?

Mozy is good

[Mike McTernan]

Mozy is good - it's offsite backup with nice shell integration. Sadly it's Windows only though :(

Re:Mozy is good

[dieman]

Except its also a-ok on Mac OS X. I use it to backup my home mac server just fine. It appears to use some hack based on rdiff-backup.

Re:Mozy is good

[Anonymous Coward]

hmmm. i use mozy on my macbook, but no shell, that i am aware of. it's automatic and free up to 2 GB.

Re:Mozy is good

[swabeui]

Mozy has saved me twice. Two massive drive crashes about a year apart. For $4.95 a month, you can't beat it.

The only downsize is the restore. I was in the middle of a huge project during the last one and it took 3 days to download everything. I had to download bits of data that I needed to get working again, but really got me thinking I should do more than just Mozy.

I have two raid1 sets, one internal and the other external. External has all my data, pictures, etc.. Internal has scratch folders and the OS. I realized that having to rebuild the OS, developer tools, adobe suite, settings, takes a couple days barebones and weeks of tweaking after. It was only another $80 to mirror it, well worth it.

'backup' to a computer is more than just a hard drive crash. Theft and Fire can just as easily wipe out a decade worth of family photos.

Re:Mozy is good

[Anonymous Coward]

It's limited to roughly 100K/sec, which is insane.
I'm using backblaze which at least gives me 500K/sec (at least, it did last time I checked).

I wouldn't mind being able to use my entire pipe though..

Re:Mozy is good

[Anonymous Coward]

Mozy ran like a DOG on my OS X machine. So does Carbonite.

I think that CrashPlan is the way to go. Their hosted solution is a bit more expensive, but you can back up to a buddy for free.

Re:Mozy is good

[Anonymous Coward]

Mozy works great. For home users it's only about $5 a month per computer for unlimited storage. Of course the initial backup takes ages to complete, but once that's done, subsequent backups are differential so they finish in no time flat. I had a drive crash that had all of my digital photos on it, and I had all of them restored from my Mozy backup in no time flat.

Re:Mozy is good

[bkpark]

Mozy is good - it's offsite backup with nice shell integration. Sadly it's Windows only though :(

Er, how could it have shell access if it's "Windows only"?

Oh, or you mean one of those "Windows shells", which is a mere shell of what *nix shells (bash, zsh, etc.) and SSH access are in terms of usability.

Easy

[Isbiten]

I have an external harddrive attached to my Macbook and Time machine takes care of the rest. And my important document and photos I upload to my dropbox That way I have a local backup of my entire harddrive in case something happens to my Macbook and one stored on the "cloud" that I can reach if my house burns down.

Re:Easy

[djhertz]

I use dropbox for sort of the same reason. I've also found that it's replaced my thumb drive and ftp server to some degree. If I have mp3s at home and I want to bring them to work, or vice versa, I just drop box 'em. It really is a lot easier then actually having to manage your data. It's also been handy if I have a friend I want to share a doc with that I update a lot, ala RockBand "These are the songs I have".xls Being that drop box allows you to give an http readonly link to your file on the cloud. And up to 2 gigs free! It's sweet.

Re:Easy

[syphax]

I use DropBox for backup and synching. It's ~$10 a month for 50GB of space. I back up generic files (a couple GB) and our photos (many GB). I self-encrypt anything really sensitive (eg copies of tax returns with SSN), in case someone were to hack DropBox or my account.

For those of us for whom no longer find designing, implementing, and maintaining a home backup system captivating, DropBox (or similar) is the way to go- set it and forget it. I also like how it keeps our various family computers nicely in sync.

Options depend on your needs

[mysidia]

It depends on how many important files you have. If you have just a few documents, you can still burn them to CD periodically, or use an online for-pay backup service such as Carbonite or Rsync.Net. The reason to use HDDs is because you have lots of data, or your computer data, including OS installs is very important to you, and you need a way to recover rapidly. (E.g. you _really_ can't wait, and it's worth the cost of external HDDs and accessorie to avoid waiting)

If money is no object, ioSafe makes some fireproof, waterproof, shock-proof drive enclosures, which could help against disaster situations. The alternative is indeed use of an offsite location. You need a lock box or safe regardless of method, to help protect against human risks to your drives. Or utilize encryption to help prevent data from fallign into the wrong hands.

Otherwise, if you use HDs for backup, consider a hard drive docking station. Like one of these or a voyager Q (who makes a model supporting Firewire800 also); docking stations are more convenient to buying a bunch of external HDs. Eventually, when you upgrade your hard drive, use the old one to store important files.

If you have a stack of old hard drives, you can actually use them also. So a dock, and some plastic cases to put your internal HDs in could be favorable to buying a bunch of external HDs. (There are companies that specialize in selling rugged anti-static plastic cases for HDs, but I just pile them in a box, and use the original anti-static bags that came with new HDs)

If you are using old HDDs for archival purposes, make sure to spin them up every few motnhs, or you suffer bit rot, and the mechanical components of the drive may fail.

Or get one dock + multiple cheap HDDs for important documents.

And possibly one large HD for a full system backup. Apple users are blessed with Time machine. Linux users can dd or rsync their files, and even have a script do it nightly (so long as you have multiple HDs, and cycle them after backups).

Windows users have got to use third-party software or do some scripting.

Read The Fucking Summary

[Anonymous Coward]

But the one thing that's remained constant is hard drives are far too large to backup via conventional means.

Maybe it's unconventional to use, I dunno, another hard drive?

From said fucking summary:

"When last this topic came up on Slashdot, the consensus was that hard drives were the best way to backup hard drives. Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week. Is there any better advice these days?"

backuppc

[Jon_S]

http://backuppc.sourceforge.net/

Get an old P3 for free somewhere and load this up on it with a big disk or two for storage, put it on your network, and run it. That's what I do and it works like a charm. I went through all the options over the years, tape, DVDs, manual copying to a server.

Backuppc backs up all my windows and linux PCs. It backs up only what I tell it to, and it does both full and incremental. Sort of a pain in the ass to set up (I use cygwin rsyncd on the windows boxes, and regular rsyncd on the linux boxes), and it works well.

Only drawback is it is still on site.

Re:backuppc

[fluffernutter]

Another big advantage of BackupPC is that it only actually backs up file contents once, even if the files exist across 10 machines. In other words, back up three windows machines at the same level and you're really only writing one instance of windows to the actual backup drive. BackupPC tracks and manages links to the actual files appropriately for each machine. I use backuppc in my house as well but it just runs on a VM on my main machine.

Re:backuppc

[flycast]

Me too! I think that any backup plan is a failure if it depends upon human intervention once set up. Backuppc happens automatically and has saved my bacon. Also you can set it to run when network traffic is low. One last thing...I believe that I am right here when I say that it pools files across users. That is...if two different users backup the same file it only stores one copy of the file and then places a symlink to that single file. Some users are reporting a backup pool of 1/10 in size compared to the number of files actually backed up. I have 3-4 users on my network and I think my backup pool is 85% smaller than if the identical files were saved redundantly.

Re:backuppc

[Anonymous Coward]

http://backuppc.sourceforge.net/

Get an old P3 for free somewhere and load this up on it with a big disk or two for storage, put it on your network, and run it. That's what I do and it works like a charm. I went through all the options over the years, tape, DVDs, manual copying to a server.

Backuppc backs up all my windows and linux PCs. It backs up only what I tell it to, and it does both full and incremental. Sort of a pain in the ass to set up (I use cygwin rsyncd on the windows boxes, and regular rsyncd on the linux boxes), and it works well.

Only drawback is it is still on site.

what about power usage? having a p3 running all the time, can this program wake the pc when the scheduled backup is due?

Re:backuppc

[emeitner]

Sure. Each client's configuration can contain a pre and post backup command. The pre-backup command could be:
wakeonlan aa:bb:cc:00:11:22 ; sleep 30

Re:backuppc

[Anonymous Coward]

Sort of a pain in the ass to set up (I use cygwin rsyncd on the windows boxes

rsync on Windows can't copy files that are in use. So how do you backup the Application Data directory and the like?

Is your data _really_ that important?

[Vellmont]

For some people their livelihood depends on the safety of their information. For most of us though, it's really little more than attachment. If you've gotten to the point where you need to backup to tape "just in case", perhaps your problem isn't so much the danger of data loss, but you fear of data loss.

Re:Is your data _really_ that important?

[HuguesT]

Written as someone who has never lost 6 months worth of work in a few minutes due to hardware failure or theft.

Re:Is your data _really_ that important?

[Vellmont]

Reading comprehension not your strong suit?

RAID

[Deltaspectre]

Just because the backup solution _uses_ RAID doesn't mean the old adage applies to it. As long as you are using it as external backups all is well.

What that phrase IS telling you to do however is not use RAID on the machine you want to back up and expect it to do what you want.

Re:Do we have to bring this up over and over again

[Zurk]

i use rsync + samba on a linux box over the network with 2TB drives in a mirror (encrypted, mirrored with debian) for primary backup and have a LG Blue Ray burner for secondary backup. I get 50GB DL blu rays for $10 each from ebay (shipped from japan in 10 packs) mostly using verbatims. I burn two blu rays at a time with copies of the sam data on both and store them separately. $20 for 50 gig is not a bad bargain.

cost

[TheSHAD0W]

Once upon a time, the computer you wanted always cost (at least) $5,000.

This trend ended in the late 80's. All of a sudden, package system prices started trending seriously downwards, because due to Moore's law, computer speed started outrunning almost everything you'd want to run on it. Not true for certain specific apps, including graphics and games, but for office use it was perfectly fine.

I remember buying a 200 MB hard drive for $500 and thinking about what a great price it was.

Up until recently hard drives were one of the more expensive components left in a computer package. Now? Most are under $100. That's lower than tape backups used to be at their lowest prices. It's true, right now the best way to back up your hard drive is a second hard drive.

IMO the big question now is where that second hard drive will be. You can stick it in your computer and mirror your main drive in real time easily enough, but that means a virus or software issue will ruin both drives simultaneously. Better to sync them once a week? Perhaps.

Of course, this won't help you if there's a house fire. The fireproof hard drives are still darned expensive. Internet-based remote backup is great, if your broadband can handle it.

Re:cost

[Mr680x0]

I had a house fire and almost lost all of my data...fortunately my secondary HD was ok. The fire wasn't too severe. The main one seemed to have failed, but I cloned all my data to the second one a few days before. Now I put all my important files on DVDs and keep them in my car, so if there's ever a fire again I won't lose the data.

Re:cost

[value_added]

IMO the big question now is where that second hard drive will be. You can stick it in your computer and mirror your main drive in real time easily enough, but that means a virus or software issue will ruin both drives simultaneously.

Mirroring the drive? You're mixing RAID and backup.

That second drive has to be dedicated to be backups. How you perform those backups is up to you (ntbackup.exe, dump, rsync, file copy, etc.), as is the strategy (monthly, weekly, modified Tower of Hanoi, etc.), but let's be clear we're talking about backing up data, and not redundancy or availability.

If the issue is where that second drive will be, then you're options are internal (preferrably spun down when inactive), external (typically USB), or local network (another computer). The simplest option for those with a single a computer (the typical Windows user) is popping in an extra drive.

The only remaining issue is whether you want the contents of that dedicated drive to be duplicated off-site (preferrably encrypted) to protect against, among others, fire and theft. For that, you'll need to be creative and likely need to do some research as to the costs and feasibility. Walking it over to a friend's house would suffice, but there's lots of options available.

Re:cost

[dkf]

Of course, this won't help you if there's a house fire. The fireproof hard drives are still darned expensive. Internet-based remote backup is great, if your broadband can handle it.

Best bet: have both a local backup and at least one remote backup. Consider carefully what to backup (it's irreplaceable data that you need to prioritize, such as photos of your oldest child's first steps). Keep more than one backup; the limit is determined by the tradeoff between your paranoia and your wallet, but the responsibility is yours as you're the person who cares about it.

(Fires are deadly for disks. If the temperature of the platters goes over the Curie temperature for the material, the data just gets forgotten as if it never existed. That's physics.)

Re:cost

[FlyingBishop]

I think my ideal solution is mirrored drives on my main computer, an external 3-drive raid, and a single portable USB that I sync whenever I need to go on the road.

Currently, I can only really afford the mirrored machine with a single usb drive backup. Eventually, I think I'll supplement with an external RAID and a laptop.

I also backup everything offsite, as my hosting provider gives me far more space than I need for my site, and I have 30GB of stuff that needs to be backed up.

Re:cost

[bkpark]

Of course, this won't help you if there's a house fire. The fireproof hard drives are still darned expensive. Internet-based remote backup is great, if your broadband can handle it.

If your workplace allows it, you can put the second drive at work (presumably some of the files are important for work as well so you can justify it). If not, unless you are a hermit you can find a friend who might be willing to host the drive. If not that, then there are always places which will give you storage for some nominal fee (and unless you start backing up your music collection and DVD collection to it, it won't be so much).

Also, your backups should be snapshot style, going back months or years at least. At least under *nix, this is very easily done with cp -al && rsync method, and I am fairly sure you can use tar-based incremental backup methods under almost any operating system (although this is more pain than snapshot, since you should re-do a full backup every now and then).

BTW, as a sibling post noted, mirroring in real time (or even in delayed time; it just gives you more time to act, not an option to go back) doesn't count as backup. That's what the old adage about RAID not being backup is about.

Re:cost

[Anonymous Coward]

Then someone breaks in your car, steals your DVDs, sends you anonymous E-mail demanding a ransom for not just their return, but money so the data on your DVDs doesn't end up on a market to be sold to the highest bidder.

Or, there is a push for anti piracy in a community, the local police have a huge grant for looking for it in vehicles (and forcing the owner to prove that the IP is licensed to them), and the presence of the DVDs seen at a routine traffic stop turns it into a complete seizure where the vehicle gets impounded. Anti-piracy laws in the US passed in 2006 ensure the vehicle is seized and sold at auction.

Or, some drunk wrecks into the vehicle, and turns the DVDs into component molecules.

Moral: Use the vehicle for transportation, but have a more permanent place to store your backups. I personally have a climate controlled unit registered to my corporation (not me personally) at a storage unit in an outlying town on my commute. Here, I store my backups, encrypted of course. The backups go into a very heavy tape safe I bought for cheap (1 dollar, but I had to pay almost $400 in getting the one ton thing moved from the company's building to the storage unit. Finally, the backups are encrypted. Overkill for most purposes, but I know that should a burglar have enough time to trespass into a building, crack the lock on the unit, then get past a Hamilton X08 combination lock, they will just be getting a bunch of encrypted tapes.

Re:cost

[bill_mcgonigle]

RAID helps make for easy backup. I usually setup a dedicated encrypted RAID mirror for backups (rsnapshot), one of the members being in an external widget of some kind (SATA hotplug being my current preference). Then, once a week, I break the mirror, bring the drive to the safe-deposit box, get the one that's there, and add it back into the array.

It's easy to work with, rather safe, and I have online versioning for when I do something stupid.

My biggest problem right now is that the new office building I moved into is within a few blocks of the bank. But my risk of not being dilligent is still higher than the risk of local catastrophe. I started this scheme when my backups were unencrypted; using a safe deposit box at this point is largely redundant.

Re:cost

[cant_get_a_good_nick]

Once upon a time, the computer you wanted always cost (at least) $5,000.

Machrone's law lives ever on...

Re:cost

[Anonymous Coward]

Would it be a good idea to keep an additional back up (of the backup drive) off site in a safety deposit box? That should cover you for the fire problem.

Re:cost

[TheSHAD0W]

Good question, but the problem is going to your bank every week to pick up your backup and refresh it (or swap between two backups) is annoying and will probably lead you to slack off the task.

The "cloud", of course.

[John Hasler]

Anything else is *so* last century.

Re:The "cloud", of course.

[fluffernutter]

Yeah entrust my stuff to some third party and use my precious internet bandwidth to get it there to boot.. I think not!

Anonymous Coward

[Anonymous Coward]

For home backups, blue ray will do a fine job.

Shoot for long term reliability

[macraig]

Cuneiform tablets work well for me. Don't store them in a flood zone, though.

Re:Shoot for long term reliability

[Anonymous Coward]

This is why you etch on to stone, you dolt.

Re:Shoot for long term reliability

[macraig]

Yeah, but my carbide-tipped scriber is busted!

Re:Shoot for long term reliability

[careysub]

Dummy! You are forgetting the finalizing step, where you bake it in a kiln. After that it is water-impervious.

Re:Shoot for long term reliability

[macraig]

Ummm... my kiln is still on back-order from the Pottery Barn. But water impervious? Maybe water resistant....

Re:Shoot for long term reliability

[tcolberg]

I dunno about you guys, but I have a vault hollowed out into the nearby (mostly) granite mountain range specifically for my cuneiform tablets and root vegetables.

Re:Shoot for long term reliability

[ockegheim]

... as the pre-Noah Sumerians must have found out :-)

Luckily all they were recording was how many animals they had, so it became pretty much a moot point.

Re:Shoot for long term reliability

[macraig]

You did WHAT to Mount Rushmore!? Your roots won't keep very long when Lincoln's face falls off!

3 backup drives and MobileMe

[kshkval]

I have so many drives from so many years of external data that I do 2 Time Machine backups to 2 separate drives, a 3d SuperDuper backup about every 3 months to a drive stored in a safe deposit box and I avail myself of MobileMe sync services. Like I said, I use SuperDuper for the backups to the externally stored drive. That way, if a Time Machine backup is corrupted or wonky, the pooch is not you know what. On an ad hoc basis I backup my user folder using SuperDuper to a portable firewire bus-powered drive from OWC... so, I really have 4 backups. Between MobileMe, my smtp email, my use of gmail and the panoply of manually run backups, I feel pretty safe. Why feel safe? Bc I have done backups to a single drive before and ruined the backup and the main drive simultaneously. It was my fault, but I learned to never depend on a single backup again. Yes, it takes time, but the 3 months I needed to sift through 5 years of data (I used Data Rescue II from ProSoft successfully) was a lot worse. The other 2 Macs in the house are also backed up to 3 backup drives each using the same approach.

Tape is king for me

[adosch]

I use a pretty wide set of media for backups for my closet of hacked together servers; anywhere from SATA hard drives that I just hot-add and copy to, to still using CDs and DVD media for a tangible backup. But in the end, I really love tape. I've got a Dell Powervault 100T 20/40GB DDS-4 external SCSI tape drive ($40 eBay tops) and with a few open source tape backup tooks from freshmeat.net and a handful of tapes, I feel really confident in my backups. To me, tapes have ALWAYS been a reliable tangible fashion to back super important data or server/OS content and get it back, reliably. I've been doing this really light setup for almost 3 years now and it's never failed me and it's really not that expensive at all if you stay on the back half of tape technology. Upfront cost wouldn't be any more than $100 if you did some serious online shopping for the tape drive, 5-10 tapes and a SCSI LV card.

Re:Tape is king for me

[vlm]

To me, tapes have ALWAYS been a reliable tangible fashion to back ....... server/OS content and get it back, reliably.

I don't backup binaries.

What good is a backup of /bin/ls from my current AMD64 server, if I need to restore to non AMD64, perhaps intel ia64 or i386 hardware?

Of if my powerpc desktop running debian croaks and I find an i386 as a cheap replacement?

I also have an ARM arch debian machine in the basement somewhere.

Can my backup plan for /bin/ls really do a better job than Debian's four hundred and nineteen world wide binary mirrors?

http://www.debian.org/mirror/list

Re:Tape is king for me

[adosch]

Who said anything about /bin/ls (sounds to me you're being a bit troll-ish and off-topic for sure). This isn't a debate about architectures; you back up what you think is necessary for; 1) importance and 2) time saving in case of a disaster. From that aspect, it's almost wiser to back up the OS; if you're patching regularly, you've added that much more overhead to your re-build again. I guess if you're intention is to quack about ~100K of disk space (or OS as a whole, 3-4GB tops per server) you're a fool at best.

Re:Tape is king for me

[DriveMelter]

I have some old Sinclair Spectrum "backups" on audio tape, not sure if they can still be read though. All of the other tape formats I've used have become obsolete in a couple of years so I can see how tape is good for short term corporate DR but not really for longer term of making sure essential photos etc don't get lost.

Re:Tape is king for me

[mlts]

As part of one way of doing backups, I backup a complete system, binaries and all. Yes, /bin/ls is sitting on a lot of repositories, but grabbing the binary and sticking it in place is a good amount of effort compared to a bare metal restore.

With a bare metal restore, I can boot from an image on a recovery CD, run the restore application, click "Restore all - overwrite anything existing". Then, after a time of some minutes to hours, I have a machine back to a known good state. If the machine has a local restore drive, it becomes even easier, because it becomes a matter of just ensuring that the destination drive is partitioned right, and letting the process run.

However this isn't to say you are wrong. In some cases, a reinstall and a copy of a home directory is just as good. Especially on some UNIX variants which have almost every application a person would need either on the OS media, and there are no config files which had to be extensively edited. In some cases, it might be faster to reinstall a box, run the package update tool (yum update, emerge, apt-get), and then copy back the data in /home. However, I end up finding that an image based restore is a lot faster in most cases.

Tape is not unreliable.

[fishbowl]

Tape isn't cheap but it is cost-effective. I love, absolutely adore, the HP 1/8 LTO-4 autoloader that I use at my office, and I'm thinking about getitng one for personal/home use. Nothing else gives the kind of throughput we get from that thing (both reading and writing) and as for reliability, I would like to hear from anyone who has ever had an LTO-4 tape fail in service.

Re:Tape is not unreliable.

[adosch]

And I'd also like to hear from anyone who has an LTO-4 in THEIR HOUSE backing up their home system. I think you missed the point: Tape is cheap for home solution if you aren't bleeding edge. I think that falls in line with *any* sort of media for home backup solution.

Re:Tape is not unreliable.

[Colin Smith]

Dusty environments are a nightmare with tape. If you keep them enclosed they're fine, but the instant you take them out of the unit, especially if they are going off site in the back of a van, the dust totally destroys the reliability of the drives.

Treat dust like a disease. Quarantine tapes and drives which are used for offsite.
 

Re:Tape is not unreliable.

[fishbowl]

>And I'd also like to hear from anyone who has an LTO-4 in THEIR HOUSE backing up their home system.

I'm working on it. I won't be getting an autoloader, but I don't think single drive transports are that bad.
Trouble is, I really want SAS.

Re:Tape is not unreliable.

[fishbowl]

>Dusty environments are a nightmare with tape.

Yes. But can you document an in-service LTO-4 failure, or can't you?

I know one person who literally unspooled a cartridge and had it strewn in a hallway for people to walk on,
put it back in the cartridge and didn't have any data loss. I know that doesn't prove anything, but, I also
think an in-service failure of properly-handled LTO tape might be important news.

Re:Tape is not unreliable.

[rtechie]

What fails on hard drives? Usually it's the motor or the arms driving the heads. The MECHANICAL bits. Robotic tape arrays have a lot more mechanical bits to break. As you point out, the TAPE itself is very reliable, but the drives and array are significantly less reliable than the hard drives. If you have a broken array, you can't get your data.

In the corporate world, this isn't a big deal because you already spent $10,000 on the array and probably have a warranty and service contract. They really don't make consumer-grade tape drives any more, so as a consumer you usually have to buy OLDER UNSUPPORTED hardware off eBay, the stuff that is most likely to break.

This solution is just too expensive. For cost of the tape array you could easily buy TWO hard drives to back up each of your existing drives, which is probably more reliable anyway.

Re:Tape is not unreliable.

[jabuzz]

Dual drive LTO-1, came with a bunch of free tapes from work when we upgraded our Total Storage 35000 to LTO-4. Has 20 slots and gives me 2TB of backup.

Old tech tape drives and loaders are cheap on eBay.

Re:Tape is not unreliable.

[mlts]

If I drop a tape on the shelf to the floor, I pick it up, check for physical damage, perhaps run a verify to be safe, and it becomes a non-issue. A DLT or LTO type of tape (media that isn't based around a consumer format) tape has to suffer heavy physical damage before data loss happens.

This is one of the biggest hard disk weaknesses. I drop a removable hard disk, there is no way to tell if the platters shattered, the heads are misaligned, or the drive just chooses to retire. I invariably would have to stop what is being done with one of the tape drives in a library, manually mount the dropped hard disk, and run a battery of thorough tests. Even then, I'd end up copying the data to a known good HDD and relegating the dropped one to forever languish in an offsite Iron Mountain tub.

As for dust and storage, that is why it is highly recommended to store tapes in the cases they come in. Even though the cases are not hermetically sealed, they do a great job at keeping dust on the outside, and not where it can affect the magnetic media inside.

Tapes are not perfect by any means . I remember one form of tape drive had the issue of sucking their leader tape (the piece that connects to the the leader of the cartridge), and being thus unusable until you got service out there, or you went and broke "warranty void if opened" seals in order to unwind it (which you don't want to do if you have a support contract).

Re:Tape is not unreliable.

[fishbowl]

I concede your point, although your $10K estimate is a bit high. A robotic transport like that has multiple drives, and HP and Tandberg models allow you to jury-rig just one drive...

I also have to admit that at work I backup about a terabyte of data, nearly all of it essential to business continuity, per *day* and I send 4 tapes off to Iron Mountain each week, which is a separate job for archival purposes. This is so far removed from home use that I should not comment.
 

Tape can be unreliable

[networkzombie]

Since when is tape unreliable? My DLT has a MTBF of 250,000 hours. I've used DLT, DDS, and Travan for years and I've seen far more HDD failures. I've seen plenty of tape drives fail, but not the tapes themselves. I trust my tapes far more than any spinning platter. Come to think of it, I trust my tapes more than any other backup I use (Optical disc, HDD, and Cloud). Once my station wagon full of tapes caught fire on the highway, but I blame that cheap-ass roach clip.

Re:Tape can be unreliable

[doodleboy]

Since when is tape unreliable?

It sure can be, especially the lower end stuff like Travan. Where I work we have over 300 remote sites, which used to have TR-5 tapes and drives that failed continuously. We replaced all of them with a local rsync to a different partition with snapshots going back a week, along with a remote rsync to a bank of servers with snapshots going back a month. We had to shell out some cash for the backup servers and some dev time for the scripts, but the savings from not buying tapes paid for them fairly quickly. The local rsyncs take the place of tapes, while the remotes provide secure off-site storage. We have been able to rebuild branch office servers using data off the backup servers with no data loss and minimal downtime. Hard drives are cheap, fast and reliable. I honestly don't understand the appeal of tapes.

Re:Tape can be unreliable

[Bigjeff5]

First off, Mean Time Between Failures does not mean what you think it means. That is simply the average time between failures, and is in no way a guarantee that your particular tape drive will last anywhere near that long.

Second, most modern hard drives, particularly low speed hard drives, have a MTBF in the neighborhood of 500,000 hours and up. As your own personal experience suggests, that does not mean the drive will last 75 years. It would, however, if you used it for say three years and then archived it and used a new disk.

Third, hard drives, even low speed drives, are significantly faster than tape drives.

Fourth, hard drives, especially low speed drives, are signifficantly less expensive than tape drives.

Since you seemed more concerned with MTBF than any other statistic, why not spend a little more (ok, a lot more, but you're obviously not concerned with the least expensive option) and go with SSD storage? You get MTBFs in the neighborhood of 1 million+ hours then.

Seriously man, I work with large tape backup systems, and tapes fail just as often as hard drives, and they tend to be less robust. The only thing I can say for them is that they seem to do pretty well when thrown in a box and shoved in a corner for a few years. If you use them a lot, there is a heck of a lot more that can wear out in a tape drive than in a hard drive.

Re:Tape can be unreliable

[vlm]

I honestly don't understand the appeal of tapes.

Historically backup estimates were always too low. Not sure its relevant anymore. So, when you budgeted for 1 terra and you now need two terra, here's the plan:

Incremental expansion of tape backups = secretary asks purchasing to order more tape.

Incremental expansion of a rsync system = dozens of hours of hours of sysadmin time, possibly at multiple sites, etc.

On the positive side, having to replace all backup server drives is a good way to rotate new fresh drives in. If you budget that every year you'll buy and install a $250 drive, regardless of how much more capacity that provides or if you need it or not, you'll probably have better drive reliability.

Re:Tape can be unreliable

[Anonymous Coward]

Second, most modern hard drives, particularly low speed hard drives, have a MTBF in the neighborhood of 500,000 hours and up.

As soon as you start pulling them from the machine, putting them in a drop box and off-siting them, the MTBF plummets and is much, much lower than that of tape. There is no way I would ever use hard drives as an off-site backup medium. Ever.

Re:Tape can be unreliable

[Anonymous Coward]

that's wierd. your spinning platter has about 10x the relability of that
tape in 8-track motion around its case.

Re:Tape can be unreliable

[dbIII]

It's generally considered unreliable due to the "I forgot to load the tape but don't want to admit it factor". I've got a few thousand 9 track reels that have been stored in a shed since the 1980s. Every now and again I send one out for transcription and I haven't had any failures yet. I've heard that DDS tape used to break a lot but have never actually seen that happen.
The tape drives are now a lot less expensive than they used to be for both LTO and SDLT. USB 2.0 hard disks are also a lot slower than most tapes drives made over the last two decades (got less than 2MB/s last time I copied 400GB to USB drive) so go eSATA if you can.

Re:Tape can be unreliable

[hedwards]

I've never dealt with tapes, but I thought the issue was that tapes aren't necessarily going to work right if you have to swap drives. And realistically you shouldn't be assuming that you're going to have the same one available for many reasons.

As well as the fact that it's a lot easier to check HDD for bit rot than it is to deal with however many tapes. A simple utility like SFV will do that for you with a couple of clicks when it's time for the scheduled verification.

Re:Tape can be unreliable

[jabuzz]

Indeed, but it has a service life of 5 years tops. My tape's have a service life in excess of 30 years.

You need to separate the tape from the tape drive.

Re:Tape can be unreliable

[Anonymous Coward]

As someone who has worked in the backup industry for over 10 years, I would have to agree. The fact that the article simply blew off tape drives as "unreliable" is absurd. By far tape drives are the most reliable of any type of data storage. Tape drives provide the best long term storage of any method, and the new ones do a lot of internal checking to validate that the data written can be later read.

Re:Tape can be unreliable

[WuphonsReach]

Assuming that there's nothing else on on the USB bus that slows operations (such as an old version 1.1 device), you'll generally see 20-25 megabytes per second for sequential writes to modern disks.

Which is still about 8-10 hours to write 700GB worth of data to the disk.

Modern hard disks (750GB, 1TB, 7200rpm) can handle about 70-80 MB/s of sequential reads/writes now. eSATA does a better job of keeping up with this, but for the most part you can't hot-plug eSATA.

BD-R DL is expensive and inconvenient

[Weedhopper]

That's still roughly 10 bucks a disk for 500GB. You can buy 2x the HD storage for less than that price.

While optical media has its advantages, the convenience of an automated backup solution to an HD or multiple HDs means it's more likely to happen, thus is more useful. I do incremental backups to an external HD on an hourly basis. How do I do that on BD without it becoming very quickly A) expensive and B) damn inconvenient?

Let's face it, at the cost of HD storage, there's really no better general case solution.

Re:BD-R DL is expensive and inconvenient

[AsmordeanX]

You are right, BD-R is far too expensive. $114 for 10x50GB Disks. They are 4x so my whole primary partition would take just under 4.5 hours to backup including swapping discs.

Or I can write to my $85 1TB external drive sitting in a $30 eSATA enclosure for the same cost but in about 3 hours with no swapping.

BD-R DL is WAY too expensive and clumsy to be a viable solution. If the disks were 250GB and 500GB it might be reasonable even with the cost premium but not as they are.

Never posted before.

[Trogdor1]

I'm curious if this is breaking some kind of record for repeated questions. I mean I know it's a common thing but come tf on.

Re:Never posted before.

[Lally Singh]

Yeah. The reason is lots of people are suddenly saving nontrivial amounts of data (primarily media-driven) and they want a moore's law of reliable backup. But, all the consumer-level stuff (HDD, optical) isn't good enough and the rest costs actual money.

Until backup is as cheap, reliable, and able to store as much as the rest of consumer tech, we'll get more of these on /. :-(

In the mean time, tape drives are worth the money if your data's the result of real work or investment. If it's archived video downloaded off the net, then a few throwaway drives are best.

Re:Never posted before.

[Anonymous Coward]

Don't let it discourage you. The tape people are very defensive about their media because they get dumped on a lot.

proprietary raid sucks

[XaXXon]

One problem I've run into using hardware raid is that if there is a problem not with the disks, but with the hardware controller, you can be locked in to buying another of the device that screwed you.

However, with RAID 1 or software raid, you can easily just put the software on another machine and be up and running. For this kind of thing where performance isn't the biggest factor, that's what I go with.

If you look up the reviewes on the drobo, everyone seems to love it until it fails. Then you seem to be completely SOL.

Re:proprietary raid sucks

[Aphonia]

http://www.snia.org/tech_activities/standards/curr_standards/ddf/SNIA-DDFv1.2.pdf

DDF is a standard for this stuff, and you might be able to migrate within a brand - I 'm not quite sure how much DDF is used but i've seen it specified for some Arcea and Adaptec.

Anyhow, Id go with backing up to more HD's, and moving them to safe places (fireproof safe) with periodic testing.

Also, I'd put multiple copies of the most important data to say optical media or what not, and move some stuff offsite, and maybe grab some space online for some more backups.

Re:proprietary raid sucks

[BlackCreek]

I own a ReadyNas Duo, and agree with you. Hardware based raid has its problems. But:

1. they are much smaller (than a small tower);
2. are (relatively) silent, and use less power;
3. Can be programmed to turn on/off at specific times;

Do you know of an alternative to hardware RAID (home usage, only 2 disks) that is just as small and low on power?

Re:proprietary raid sucks

[Anonymous Coward]

Any modern ACPI-based motherboard can be programmed to turn on at a future time from within the OS. So you can have a Linux system boot up, do its intended work, then when it is done program the next wakeup and shut off. My MythTV machine does this now, so it wakes up at varying times based on its internal program-recording schedule. I can also turn it on manually to watch TV when it is otherwise off during an "idle" period. It stays on until recordings are done and also if interactive applications are running.

After getting this full-blown, multi-core, multi-drive (RAID5) system down to about 70W idle usage, I realized that turning it off for most of the day was going to save much more power than the impossibly expensive task of getting idle usage down to 20W or lower.

I have to admit, the geek in me would spend good money on a hybrid system that could run Linux 24/7 on an Atom-class CPU with active LAN and cheap flash drive, and then dynamically power up a full PC-class processor (and faster RAM, buses, drives, etc.) when needed by applications.

Depends on the files

[obarthelemy]

For really valuable files (the ones I won't ever be able to replace if I lose them: my own documents, my photos), I burn a monthly DVD and drop it alternatively at my parents' and brother's.

For the rest of the junk (media files: music, videos, books...)that are very large but not that important (or easily replaceable), I have a large external HD to which I clone my main HD once a week. I then keep the Backup HD off-line until the next time.

Re:Depends on the files

[Anonymous Coward]

I wouldn't count on DVD-R's for archival storage if I were you. There have been several recent articles pointing out that they are only shelf stable for a couple of years, maximum.

DVDs are just fine.

[bistromath007]

So what if you have to use a bunch of them? You can buy a giant stack of them for dirt cheap. That porn collection will impress your friends more when it takes up physical space.

Re:DVDs are just fine.

[WiiVault]

The problem with 50-100 DVDs is that there is a very high chance one or two of them will fail.

Re:DVDs are just fine.

[smash]

DVDs are a problem because you need to be physically present to baby sit the DVD swapping process, and they're slow.

If you want to back up several hundred gigs of data, you'll spend more time backing shit up than actually creating it.

Plus, in terms of $/gig, hard drives are not really much more expensive now.

USB or SATA?

[ericlondaits]

I'm thinking of backing up to another hard drive and I'm torn between an external USB drive or another SATA disk. Considering I would probably (if we're being honest) leave the USB disk connected at all times (for a daily backup) with full knowledge that in the case of a fire it'd be toast along with the machine... is there any incentive to use USB over SATA? It's pricier and slower, right?

Re:USB or SATA?

[WiiVault]

Depends on if you have multiple PCs to backup. In my case I have my basement PC, my work Mac, and an old iBook I use if I'm on the road. I'm also under the impression that more drives get fried when the PSU goes out than over USB.

as always, it depends

[digibud]

it depends mostly on how much data you have. I have a couple of terabytes. I also have a daughter living 5 miles away so I backup at my house to external drives and swap those backups with another set kept at her house. If you have 50megs of data (many people have very small requirements) then an online backup strategy might be very handy. You can even get 32gig and larger(?) USB flash drives that are more than adequate for most people who just want to backup their email and pictures/data. Tape drives are fine for geeks but access is slow and rebuilding a drive becomes more of a chore. Definitely not for mom and dad if they aren't geeks. External drives give total bootability (or the potential for it) and for most people are the easiest way to do a complete mirror of your HD. For data, most people can fit all their data (if they can even find it) on a USB Thumb Drive.

Re:Do we have to bring this up over and over again

[node 3]

rsync.

Ok, that covers the <5% of users who can set up and maintain a backup systems based on rsync. What about the other 95%?

As an interface to set up a backup system for a moderately adept geek with sufficient focus to set up and maintain a recurring rsync backup, an above average grasp of the layout of their filesystem, and the presence of mind to alter their rsync script as their computer changes over time, rsync is extremely powerful. For everyone else, it's next to useless.

RAID + one external drive

[Kocureq]

hardware RAID 5 for minimising impact of disk failures. And one external hard drive backup, kept in offsite location. Also, sync the most important data with your laptop, if you have one, so you have kind of two external backup media.

Let's define backups.. again...

[obarthelemy]

Backups are:
- off line (viruses, power surge, sabotage...)
- off site (fires, theft...)
- tested (i've got horrors stories of people that THOUGHT they had backups...)
- multiple (... and of backups that turn bad at the worst possible moment)

So backing up data is a hassle, and can be expensive depending on what you need: a DVD, a BD, an HD... But pretty much the only foolproof solution anyway is to burn your data onto a media you then send away to your parents' or other trusted 3rd party. Once a month is the very minimum.

If you're using HDs, you may want to re-use them after a while, but don't forget to keep some very old ones, for when you realize ages after the facts that one of your files got corrupted.

Levels of importance

[fishbowl]

There are three kinds of data:

1. If you lose this data you will go to jail.

2. If you lose this data, your business will be impacted.

3. If you lose this data, you will have less options for entertainment.

#1 tends to be a megabyte or less.

#2 tends to be a few hundred megabytes of documents.

#3 tends to be terabytes.

My company has a PDF of every document that we've touched in the past decade (federal law requires this retention), and our entire business continuity backup fits easily on four LTO-4 tapes, plus a very less-than-full tape that we rotate for offsite storage weekly. We've explored every backup system out there and this is by far the most cost-effective for us.

I don't understand why the OP claims "tape is unreliable", as I have not heard of a single instance of in-service failure of an LTO-4. As for it being expensive, it is, but before we went to tape we were using Firewire800 external drives, much more expensive than tape cartridges, and not as reliable as some people have been led to believe.

USB and FW external drives almost never fail as long as they are powered on. They fail in storage, which seemed pretty weird to me, since they should be able to sit on a warehouse shelf indefinitely. My low-sample unscientific data from experience says otherwise.

Since everybody is going from LTO-2/3 to LTO-4, you should be able to get LTO-3 transports pretty cheaply.

But my first advice is to identify the data in categories #1 and #2, where you might realize that it's a good practice in any case, to store the important stuff with its own priority. This is the hard part. Identifying what's actually important. If you don't do this, no matter what backup system you end up using, you're going to be burying the important stuff in the noise, introducing risk.

The OP also mentioned Drobo. I have a Drobo and I love it, but I must warn you that it's pretty slow, even with really fast drives. Don't expect to be able to copy a terabyte to it in less than 40 or 50 hours, even with firewire 800. This is the problem that drove us to tape, which is much faster than any filesystem we can feed it from.

Re:Levels of importance

[value_added]

There are three kinds of data ...

Perfectly valid when evaluating the finer points of different backup (or RAID) strategies, but I'd suggest for most people, it's leads down a path to grief.

Consider the typical argument where someone says "I can always reinstall Windows and my programs but I can't risk losing my [files]." That translates as setting aside their weekend to do just that (or paying someone to do the same), before laboriously restoring their Important Stuff from backup. Then, there's the following week where all the little things no one considered really that important (or simply forgot) that need to be addressed or otherwise fixed.

Me, I can, for example, perform a PXE boot and automated install in minutes, but I'll take the dump and restore approach over that any day.

Seems to me that if you're unsophisticated, you should consider everything important and back everything up. Life's much simpler without analysing the compromises and dealing with their shortcomings when things go wrong.

Re:Levels of importance

[dkf]

Seems to me that if you're unsophisticated, you should consider everything important and back everything up. Life's much simpler without analysing the compromises and dealing with their shortcomings when things go wrong.

If you have the space to do a full backup, sure do that. Consider also keeping another backup of the irreplaceable stuff; for most users this means the things within their home directory (and subdirs of it). It's only the semi-skilled who are really at risk; they know enough to have multiple drives, but not enough to ensure that things are properly backed up. (With real ordinary users it's actually easier since they usually won't squirrel away stuff they care about all over the place. Keep it all in a big unsorted heap on their desktop, yes, hide it on a hidden partition, no.)

Re:Levels of importance

[Anonymous Coward]

"#2 tends to be a few hundred megabytes of documents."

#2 may be a few hundred megabytes FOR YOU. For me, and anyone else working in media (stills/video/audio), #2 can range from a hundred gigs to tens of terabytes. I'm a very small independent contractor, and my current stills-only project is over 100gb at the moment--and will only continue to grow.

Re:Levels of importance

[Anonymous Coward]

This topic is specifically called "best HOME backup strategy now". Yes, complexities are introduced if you are a business, government agency, or the president of the free world.

Re:Levels of importance

[phelix_da_kat]

Exactly, it depends on your: 1. level of of importance 2. cost vs convenience (and how much data you want to secure) I have about 300GB (or 7yrs of photos, music and audiobooks). So online backup is too expensive. The library is pretty static, month to month, some new music, photos, a couple of books. Its just not worth trying to rebuild my library from my DVDs/CDs, friends photos etc. I wanted a mirror of everything, including the OS. With ZERO (well depends on when the last backup was) downtime. Therefore I kept it simple.. I have bootable backups. a) RAID 1 (1Tb, ie same size as my desktop) courtesy of OWC's Guardian Maximus, which is a simple hardware raid. Automated SuperDuper to do a smart backup every day. Pretty quick, over FW800 - This is currently connected to my desktop, although later I should do this over my network and keep the RAID 1 in another room. b) A portable drive that is updated either monthly, after a large session (photos) or just before a major update (service pack/update). This is kept off site. BTW i wanted a Drobo until i read the small print about it using it's own proprietary RAID. There's no problem if one of the HDs fails, you just rebuild it, but if Drobo fails... you need to buy another Drobo, or likely something from US Robotics (the makers of Drobo). I would suggest keeping to industry standard RAIDs. For most homes, RAID1 is the simplest.

Re:Levels of importance

[phelix_da_kat]

BTW...

in the OP.. when they say RAID is not backup.

They are technically correct.

RAID alone is definitely not a solution, although it can be part of the solution. The poster is being too literal. One has to identify what is important and how much they are willing to spend and how convenient they want it to be.

My solution works for me as it keeps my possible downtime to a minimum and I was happy to buy a RAID1 (1Tb) and had a spare portable drive. The RAID immediate protects me from my desktop failing (but not if I accidentally erase anything or something becomes corrupted). The portable is the backup for something happening to BOTH my desktop and RAID1 (which has 2 mirrored drives hence 2copies of my desktop).

The post is too generic.

If they identified in their post what was at stake, budget, their technical expertise...

I am sure readers of /. could focus their responses.

Re:Levels of importance

[durin]

I wouldn't call my personal files (pictures of my family and so forth) "less options for entertainment", so you better squeeze in another category there.

Re:Levels of importance

[Anonymous Coward]

How about: if you lose this you won't have any photos/videos of your kids or your family.

Re:Levels of importance

[swillden]

There are three kinds of data:

1. If you lose this data you will go to jail.

2. If you lose this data, your business will be impacted.

3. If you lose this data, you will have less options for entertainment.

True for business use, but for home users there's another category:

1.5. If you lose this data, you'll have lost irreplaceable records of important life events.

Much of my kids' lives is chronicled in bits stored on hard drives. Photos, video, school work (you might be surprised how interesting a 4th grade paper can be years later), etc. This data is significantly more important to me than business data, and there are many gigabytes.

My solution is peer-to-peer backup. Most everyone has extra disk space they're not using, and if they don't, well, a 1TB drive is $60 on newegg. So, I've set up a "storage grid" with my family members using the allmydata.org Tahoe network. Tahoe encrypts every file with 128-bit AES, splits each via Reed-Solomon encoding into six pieces, any three of which are sufficient to restore the file, and spreads those pieces around other machines, all of which are off-site.

Because Tahoe's backup facilities are a little primitive, I'm writing my own backup solution, called GridBackup, which separates file change detection and upload, because upload is inherently slow on typical ADSL/Cable Internet connections. GridBackup also compresses whatever can be compressed and uses rsync-style signatures and deltas to upload diffs for small file changes. Uploading is done in priority order, where the default priority algorithm favors user files over system files, small files over large files (on the theory that if you have to pick, it's better to back up many small files than a few large ones), and recently-modified files over long-unchanged files (on the theory that long-unchanged files are likely to stay unchanged for a while longer).

Re:Levels of importance

[fishbowl]

>#2 may be a few hundred megabytes FOR YOU.

It's about a terabyte a *day* for "us". Aerospace engineering docs, things that have to be retained by law...

Our situation is uncommon. Media stuff is always an exception, and, jokes about the porn industry driving all storage and distribution technologies aside, media storage and backup considerations are always outliers in this arena.

I know a thing or two about this; I'm an avid photographer and I have a side career in music production.

Re:Levels of importance

[atamido]

You might consider adding the ability to distinguish nearline storage. That way you could add computers around the home as devices that could be backed up to and restored from quickly, but that there would still be remote computers will full copies of the backup.

I'm curious how you are doing rsync diffs for encrypted Reed-Solomon fragments.

I'm also curious why the decision to break up files into three pieces and create three parity pieces. Is there a benefit/disadvantage to using this over pieces of fixed size (say 1MB)? If you have more peers, can you create more parity pieces? What about creating more parity pieces initially and using them if you add more peers (or is that too bandwidth inefficient)?

Re:Levels of importance

[swillden]

You might consider adding the ability to distinguish nearline storage. That way you could add computers around the home as devices that could be backed up to and restored from quickly, but that there would still be remote computers will full copies of the backup.

That is part of my longer-term plans, though not exactly for the reasons you suggest. My main reason for adding a nearline storage layer is to address the fact that most people use laptops these days, and although my solution does a very good job of allowing backups to be interruptible, backing laptops up directly to the grid would require leaving them turned on and connected for large amounts of time.

Also, fairness requires that you provide as much storage to the grid as what you're consuming from the grid. But you can't use an intermittently connected laptop as a storage node. Or, at least, it's would be a pretty unreliable storage node.

My solution to that is to back laptops up to nearline, always-on storage at LAN/WLAN speeds and then have that trickle the data up to the grid at ADSL/Cable speeds. In particular I want to take OpenWrt or similar and add a Tahoe client node, so that you can attach, say, a 1 TB USB drive to your wireless router and use it as nearline backup plus give it responsibility for getting the data pushed out to the grid. And it will also act as a storage node in the grid.

Using the default parameters, a 1 TB drive would give you about 230 GB of nearline backup, plus would provide your contribution to the grid for online backup of that much data (storing 770 GB of others' encoded data). So for the default encoding parameters, you'd need 4.33x as much nearline storage as what you're going to back up and that would effectively give you 4.33 backups of every file, one of them nearline and the others offsite. Less conservative parameters would reduce that amount.

I'm curious how you are doing rsync diffs for encrypted Reed-Solomon fragments.

I do the diffs on the files before encryption and FEC (Forward Error Correction) encoding. Because the storage servers cannot decrypt the files, there's obviously no way to "patch" the backed-up files, so the diffs have to be stored as a chain of forward deltas. Forward deltas suck for many reasons, but particularly they damage reliability, since the loss of any delta in the chain means that all subsequent versions are lost as well. I'm currently addressing this by limiting the length of the delta chain before uploading a new full version, but my plan eventually is to make those decisions statistically.

I have written an incomplete paper describing how to calculate loss probabilities in a distributed, FEC-based grid, and I eventually plan to offer GridBackup users the option to set their desired reliability level, with some description of the impact it will have on upload times, storage used, etc. I'll then use that reliability figure to automatically select FEC encoding parameters, and to choose when to break delta chains, so that the user's reliability requirement is met.

The other major downside of forward deltas is that if the chain is long enough, it can take much more time to download the chain than it would to download a final version. Because of the ADSL/Cable asymmetry, my design focuses on optimizing upload performance, but when I move to using statistical criteria for chain-breaking decisions, I'll also do some size calculations. The Mercurial developers (Mercurial also uses forward deltas) have found that a good rule of thumb is that when the size of the delta chain exceeds twice the size of the full version, it's time to break the chain.

Oh, you may also have been asking about how I create the diffs efficiently. I store local copies of the rsync file "signatures" for files that have been backed up into t

Re:Levels of importance

[WuphonsReach]

Tape is considered undesirable if you don't have:

- the staff to babysit the drive every night, and clean it regularly

- multiple tape drives, so you can deal with one dying, and you can do a restore without mucking about with the primary drive

- the deep pockets to sink $2k to $3k for each drive, plus all of the tapes you need to do a typical rotation (figure another few thousand)

- and you'll need to purchase a new, larger drive every few years, along with all new tapes

On the flip side, hard drive based storage is extremely attractive because:

- low cost to get started (2 drives and 2 external enclosures)

- no special hardware required to read it on any machine

- random access to your backup data, no 15-20 minute restore sessions (only to find out that you specified something incorrectly)

- easy to upgrade

Tape works well for high-volume applications where you're dealing with multiple tapes per day, have strict requirements, and you're working in the realm of hundreds of tapes or more. Basically, Fortune 500 sized businesses.

For smaller companies / organizations and individuals, tape makes a lot less sense.

You're the first to ask "WTF am I backing up?"

[Colin Smith]

I just can't be bothered with slashdot any more. It's full of dummies with mod points. How do I get off the Internet?

Do I need a megabyte of backup capacity for every megabyte of storage? No, I decide what's important and how long it's important for.
 

Re:You're the first to ask "WTF am I backing up?"

[Nethead]

Exactly! When it comes down to it the really important stuff I have could be backed up on paper tape. My resume, my tax returns and some other odds and bits. I use to try to save all sorts of crap, tried to "download the Internet." Ya know, I never looked at it again. Once in a while I'll find an old drive in a drawer, mount it up and then wonder why I was saving all my killer CGI scripts from '96. (Most of those "send a comment" scripts today would be called a spam-proxy :)

If the stuff is that important then that is what hard-copy and fire safes are for.

Rule one: If you got it from bit-torrent, then you don't need to archive it. If it ever was on TV, it will be again. If it's porn, there is lots more where that came from.

Rule two: If it's for work, then ask your boss how she wants it backed-up. Then you're covered.

Rule three: If it's 3 TB of video of the first year of your kid's life then edit it down to 5 minutes because that's all that anyone will watch (willingly) anyway.

Rule four: If it's killer code then tar-zip-gmail is your friend. Ask some other project if you can stash a copy on their CVS server.

Rule five: five-nines of everything is crap. Live now, not in the past.

Re:You're the first to ask "WTF am I backing up?"

[SpooForBrains]

Rule three: If it's 3 TB of video of the first year of your kid's life then edit it down to 5 minutes because that's all that anyone will watch (willingly) anyway.

What if your child dies in the same house fire that takes out all your primary storage?

Re:You're the first to ask "WTF am I backing up?"

[tangen9000]

I recently lost about 700GB of data when one of my 1TB external hard drives failed. On top of that, my laptop bricked the very next day, refusing to boot from anything (including a live CD). All in all, I lost about 1TB of data in the space of 2 days (I only own about 2TB of data, total, so this was a substantial chunk of my stuff).

Tragic, right?

Not really. Out of that TB, 90% was replaceable media -- music, movies, TV shows. It'll probably take a month or so to get it all back, but I can get it back.

As for the important stuff, that remaining 10% of data wasn't completely lost. As I had upgraded over the years, I had always copied the files on my old drive to my new one and then kept the old drive. So I had maybe 2-3 copies of old code and photos sitting in my closet on old hard drives. Any newer photos were on my Facebook -- they weren't the original, hi-def versions, but relatives and friends wanting to see the photos really do not care about that. As for newer code, I had gotten a small external drive and was using it to keep all of my code in one place and it was still working fine. I could even get my music back easily, since I used Rockbox for my iPod's firmware and I could just copy-paste everything as-is without worrying about special iPod transferring programs. All in all, I lost almost nothing important.

tl;dr or the lesson I learned:
I am now way less attached to my TBs of media. After losing so much I realized that the only things worth saving could easily fit on a DVD. Sucks for people that have more than a few DVDs' worth, but as for me, I'm planning to keep a small partition on my main hard drive dedicated to important data and back up the entire thing to DVD every month. I'm fairly certain that months worth of DVDs AND a hard drive won't fail me at the same time (even with my current bad luck re: external + laptop) so that should be enough for me.

Re:You're the first to ask "WTF am I backing up?"

[Anonymous Coward]

Rule two: If it's for work, then ask your boss how *she* wants it backed-up. Then you're covered.

You work for a woman?!? Have you no self respect man? :P

Re:You're the first to ask "WTF am I backing up?"

[seramar]

Rule three: If it's 3 TB of video of the first year of your kid's life then edit it down to 5 minutes because that's all that anyone will watch (willingly) anyway.

Do you have kids? If so, then I'm curious as to whether or not you've ever gone back to watch old videos of your children from when they were just days, weeks, months or a few years old. You can't stop watching it. The emotions that come bubbling up are undeniable. You want to hang on to three terabytes of video of your kids for _your own sake_ - not anyone else. Do you not have kids? If not then I recommend you not open your mouth on topics you've got no experience on. To the rest of us (with experience) it just makes it obvious how ignorant you are on the subject and, due to association, casts everything else you say into doubt.

Re:You're the first to ask "WTF am I backing up?"

[Nethead]

Considering my age that would be setting up the Super-8 projector, hitting ebay to find a good bulb and hanging a sheet. We also have a box of photos that have been scanned, burnt to CD and hard drive and stored at my sisters home. And yes, we get the weekly video updates of the grandkids and archive them on gmail. But we don't archive everything that hit the cutting floor.

Re:You're the first to ask "WTF am I backing up?"

[seramar]

Neither do my parents archive the entire collection of raw video data of my kid - but I do. Because, to me, it's worth while to watch 3 hours of my infant daughter rolling around on a blanket and chewing on her stuffed animals. It feels good to relive the moments of me playing with her at that age and seeing her react and respond to me. While I could live without that footage... well I'd just plain rather not.
Since this is a home backup solution we're talking about I think most of the data COULD be lost without extreme consequences... and so all of it would fall into that "I could live without it but I'd really rather not."

Re:You're the first to ask "WTF am I backing up?"

[Anonymous Coward]

I love a good smack-down among low user ID'ers. Where's (447) with the rebuttal?

Cheap drives and Bacula

[Etherized]

Well, I don't think there's an ideal solution, but I can tell you what I do.

For software, I find Bacula to be a very effective solution. It's open source, cross platform, and very flexible. Bacula was designed with tape in mind, so it takes a bit of wrangling to make it work well on hard drives - but once you get it set up properly, it works quite well with disk.

Now, I back everything "critical" up using bacula onto my ZFS array of cheap drives on my Solaris box. I just let ZFS itself do the compression (I didn't benchmark this, Bacula's compression may be more effective) and retain the backups on disk for about 2 months. I do nightly incremental backups and monthly full backups, but Bacula gives you lots of different options in this respect.

I then take the Bacula backups and rsync them over to external media weekly. I also take and keep 4 zfs snapshots of these backups on my external media, so that I can go back 4 weeks prior if I need to. I also rsync over to a separate smaller external drive "every now and then." I keep that other drive in the opposite side of the house (better would be to leave it at a friend's house, but I'm lazy - I just have to hope that one end of the house survives a fire / theft unscathed).

I have another class of data - data that I deem important, but also capable of being re-acquired at minimal expense. This data gets no incremental backups, and is only rsync'd around. It gets put on the larger external hard drive, but not on the smaller secondary drive. Beyond that, I have a third class of data, which I deem completely expendable. This is mostly normal recordings from my MythTV machine, which I consider an acceptable loss, and these aren't backed up at all.

At the end of the day, there really is no magic bullet. I really like disks + bacula, but what works best for you will depend on what you're trying to back up (and how much value you place on making sure that this happens properly).

Backup strategy

[Tet]

The backup strategy is the same as it's been for the past 40 years or more: keep copies of your data offsite, where you can get at them in a reasonable timeframe should anything happen to your local storage. The technology used to implement that strategy changes with time, though. Personally, I made the decision many years ago that removable media was a waste of time. The only sensible option is to have storage powered up and accessible at all times. So I rsync to a hosted server in a remote datacentre (the initial sync took forever over ADSL, but the nightly diffs are pretty quick)

Get out of your basement

[Anonymous Coward]

Here's a silly idea: buy a hard drive and a low-power always-on computer and negotiate with a friend to do the same. Backup over internets to each other's new computers. Then you can get away with just one backup device that also happens to be offsite.

Re:Get out of your basement

[as400master]

Not so silly idea actually, WUALA is doing exactly that except your data is scattered across multiple nodes (fully encrypted, multiple copies etc. etc.). The more storage you donate the more storage you can use.

Back up the Data Files to the Cloud

[Orion Blastar]

Ghost Virtual Machine gives 15gigs of Amazon.com data storage and right now if you use the promotion code of "launch" you get 10Gigs more as a bonus for 25Gigs. If you want to give me a referral my id is orion_blastar there, and each person you referred grants you 5Gigs more in a bonus.

Google Docs also has document storage but does not give as much as G.ho.st does. The Ghost Virtual Machine can access your Google Docs drive as well.

Here is a review of the top 5 online cloud storage sites so you can take your pick.

MyBloop offers unlimited free storage, but I am not 100% sure of that or their privacy policy.

Lifehacker talks about using your Yahoo Mail account for unlimited storage and also that Google's GMail almost offers the same service as well.

Re:Back up the Data Files to the Cloud

[Anonymous Coward]

Don't forget Crashplan www.crashplan.com. It features cross platform encrypted incremental backup in three flavors:

1. Backup to a harddrive you have in your home network (or attached to your computer). Costs nothing
2. Backup to a friends' computer (encrypted of course) over internet. Costs nothing
3. Backup to Crashplan's secure central facilities. Costs money.

There's a client for Linux, OSX and Windows, installation is easy, backup is automatically scheduled and notifications are mailed when backup is not complete (configurable). I'm using it for over a year now, and it just works!

Re:Back up the Data Files to the Cloud

[Anonymous Coward]

I just checked out g.ho.st, but it only uses https for logins and "state of the desktop". Not. Good. Especially when the whole idea of it is to have a common desktop available from any computer. No way will I log on here at Starbucks.

Re:Back up the Data Files to the Cloud

[Anonymous Coward]

where do you enter thr ghost promo code?

Re:Back up the Data Files to the Cloud

[Orion Blastar]

When you log onto the virtual machine after verifying your email. It will run a setup wizard and ask you your name, and then click next and it will ask you who referred you and if you have a promo code and then you enter it.

If it says you have a problem with your name, it is because it does not appear in their dictionary, just click the "Finish" button to exit after entering your referral name and promo code, in order to get the extra 10gigs of storage credit.

Re:Back up the Data Files to the Cloud

[Kaell Meynn]

I took a peak at it. Your referral ID didn't work though. Had to set the referral area blank in order to continue. Too slow for me to use as a replacement OS, might be okay as a backup though. I haven't looked at the ghost-drive part yet though. So far Dropbox, Mozy, and SpiderOak look the best for me.

Do nothing

[Anonymous Coward]

Do nothing(TM) is the cheapest solution to the backup problem. By doing nothing you ensure that no amount of time will be wasted looking fr the backups if you lose your primary storage !

Re:Do we have to bring this up over and over again

[godrik]

As an interface to set up a backup system for a moderately adept geek with sufficient focus to set up and maintain a recurring rsync backup, an above average grasp of the layout of their filesystem, and the presence of mind to alter their rsync script as their computer changes over time, rsync is extremely powerful. For everyone else, it's next to useless.

On the other hand, if you thought you could ask on /. you probably match this description...

Online+spare HD

[beegle]

Like most people, I have a small amount of truly irreplaceable content (documents, pictures) and a whole bunch of "it'd be annoying if I lost that" content (music, movies). One of the really convenient things about this split: the truly irreplaceable stuff is not very large. My docs and pictures occupy about 15 GB, and most of that is pictures.

I have an external hard drive where I back up everything at least nightly. This protects me from accidental deletions and a failed hard drive. It doesn't protect against fire or theft, though.

Services like Mozy and Carbonite offer off-site backup for about $5/month (there are many others -- these are the two best known, I think). I could string together something with a spare drive and a friend, but frankly, it would take a year or two before that approach matched the cost of Mozy et al., and frankly, I just don't WANT to worry about this crap. I'll pay the $60/year to make it someone else's problem.

One interesting option: Crash Plan at http://www4.crashplan.com/consumer/index.html . They offer free backups to friends' machines, and paid backups to their own fileservers. Sounds like the best of both worlds, but I haven't gotten around to trying it yet.

Drobo

[KDingo]

Now first of all, I think it is perfectly fine to use RAID as a backup as long as it is *not the primary storage* of the machine you are backing up.

I have been considering Drobo as a backup device but the high price keeps putting me off. However, unlike standard RAID implementations, Drobo's is unique where it is expandable. Just pop out the smallest disk and pop in a larger one. I think for people as myself that have growing libraries and growing home directories, maintainence of a backup device that easy should be appealing, since it gives us more time to think about how to migrate data off primary storage to a larger disk, or how to add an extra disk to the LVM, etc.

Re:DROBO

[itsthebin]

what is the price difference between a DROBO and a ATOM or Celeron MOBO with TB drives and FreeNAS ?

Re:Do we have to bring this up over and over again

[blueskies]

Aren't external usb/firewire/esata drives cheaper than $200/500 gig? I think they are around $100, so you get $10 per 50 gig. It's secondary backup while they aren't permanent you can rotate through them and reuse them.

Got a hoe? No, Go Tahoe!

[jafo]

Tahoe ( http://allmydata.org/~warner/pycon-tahoe.html ) uses crypto to allow friends to share their storage space without exposing what is being stored.

So, you could get a bunch of friends to agree to get 1TB drives and set up Tahoe so that you could all push backups to these drives for backup purposes. However, I'm not sure how you would push the backups to the Tahoe file-system. Since the company behind Tahoe (All My Data) uses it to provide backup services, there must be some thought that has gone into this, but I don't know if the open-source side of it has any tools for it. I've heard rumors that something like Duplicity will support Tahoe as a backend, but I haven't looked into it.

But, this would take care of the off-site secure replication component.

As long as you don't lose your crypto key to the data when your house burns down, of course...

Sean

Different kinds of backups for different failures.

[kabloom]

We must lay out the kinds of failures and goals of a backup to determine how best to back up.

1. We would like to protect against mechanical drive failure. This can be done with a RAID.

1.5. We may also want to protect against the failure of other components of the computer. I recently had a computer die because its motherboard died, and it took about two weeks to get a new computer, and the new computer was a significant upgrade so it had SATA instead of IDE. In the mean time, I needed my data on other systems, and when the new computer came, I needed to borrow a USB-IDE bridge to recover some stuff that I wasn't backing up.

2. We would like to protect against accidental deletion of files, file corruption, or edits to a file that we have now reconsidered. This can be done with snapshotting. In source code, to reconsider and edit to a file is fairly common, and is the reason why most programming projects use revision control systems. Other options like nilfs or ZFS snapshots can also fill this goal. This goal is accomplished more easily if the backups area automatic and the backup device is live on the system.

Depending on your needs, this goal may be counterbalanced by a need to not retain the history of files for legal or other reasons, and this should inform your choice of backup strategy.

3. We would like to protect against filesystem corruption, whether by an OS bug, or by accidentally doing cat /dev/random > /dev/hda. This can be done by having an extra drive of some sort that isn't normally hooked up to the computer. Tape drives, CDs, and DVDs have traditionally fulfilled this purpose, and this is where the use of additional hard drives is being suggested. Remote backups, via rsync can also accomplish this. For this I use git.

4. We would like to protect against natural disasters. For someone living in New Orleans, it would be nice to have a backup somewhere outside the path of Hurricane Katrina. Remote backups may be pretty much the only way to accomplish this, unless you're a frequent traveler and can hand-deliver backup media to remote locations.

5. In addition to any of the above, the code you use create said backup may be buggy, or may become buggy or misconfigured over time. Checking the integrity and restorability of your backups after creating them, and keeping several (independent) previous versions of a backup may help here.

You may not be concerned with the various modes of failure described here occuring simultaneously. For example, it may be unlikely that you need to deal with file system corruption at the same time that you regret one of the edits you made on your file. In that case, your offline backup device doesn't need to hold all of your snapshots.

Encrypt and twitter

[Anonymous Coward]

Just encrypt and twitter your backups. As long as you use decent encryption, should be fine for a decade or 2.

rdiff-backup

[Anonymous Coward]

I just rdiff-backup to an external drive connected by USB every night. Just put the command in cron.daily and it will run at about 3:00am. If you have to take action yourself to get a backup, you aren't doing it right. You need to have a process that runs for you so you can forget about it. Rdiff-backup provides a capability like Apple's time machine so you can get snapshots of data from past days. It is available for linux, mac, maybe windoes. The external drive is in case some tragic event happens to the computer and fries the components, the external drive may still survive.

When the backup got too big for my old external drive, i just swapped it for a bigger one. The old one then became an "archival backup" that I can save for long term storage.

Tape is expensive and can be unreliable

[VincenzoRomano]

Can you find some thing that can retain data for, say, 40+ years and be still able to restore them?
Everything comes to a cost. If the tapes are too expensive, then just buy the same storage you are using now and make just a second copy. But that would not be a backup. Just a second copy as unreliable as the original.

rsync

[Anonymous Coward]

My files are rsync'ed every night to an off-site other machine. And the other machine's data are rsync'ed to mine. After the rsync, the remote machine creates an incremental tarball of all files that have changed since the last tarball. Occasionally, two consecutive tar increments are merged into one to save space.

Now go away or I shall taunt you a second time

[Eil]

I hate to be that bitter old pessimist, but this has been debated to death and back here on Slashdot many times over. I swear, it should be in the FAQ by now.

• There is not one backup strategy that covers all situations, even if you think there ought to be.
• You have to do the work to find one that fits your needs, or hire someone else to.
• Cheap, easy, reliable. Choose any two.
• Slashdot: Not your personal army.

All of the times this question has come up (feels like at least once a month), there have been many very good suggestions. Why should we rehash them for the nth time?

Re:Now go away or I shall taunt you a second time

[darpo]

Other questions that come up all the time:

• I'm an aging IT guy, should I go into management or stay technical?
• What hardware/Internet connect/etc. should I use in some backwards 3rd world country?
• Should I go to college or work/self study?
• College X uses Java in its classes, College Y uses C++, which is better?
• Why am I such a big, fat nerd?
• How do I get experience when no one will hire me?
• How do I get work in the computer games industry? (related question: am I a closet masochist?)
• How should I, as some lazy, dipshit computer nerd, get exercise?

Re:Now go away or I shall taunt you a second time

[mlts]

The problem is that there is no good single solution. Just like there is no one single lock that would secure a building. You have to have multiple backup solutions, with some being onsite for fast rebuilds in case of RAID failure or malware infection, and slower, offline backups that takes a bit to get to, but are insulated from a severe and through attack by a blackhat or malware.

First stage: Consider mirroring of OS partitions, and if your OS/filesystem supports snapshots, have it do those often. One can do this in Windows Server 2003 and 2008 quite easily per volume, Vista is a bit harder, but it does have some snapshot functionality. This layer is for recovering deleted/corrupted files or directories without having to reach for restore media.

Second stage: Have an external drive connected for Time Machine, Vista's file backup, Acronis TrueImage, and set to backup at least nightly. This layer is intended for completely rebuilding a machine due to hardware failure, or to restore files that are not stored as a filesystem snapshot.

Third stage: Have a dedicated backup server, running some network backup program (Retrospect, Backup Exec, Networker, TSM), and make sure the backup server not just is hardened from network attacks because it has all the crown jewels, but the drives are encrypted so its theft would not be catastrophic. One can use EncFS, loopback mounts, TrueCrypt, BitLocker, PGP, or program of choice for this. I leave the data volumes unencrypted for speed, encrypt the boot volume, and use the backup program's encryption, so if someone steals the RAID array, it will be little more than a hardware gain. On the backup server, have a DVD writer available, ISOs of OS media, and a database of CD keys so a machine can be installed and reactivated without having to hunt for the right string of characters.

Fourth stage: Disk to disk to tape. You have a tape drive (or a library) attached to the backup server. This way, machines can back to a RAID array, minimizing the backup window, but you can get backups moved to tape or perhaps external hard disks for safekeeping and offline storage. The advantage of tape is that you can physically set the cartridges read only, so should the backup server get infected, it cannot alter data on the media. Same with tape drives that use WORM functionality like DLTIce. Consider a good tape rotation, and offsiting the tapes. Also, when offsiting a bin of tapes, don't forget to include a DVD with media and license keys for both the OS and the backup program to be able to start restores. Tape is expensive, but there are removable hard disk solutions to mimic the good part of tape usage such as what RDX technology sells (such as write protection).

Fifth stage: Offsite backups. I have had great luck with Mozy, and you can specify it to use a dedicated keyfile for encryption, so even if someone breaks into the Mozy account, without that keyfile, the data is useless to them. Of course, I keep a copy of the keyfile offsite and encrypted with a long passphrase to prevent chicken/egg scenarios.

Sixth stage: Critical files and documents (tax records, legal info) copied to a DVD-R or USB flash drive every quarter or so and stored in a format decodable in any platform. This includes the Mozy keyfiles, the BitLocker recovery keys, ISO images of the TrueCrypt volume recovery disks and header files, PGP keys, and any other cryptographic info. Since gpg files are decodable by almost any OS out there, I use that. This way, should real disaster happen, at least critical records would be recoverable.

Seventh stage: Hard copy printouts of critical documents (payroll, tax records) every six months to year. This even includes source code trees. These are boxed and stuck in storage. This is the extreme of offline copying, but it ensures that data will be around and usable should something like a long nationwide power outage occour (on the order of months to years). This is rather extreme, but it allows a company to go back to paper

Re:Now go away or I shall taunt you a second time

[FlyHelicopters]

"Cheap, easy, reliable. Choose any two" I disagree... I use Mozy, it is cheap ($5/month), easy (it runs in the background), reliable (I have tested restores, they work). What more could you want? Now, I'm not backing up CIA info here... If I was I'd do more. This is family pictures, and this is a better solution than I used to have, which was none.

Re:Now go away or I shall taunt you a second time

[Anonymous Coward]

Why am I such a big, fat nerd?
How should I, as some lazy, dipshit computer nerd, get exercise?

Too funny but, the body should be treated like any other important piece of hardware.
Take care of it unless you can aford to replace it.
There are lots of manuals so read up.

 

Toshiba will sell Blu-ray.

[Futurepower(R)]

Blu-ray will become cheaper as soon as the new Blu-ray competition gets started.

Oblig. Futurama quote...

[Dusty101]

Bender: "Hey, what's this? Hermes' dreadlocks, and his arm? Leela, I'm shocked! Food goes in the disposal, hair and flesh go in the trash."

Home NAS coming into it's own now

[djdevon3]

I really think a home file server (NAS) is the way to go now. Yes, still hard drives. If you really want to keep your data safe you should plan on purchasing duplicate drives each time you shop. 1 will remain in your home NAS. The NAS could be as simple as an old 800 mhz computer that you throw into a case with 10 internal bays. Personally I use an open source solution called SyncBack. It's extremely basic but it allows me to backup any drive connected to my home network to my NAS. Similarly, there are a lot of suggestions for Acronis. I like my SyncBack just fine.

Dan

[Anonymous Coward]

Back it up to a server on the moon

Drives expanding faster than my data does

[vlm]

But the one thing that's remained constant is hard drives are far too large to backup via conventional means.

Since drives are expanding faster than my critical data, all my computers and external drives have the same directory unison synced to a master copy on my mail fileserver.

Machine specific files live in a subdirectory of the shared backup directory. For example ~/backup/server or ~/backup/mythtv

Task specific files live in a subdirectory of the shared backup directory. For example ~/backup/genealogy

Sensitive files are mcrypt'd to a .nc file. For example, financial spreadsheets, scans of important documents.

Portable devices generally spend most of their time far away from the computers or are clipped to my keychain.

I also occasionally manually run a handy backup script on my computers to copy /etc/network/interfaces to ~/backup/$machine-name/etc/network/interfaces which is later synced. I backup about 10 config files on all my computers, and more on special purpose machines.

I have no idea if unison and mcrypt are available on non-linux machines... I don't use windows or mac to do anything important, only to play games.

So, pretty much every mass storage device I own has a complete and full backup of all my important, partially encrypted, data. As of this morning, all my important stuff is about six hundred megs.

I do pretty much the same thing with bulk media such as music collection, etc.

People whom are digital packrats might have trouble with this lifestyle.

automated

[Anonymous Coward]

The first pre-requisite is that it's automated.

Whether that's through cron or Windows Service or whatever, it needs to happen transparency at a regular interval, because otherwise people will forget about it.

And setting up a calendar remind doesn't count, as it will be simply delayed/dismissed. At most you may have a reminder to swap offsite disks (put the current one in you bag/briefcase NOW to take away).

Re:Do we have to bring this up over and over again

[iamhassi]

"Another hard drive?"

Yes. Tape has not kept up with hard drives, in fact they're now more per gigabyte than hard drives, very different from 90s prices. Even just the tapes themselves are more expensive than hard drives per gigabyte, and that doesn't include the thousands it can cost for a multiple terabyte autoloader tape drive.

There is really nothing else as cheap as hard drives per gigabyte. I use a external USB2 SATA dock and swap a few SATA drives. And honestly I'm not all that worried about backing up with modern operating systems. We've come a long away from Windows 95, where I was restoring from my Eagle Travan-3 1.6/3.2gb once a month.

Re:I use Jungledisk / Amazon S3

[wintermute3]

Jungledisk is a nice front-end to Amazon S3 for which I pay around $1 a month. The transfer fees for S3 are irrelevant over time, and I end up paying around $15 a month to back up my entire personal dataset from 3 different machines. I can get to any of my stuff from anywhere if needed, and the incremental backup system keeps things current with no overt action from me. I include my personal mp3 collection (big), digital photos, and everything else I care about. I spend a lot more than $15 a month on soda, so the $15 seems like a bargain to me.

There may be cheaper solutions out there, but why bother? This one is plenty cheap enough, and works great for me.

How can you trust online backup?

[HalAtWork]

Online backup solutions are only as safe as the administrators make it, and they have to rely on a trusted method. What method do they use, and why is it any better than one we use ourselves (i.e. a redundant backup in a different location using said method)? What's more, online backup solutions could put our data at risk in other ways.

Re:How can you trust online backup?

[Swampash]

Every backup method has its own advantages and its own vulnerabilities. If you're backing up to an external drive, the drive could die suddenly and unexpectedly; if you're backing up to a second machine, you're at the mercy of hardware failures on that machine (PSU, memory, its own disks, etc); if you're backing up to an online service like Amazon S3, you have to be prepared for your data to be unavailable with no warning (anything from local ISP problems to the service going out of business).

The only sensible thing to do is use ALL of them, and hope that the advantages of all of them cover your ass for the vulnerabilities of one in particular.

(FWIW I back up to an external drive every fifteen minutes, to another machine on my LAN hourly, and to an online service nightly.)

Re:Do we have to bring this up over and over again

[johnw]

That's the protocol. Now what media do you recommend? Another hard drive?

Personally I use a dedicated backup server which has 2 x 1TB discs in a RAID1 configuration. The cron-scheduled backup uses Dirvish to create multiple historic backups, and the backup server is in a different building from the machines which it is backing up.

Not quite as good as multiple historic tapes, but have you seen the price of 1 TB tapes these days?

Accidental file deletion Check
Disc failure Check
File system corruption Check
Fire damage Check

Obviously there are other risks which it doesn't cover, but you sometimes have to ask yourself just how important is your data? I'm not running a business these days.

Rsync with another hard drive

[Anonymous Coward]

Rsync -av --delete my-hard-drive offsite-hard-drive

Simple.
Effective.

Re:Do we have to bring this up over and over again

[Anonymous Coward]

*GASP!*

It's N-sync!

backups and FUSE

[Anonymous Coward]

... Amazon S3 mounted on FUSE. USD $0.17 per gigabyte to store offsite, and with FUSE I can browse it like it's local. I am very happy, and saving money too.

What happens if some malware comes along and does an 'rm -rf [...]' at the location of your FUSE mount point?

I think one pre-requisite for a proper backup is that it's offline (or somehow detached from the live system). If it's online then the 'backup data' can be taken out just like your 'regular' data in some cases.

Of course it's up to you to determine how much of a risk this is, but for me swapping external hard drives once a week between home and work is quicker and easier.

Hit by lightning

[kshkval]

That's exactly what happened to us... the hit took out 2 Macs and the power bricks/adapters for nearly everything else electronic in the house. And it was a strike across the street that travelled thru the dsl line, not the well-protected outlets. I always have at least one backup NOT connected and stored off-site since then. The other awful thing that convinced me to use 3 external drives was backing up to a single drive and having a bad thing happen to both the main drive in my PowerBook and the backup at the same time. The screwup was a funky restore from backup (I'll never use Intego Personal Backup again). Yes, the stupid things happen and you'd better be ready...

Re:Hit by lightning

[Helios1182]

Or, someone breaks a window and walks out with your computer and external hard drive. Having a full backup off site makes recovering from a burglary slightly less awful.

Re:Hit by lightning

[kshkval]

Yep, that's exactly what I thought as well after my next door neighbor had a break-in. It's a safe area, but why risk 10 years of digital photos? We have the safe deposit box anyway, so I bought an OWC drive that fits into it like a Russian nested doll. I back up all 3 Macs on the partitioned drive. The safe deposit box costs about $55/year, cheap at the price.

Re:Hit by lightning

[TheLink]

I've got my dsl line protected by having 250mA fast blow fuses on each wire (you can pick lower amp ratings if you're paranoid), and gas discharge tubes, MOVs across the lines to ground, after the fuses. So when there's a big enough lightning induced surge, I just have to change the fuses (once I had to change the RJ11 connectors and sockets - they got welded together - but the modem was OK ;) ).

So far my previous adsl modem only failed after many years and it failed after a power failure, not a storm. So maybe it was a power line surge that did it, or it failed due to stress of powering up. Thing is, it was on a UPS and that should have protected it from the power supply.

But I'm well aware that if lightning actually hits stuff, all that might not help at all - everything could just be vaporized or burnt up.

I'm personally not very comfortable with "fancy backup software" at least for personal use. Copying files is fine. dd + lzop for imaging is fine (careful to have conv=sync if you use noerror - using noerror is a good idea since if stuff is just starting to fail, you probably want to start copying as much as you can out ASAP).

Never rebuild your hard drive

[Goldenhawk]

Yeah, "another hard drive" is a good system. My policy is to never rebuild a flaky hard drive. If Windows goes wonky, which seems to happen every so often (with four XP machines at home, I seem to need to do some form of system repair every year at least), I just buy a new hard drive and either restore to it or start from scratch.

Then, the older hard drive gets shelved. That way, I have at least a snapshot in time, even if it's a few years old, and even if Windows won't boot I do have the critical files accessible. It's easy to access them by sticking the drive in a USB external shell.

But my DAILY backup routine is Norton Ghost doing weekly baselines and nightly incrementals, to external USB drives. This saved me many hours of rebuilding my wife's machine this week, when XP encountered the dreaded MUP.SYS freeze on bootup. I simply bought a new (larger) hard drive, shelved the dead one, and restored from Norton Ghost's backup. Aside from a few hours of MUP.SYS troubleshooting (to determine it WAS a dead drive and not some wonky peripheral), total restore time to a "just like she left it" system was a few hours, mostly waiting for the 100+ Gb to transfer from the USB external.

Finally, every so often I dupe my backup externals to a second external. These "safety" backups get stored in a second building on my property, so I have fire insurance.

Regardless of medium ... remember the rules

[urbanotter]

Rule #1 - It's not backup until its out of the building (3.5 kilometres away for insurance purposes) Rule #2 - Do 2 backups with different brands of software ( system & retail ) Rule #3 - Do a restore frequently to check the backup (don't trust the logs)

My solution: NAS as backup store + offsite second

[Ptur]

I just got myself a QNAP NAS (http://www.qnap.com/) on which all my systems backup. That NAS gets replicated to a USB disk from time to time, and I keep that disk offsite. My plan is to get rid of the USB disk and buy a second QNAP NAS to put at the remote site, and let them replicate over the internet. Reason: I can schedule that so it is not forgotten, and no more hassle of moving disks offsite.
Reason I went with QNAP: runs linux and is hackable out of the box :)

What I Do

[Anonymous Coward]

I back up all my hard drives with external hard drives. I have two internal RAID 0 and I back up each of them with a single encrypted hard drive (example two 640's backed up to 1.5T, two 500's to a 1TB). I then stick them in a copper mesh lined, fire and water proof safe.

I'm pondering building an ejection seat/escape pod to activate whenever the safe is in danger. But then I'd have a giant hole in my roof that I'd have to explain to my landlord.

3 HDs and 1 safe deposit box

[Charl`za]

I've run with two hard drives of equal capacity for around 8 years. For a while these were stored on a linux box. Every once in a while I'd simply run a cp with update to basically move only changed/new files. This saved me many times. These days I have 1 internal 1TB HD and 1 external 1TB HD in my Windows desktop. I'm eventually going to purchase a third external 1TB HD and swap it out in my safe deposit box at my bank. Luckily, my bank is across the street, so swapping that will be no problem and take very little time. I don't plan to do this once a week...maybe once every 4-5 weeks. The odds of a hard drive crash are probably greater than my apartment burning down anyway.

tape != unreliable

[russlar]

Can you find some thing that can retain data for, say, 40+ years and be still able to restore them?

The only reason those tapes failed is that some idiot decided to erase them.

Distributed File Solution.

[dcray2000]

The only true redundant solution is DFS and DFS Replication with the addition of Shadowcopy. If anyone knows of a linux option that rivals these please speak up, I've been looking for one for years. RAID is just a really large single point of failure.

DFSR gives you the option to replicate live data and it works REALLY well. Plus you can add extra replicas on a replication lag. You could even have an extra replica on a replication lag across a VPN link to someone else's house. There are other off-site options like Mozy which work well.

Of course, this is all pretty complex. When it's working, though, your data is pretty safe.

Re:Distributed File Solution.

[Todd Knarr]

Replication isn't useful for backup. Having a copy of the current data is utterly useless if that data's been damaged. Replication will simply have insured that the replica is also damaged. Replication lag doesn't improve the situation either. Half of the point of backups is to be able to go back and get the data as it was 4 days, or 4 weeks, or even 4 months ago, and to be able to pick reasonably close to any arbitrary point in the past. A good backup solution will let you simultaneously deal with the file someone deleted this morning by retrieving yesterday's copy and with the software configuration change you made 2 weeks ago that now is turning out to be a nightmare and messing up your data and you need to get your old configuration back plus take the data from right before the change and re-apply the last 2 weeks' work to it.

Re:Distributed File Solution.

[dcray2000]

That is why I noted Shadowcopy in the first sentence. Also, the extra replica with a replication delay assures even more coverage of the issue you noted.

Re:Distributed File Solution.

[cyberdrop]

We previusly had a solution using DFS-Replication, but DFS-Replication is not realy build for this kind of use.
We switched to a Microsoft DMP Server 2007 which uses agents on each server and provides a nearly realtime replica with daily snapshots of Files, Exchange, Databases,... Users can restore files on their own and there is virtually no admin work for daily use.
We make monthly manual off-site backups of the DMP-Store with Windows Backup on separate Harddisks.

For this solution you need a seperate server with about as twise as much harddisk space as your normal backup size. The daily snapshots lasts about two or three months.

Re:Distributed File Solution.

[dcray2000]

That's great information @cyberdrop. Especially for an Enterprise level solution. However, I think the original post was about the "Best Home Backup Strategy."

Also, I'm assuming by DMP you mean DPM, Data Protection Manager as a part their System Center solution. As for DFS(R) not being built for this "kind" of use, please elaborate? We use it in an enterprise of over 100K machines. I also know several people who use it at home for the exact purpose the original article was asking about. They all rave about how well it works, especially with Shadowcopy.

Re:Distributed File Solution.

[cyberdrop]

Sorry, I meant DPM. And shure it might be overkill for home use ;-)

DFS Replication was originally designed for some kind of load balancing if I recall correctly.
We had some issues, especialy when restarting servers, where replication did not start correctly. We had also issues with locked files and DFS Replication is no solution for backing up database and exchange servers...

Re:Distributed File Solution.

[dcray2000]

DFSR was really just meant to replace FRS, which was a horrifying. Overall they did a pretty good job. MS says you can use it for redundant file sharing, however, this doesn't take into account more than one user on different replicas changing the file. In the end, the last saving user wins out. Their statement is really only true when only leave one replica in referral at a time. Then you shuffle users around without them knowing. For a home user you can weight everyone to a main instance, then if it's down for any reason they will automatically go to the other replicas until it's back.

It is pretty good for load balancing or just namespace virtualization. It's also great for login script virtualization, gets them out of SysVol. I'm really sorry to hear it was problematic for you, we've had no issues with it on R2.

When it comes to backing up databases, though, it depends on what you are doing. I totally agree you are better off shadowing databases in the enterprise with DPM, that's what it's meant for!

Not LVM!

[Anonymous Coward]

Don't tell us you're using LVM for critical data such as backups.

LVM does not implement file system barriers.

Betcha didn't know that, did you? Scary, ain't it?

Heck, might as well google for why barriers aren't enabled by default in Linux....

Re:Not LVM!

[Nutria]

Don't tell us you're using LVM for critical data such as backups.

Why not? Such volumes are only powered up and mounted for a short time every so often.

LVM does not implement file system barriers.

Either lvm2 silently ignores barriers, or Kernel 2.6.30 and lvm2 2.02.44 now implement barriers:

[ 287.501633] EXT4-fs: barriers enabled
[ 287.519049] kjournald2 starting: pid 4891, dev dm-1:8, commit interval 5 seconds
[ 287.519457] EXT4 FS on dm-1, internal journal on dm-1:8
[ 287.519460] EXT4-fs: delayed allocation enabled
[ 287.519461] EXT4-fs: file extents enabled
[ 287.840890] EXT4-fs: mballoc enabled
[ 287.840910] EXT4-fs: mounted filesystem dm-1 with ordered data mode

Re:Not LVM!

[arkane1234]

LVM is a totally different layer. RAID is below that layer.
You perform software or hardware RAID below LVM, just like you would with Ext2/3, xfs, etc.

Safest home backup solution

[OzFalcon]

Safest home backup solution

Use standard commodity parts for easy replacement/reconstruction.
ie. Consumer grade writable disc's and external hard drives.


Backup Home data to DVD/BR/CD regularly (Monthly)

Backup Bulk data with 3 Hard disks!!!!

1) Use one Live HD

2) Use one to backup Live

3) Use ANOTHER to backup Live again

The Reason?
If your backup is large, You will probably need to erase the target drive.
At this point you are VERY vulnerable.
With your only backup erased,
You had better hope your live drive doesn't fail while backing up again.
Or you may accidentally delete something from the live drive WHILE backing up.

With a 3rd hard drive, At least you can recover from your last backup if your current backup somehow nukes itself.


Dangers....
1) Don't have a live drive and think your safe with another live drive synced, You really want to keep hard drive backups IN
STORAGE. If it's live it can and probably will die. It's just a matter of time or circumstances. This goes for raid too.
2) This is home storage, So 6/12 month hard disk backups should root out any bit rot or hard disk seizures.

Re:Safest home backup solution

[WuphonsReach]

No need to erase the target on Windows. Use something like "Second Copy 7" (the alternative on Linux would be an rsync style tool).

You can even tell it to put changed/deleted files into another location, and to keep up to N-versions of changed files.

Combine that with Acronis True Image (or Norton Ghost, or even a Linux boot CD with the proper tools) to make an image of your operating system drive periodically.

Re:Do we have to bring this up over and over again

[Hymer]

"What about the other 95%?" Over the years I became an old and bitter sysadm... you know what ? They just need to do what the 5% did: Put their asses in a chair and Read The Fucking Manual... and read again, and again until they understand the subject.
No, I'm not user friendly, I do not need to be... people are asking me for help anyway.

Frequent duplication is NOT the answer

[Peter (Professor) Fo]

Hot copy, nightly duplication etc. may protect against catastrophic system failure, theft etc. but they are poison if you want to protect against lurking data corruption.

For example you deleted a file last week that you need now. A supplier has sent this month's data with last month's filenames and overwritten last month's data. Your database has 'acquired' corruptions and now you need to go back to find a working or clean version. A duplication strategy just means you have two copies of the bad stuff!

Here is what I do

1. Complete snapshot to USB hard drive 'monthly'. (Actually just an update excluding delete)
2. Selected directories and files: (daily)
   • Copied to USB flash if changed or new
   • Copied to a history chain on the same HD if changed or new

The history chain has extending time gaps between copies eg 0.9,1,3,7,15,30,90 days. So for a daily backup of a file that changes every day the two most recent copies are always shifted down the chain and every 3 days the second is bumped down to 3rd position which in turn might bump 3 to 4 if 3 is more than a week newer than 4. This is ever so easy to program - I even did it in a DOS batch file.

Let's review what happens if the computer goes bang! - Reload from USB hard drive and flash. Alternatively if data gets corrupted - Trawl through the history on the HD.

Re:Frequent duplication is NOT the answer

[careysb]

This touches on one of my pet issues: how do you tell when a file gets pinged by bit rot? It would seem that you'd want to do some sort of comparison between an entire file's contents and a signature file. That you'd want to do this on both your primary storage and your backup storage. Maybe using something like an MD5 signature or maybe something like Reed-Solomon codes which might give you a chance of recovering a damaged file.

I use file/directory mirroring instead of proprietary methods and file format. This saved my butt when my XP system died and my new system was Vista.

Currently I use "Karen's Replicator" (freeware) to make mirror images of my data directories. I wish KR had some sort of bit-rot test built in. I've been contemplating writing my own.

I also make occasional total system backups but not as frequently as I backup my data.

Carey

Re:Frequent duplication is NOT the answer

[1s44c]

The history chain has extending time gaps between copies eg 0.9,1,3,7,15,30,90 days.

That's exactly the kind of thing that BackupPC does, plus it does file pooling and compression.

I used to write all kinds of funky stuff to do backups but now I just use BackupPC and it all works better.

Re:Frequent duplication is NOT the answer

[mlts]

Most backup programs keep some type of checksum to check if files in a media set got damaged. However, if you want error correction, you likely will have to use an archiver like WinRAR, StuffIt Deluxe, QuickPAR, DVDisaster, or another utility that has this functionality built in. I have yet to encounter a backup app that has an option for Reed-Solomon ECC codes.

Re:Frequent duplication is NOT the answer

[WuphonsReach]

Hot copy, nightly duplication etc. may protect against catastrophic system failure, theft etc. but they are poison if you want to protect against lurking data corruption.

rdiff-backup on Linux or Second Copy 7 on Windows.

Both allow you to keep previous versions of changed files.

You can also do something similar with rsync or rsnapshot, but I prefer the way rdiff-backup works.

Re:Frequent duplication is NOT the answer

[schweinhund]

Along these same lines I have been hoping to see that some Java wizard will create an applet for Azureus/Vuze that will run an MD5 checksum against the completed data and leave an MD5 file in the directory. That way I don't have to manually run MD5s manually since I include them with *everything*.

Something like this:

Torrent DL ==> Check Pieces ==> Completed ==> Run MD5 ==> Leave MD5 File

Another idea is a workable MD5 (+SHA1/*) checksum generator for X that supports drag and drop, queueing, etc. along the lines of the Win32 app hkSFV or md5summer. md5summer is not bad, but it needs drag and drop and context menus.

Moderation

[xrayspx]

This is the only story where being modded "Redundant" is a laudable goal.

Personally, In-home backup to mirrored Iomega NAS, and offsite to a USB enclosure which goes away weekly.

This is an ugly but workable script I wrote for linux/mac to make multiple snapshot backups where the data set is rather smaller than the target media. That's not the case with me, or probably most of slashdot, but it works for Moms and laptops.

Re:Do we have to bring this up over and over again

[node 3]

On the other hand, if you thought you could ask on /. you probably match this description...

If you had to ask on /., you already don't match the description.

Off-site backup rotation

[tepples]

Of course, this won't help you if there's a house fire. The fireproof hard drives are still darned expensive.

Data backed up to a set of USB hard drives in rotation, with no more than one in the same building at the same time, should survive a burning building as long as it's not part of a fire of 1871 scale.

Re:Do we have to bring this up over and over again

[node 3]

"What about the other 95%?" Over the years I became an old and bitter sysadm... you know what ? They just need to do what the 5% did: Put their asses in a chair and Read The Fucking Manual... and read again, and again until they understand the subject.

That's not what they did.

First, they were born/nurtured in such a way to have above average technical aptitude.

Second, they were interested enough in how computers work to tinker and learn and gain a broad base of knowledge about their computer and OS.

Only then did they "put their asses in a chair and Read The Fucking Manual... and read again, and again until they understand the subject."

If you expect the 95% who did not go through the first two parts to skip right over into the third part, you're in dire need of taking your ass out of the chair and meeting some Real Fucking People.

No, I'm not user friendly, I do not need to be... people are asking me for help anyway.

Do what I do. Tell them, "yes, there's a way but it's rather complex. Do you want me to explain it?" The answer is almost always "no". Because they really don't want you to explain it, they want you to do it. If they say yes, you'll probably be asked to stop in less than 60 seconds.

Delayed mirror external HDDs and Gmail

[darpo]

1. For my media I do two 1 TB external HDDs with a robocopy run every 48 hours (to avoid the "oops, I deleted something I shouldn't have" RAID 1 problem)

2. For my cannot-lose-it small documents (KeePassX file, financial spreadsheets, etc.) I put them in an encrypted archive format and upload periodically to Gmail, to work as an offsite backup.

It'd be sweet if I could simply back up everything offsite, but with only 100 KB/s of upload bandwidth to work with on my DSL connection, that's just not feasible.

Ubuntu has Back In Time

[El_Oscuro]

Ubuntu users are blessed with Back in time. All it is is a gui interface to rsync, but the gui is quite nice. Since the rsync keeps the same directory structure, you can always check your backups by simply viewing the files on the backup drive.

Nope.

[MooUK]

Is there any better advice these days?

Put simply: No, there isn't.

No and yes.

[delphi125]

Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week. Is there any better advice these days?"

There isn't any better advice, but you ALWAYS have "really important" data.

The most likely things you are going to want to back up are documents and spreadsheets, pictures, videos and of course code. If you can't afford more than one external drive, or even don't want to spend anything at all, the big G (yeah, I'm a fanboy, but there are probably equivalent options) provides help. Google Docs for the first two: search for "google docs synch" and the first option is freeware (not that I've used it - nowadays I just use Docs itself). Picasa allows you to keep pictures unpublished, not so sure about youtube etc. And code you can mail to yourself, (g)zipped, although it might be on your home test/dev machine and also on your commercial web server.

Only backup essential data, not programs.

[olsmeister]

I store and backup all my "couldn't live without" files in a particular place, and back it up regularly to a

Dirvish is a great tool!

[GPLHost-Thomas]

I agree that HDD is the best solution. But as for the software, dirvish has our preference: it does nice incremental rsync backups over ssh, using hardlinks for files that are identical. It also warns you when a backup fails.

Re:Do we have to bring this up over and over again

[jonbryce]

They should use Time Machine.

Re:Do we have to bring this up over and over again

[Voyager529]

Where are my mod points when I need them...+65535 insightful

RAID good if used properly

[Carl Jacobsen]

Yes, "RAID is not backup", in that you shouldn't simply RAID your primary drive and consider the backup problem solved, but backing up to a RAID array can be advantageous -- you do disk-to-disk backup (via any of a variety of methods), and monitor the health of the RAID array closely -- if any disk in the array goes south, replace it promptly and your backup stays consistent. And, if you keep a spare drive or two around, you can swap a drive out occasionally to take off-site (and let the array rebuild onto one of your spares).

Personally, I like the ReadyNAS Duo a lot more than the Drobo (hard to explain, I just trust their tech better, and the ReadyNAS is natively networked, rather than needing an afterthought add-on). Last I checked, Amazon will sell you an empty ReadyNAS Duo and a couple WD Green 1TB drives for ballpark $500. That said I haven't got a ReadyNAS yet (because money has so many uses these days); I'm using my second most favorite backup setup, a 500GB laptop drive in an external bus-powered FireWire enclosure. I'm using a MacAlly PHR-S250CC enclosure (which I'm very happy with), using a drive I already had, but for a complete setup, I'd probably go with one of Other World Computing's packages for about $150. This loses RAID (which I ultimately want very much to have, for reliability), and isn't networked (which would be good for backing up multiple machines, and ease of use), but the bus-powered drive is so damned easy to use that I actually do it every day (set the drive next to my laptop and plug one cable between them, Time Machine notices the drive and starts a backup, 5-10 minutes later it's done, and I unmount the drive, unplug the cable, and put it back on the shelf).

My primary machine is a Mac; I use Time Machine for daily backups, and use SuperDuper to clone my MBP's drive onto the same backup disk every few weeks (minus a number of large directories that I know Time Machine is getting anyway); this gives me a backup drive I can boot from (via SuperDuper), and a lot of incremental history stored in a very usable manner (via Time Machine). And a backup system that I actually use because it's painless.

Add a ReadyNAS, and I could have my laptop automatically backing (hourly) up any time it's on the home network.

As far as on-line backup goes, I haven't been convinced yet. It eats a lot of bandwidth, and it means that someone else (that I don't know personally) has a copy of all my data, with only their promise of encryption keeping them honest. Sure, there isn't much there for anyone else to get worked up about (a variety of legally purchased music and software, a bunch of old email and vacation photos), but if it's not out of my hands, then that's one less thing I have to worry about. I do love DropBox for moving non-confidential files around, but I wouldn't use it for backup.

Time Machine works well

[Helios1182]

I use Time Machine + 2 external drives for my laptop. One drive is in my lab at school, the other drive is at home. It gets synced to both regularly and I have an offsite solution in case one burns down or gets stolen. If you have a desktop you could always swap external drives once a week and keep one at the office or other remote place. If you were more paranoid you could leave one in a safe deposit box as well -- a 1TB external drive is all of $100 now.

Two types of data

[Anonymous Coward]

Like others have said, there are usually a few different types of data you have.

I categorize mine as follows

1) Entertainment. Music, Movies etc

2) Important stuff. Photos, cv, code etc

For me, there is no need to back up 1. If I lose it, I can buy it again or do without. 2 must be backed up. Tar and compressed by date onto my one and only local HD. Each time I tar, I copy changes to a 16GB usb key. It's cheap and simple

Here's a partial list of my backup directory

gav@r2d2:/data/sda4/backups$ ls -l
total 2257188
-rw-r--r-- 1 gav gav 99853231 2009-02-08 11:38 2008.02.08.home.tar.gz
-rw-r--r-- 1 gav gav 235315200 2008-12-06 15:52 2008.12.06.docs.tar
-rw-r--r-- 1 gav gav 126540165 2009-01-04 10:55 2009.01.04.home.tar.gz
-rw-r--r-- 1 gav gav 328816640 2009-01-12 23:09 2009.01.12.home.tar
-rw-r--r-- 1 gav gav 98706437 2009-04-05 18:07 2009.04.05.devkitPro.zip
-rw-r--r-- 1 gav gav 6322458 2009-04-05 17:45 2009.04.05.documents.zip
-rw-r--r-- 1 gav gav 91485066 2009-04-05 18:06 2009.04.05.workspace.zip
-rw-r--r-- 1 gav gav 5239082 2009-04-22 22:12 2009.04.22.documents.tar.bz2
-rw-r--r-- 1 gav gav 249277582 2009-04-22 22:16 2009.04.22.home.tar.gz
-rw-r--r-- 1 gav gav 230007959 2009-05-17 10:43 2009.05.17.home.tar.gz
-rw-r--r-- 1 gav gav 574637109 2009-04-05 18:54 2009.05.17.photos.zip

mozy == No Linux support

[higuita]

mozy == No Linux support

until that happends, its trash for most of us

Multiple revisions and off-site mirror

[frambris]

Here's what I do.

Nightly (my data do not change that frequently) I backup important directories to another drive (used to be internal but is now an external 1TB disk). I have set rdiff-backup to keep 30 days worth of backwards diffs. I weekly mirror the latest copy to an off-site disk which happens to be connected to my workstation at work as they don't mind. I do not mirror the backwards diffs that rdiff-backup use to get old revisions of files.

This gives me screwup protection up to 30 days and theft/fire protection of the current data. The disk at work is encrypted. I thought of encrypting the external 1TB disk at home too but the disk of not being able to restore the primary backup outweighs the protection from datatheft if the disk should be stolen in a break-in.

No matter what backup software or script you use to backup your files see to it that you can go back at least a couple of revisions to be able to recover from the most common source of data corruption... you.

Disks are pretty cheap these days. Some might argue that the space needed for backups are too much, as much as the data you are backing up. Yes it often does but it's worth it. Once you have lost a great amount of files in a screwup or disk failure you gladly pay the price for an extra disk.

Now if I only could find a very user-friendly backup software for Windows that gives rdiff-backup functionality and is also wallet-friendly I could convince my father to start backing up his data.

OP is misinformed...

[mnslinky]

And as we've all had drilled into our heads 'RAID is not backup.'

What the phrase means is that simply installing a RAID on a given system is not a backup for that system. It is still more than acceptable to use *another* system, with a RAID or not, as the backup for the original system. For years, we've been running one large systems with tens of disks in a large RAID array as our backup system, for the other systems. The point of backups is to have the data in more than one place, regardless of where that place is. DVD, tape, an external disk, etc.

Drobo, or something similar, would be a perfect device for a home backup server.

Online? And let others see my files? No thanks.

[EWAdams]

I have data on my hard drive that is probably illegal in certain jurisdictions. Now that governments are prosecuting people for merely possessing information, like demolition training materials or photographs of one's own children playing in the bathtub, regardless of what jurisdiction you live in, there is no way I'm going to let that out of my physical control. If I encrypt it before uploading it, they'll just discover that it's encrypted and use the rubber hose attack until I give them the key.

I keep an external hard drive that's five times as big as my normal one, and do a daily backup keeping up to five previous copies. Of course most of them don't ever change, like executables, so the drive doesn't really need to be that big.

Linus' Solution

[Jonathan A]

Only wimps use tape backup: _real_ men just upload their important stuff on ftp, and let the rest of the world mirror it ;)
- Linus Torvalds

http://en.wikiquote.org/wiki/Linus_Torvalds

Re:Linus' Solution

[Lally Singh]

Oh good, I'll do the same thing for my Quicken data...

mod parent up

[godrik]

This is a pretty cool sumary of why and how to backup data. It deserves a +5 interesting.

WIndows Home Server + Offsite storage

[srwood]

WHS provides automatic backups, multiple images. Use an offiste storage solution such as Mozy for Uber important files.

Re:backuppc +1

[higuita]

I too use backuppc to a external HD to backup my main computer and my family systems

it do data deduplication (file level, not block level) and compression... its easy to use and works perfect.

i swap HDs every week

now backuppc needs to add a option for block level deduplication (usually is a little more efficient in size for many files) and, more important, upgrade its rsync protocol (the latest versions of rsync improved alot the protocol, faster and smarter)

Cheap and Good Offsite Backup

[LVSlushdat]

I have about 60GB of *stuff*.. the usual digital excrement, with about 6GB of absolutely irreplaceable stuff, so I rsync the whole 60GB of stuff to a USB 500GB drive, and to my home server nightly, and then once a week do a sync of the 6GB of REALLY important stuff to Amazon S3 via Jungledisk.. Two fantastic products for backing up stuff AND getting it offsite in case of disaster.. I used to use Mozy, but they don't support Linux and even if I was still on Windows, their backup manager is pretty braindead.. The cost for Amazon S3 is dirt-cheap.. My initial upload of the 6GB of stuff and 3 weeks of incrementals after made my first Amazon S3 bill a whopping $3.42.. Now with just incrementals, it averages around $1/mo... Don't have any financial interest in either of them, just like them...

aws.amazon.com
www.jungledisk.com

CrashPlan?

[Anonymous Coward]

I use CrashPlan. I set it up for my family and you can then backup to their PCs and they can backup to you for off-site backup (via Invites), likewise, you can still backup to an external drive internally. I've been using this method the past year and its been fabulous.

Re:CrashPlan?

[douggmc]

I use Crashplan too. It is free, easy, and here is the key ... easily implements redundancy. The whole point is you can backup to multiple "destinations". It has Mac, Windows, Linux, and Solaris clients. One of the cool things I do is run it headless on my Linux server. I can log into the GUI remotely using port forwarding from my Mac. Here is my backup scheme: Mac: a) Directly connected USB drive (which has 2 partitions ... I use one partition for Time Machine and one for Crashplan. This is my main/most important machine w/ pics, music, movies, documents, etc.). b) To my Linux file server. The file server is ClarkConnect (a Redhat/CentOS based Linux distro that does a bunch of cool things by the way). c) To my work laptop. Runs XP. Work laptop (WinXP): a) To my Mac. b) To my Linux file server. Netbook (running Win7): a) To my Linux file server (I hardly keep anything unique/important on it). Linux file server (ClarkConnect): a) To my friends file server. I let him use my Linux file server in return. b) My parents computer. They don't even know it. They use their ridiculously overpowered Dell with Vista as a "super magic email device" and Solitaire machine. I put in an extra big honking HD when they got ...and installed free Crashplan. It hums along quietly without them even noticing.

Network storage is the way to go.

[Anonymous Coward]

It depends on what you want to backup. I, for example, like to keep an image of my primary boot drive available for each of my systems just in case I do something to destroy my OS. For this, I have a 3.6TB RAID5 managed by Slackware 12.2 on a headless box. The RAID5 ensures that my backups won't be lost to a hard drive crash (the system has 5 1TB drives), and the amount of space allows me to have multiple backups of my primary systems (each is partitioned to have a 60GB partition for the OS/major programs and the rest of the space is available for games/other fun things). Fortunately, the 3.6TB is overkill, so I also use it as a media server for my house. By the way, I built that system for $700.

-Rigor Mortis

WHS

[kabdib]

Not popular on Slashdot, but I'm very happy with Windows Home Server. We've got a bunch of Windows machines around, and it images everything automatically. I do manual backups of important stuff (e.g., vacation pictures, mail archives) on DVD and thumb drives, and do a total clone of the WHS backup directory every two months (it's less than a terabyte, and fits comfortably on a cheap HD).

The Linux boxes just have source code, so a git push to a Windows box suffices there.

Backups are stored both off-site and in a local fire safe.

RAID *alone* is not backup.

[toby]

But RAID is a perfectly acceptable backup when it is a backup - as in a second system.

The first data protection should be to run ZFS with redundancy - that is already much safer than "RAID" (for reasons easily discoverable by brief web reading).

This can be mirrored to a second system.

Too Big to Fail

[masmullin]

Like the major banks, my HDD is too big to fail :)

My system of backup

[Anonymous Coward]

Personally, I have a two-fold system:

My real important data, textfiles, spreadsheets, business-related data, etc. (100MB at the moment) lies encrypted (via encfs) on my main machine, my laptop and another off-site-machine. The encrypted folders are synced via Dropbox.

My not-quite-so-important-data like, uhm, self-made and legal rips of my bought dvds and cds reside on external harddrives. Metadata of all of this (folder- and filenames, avi-metadata, varying checksums) is backed up with the real important data[tm](c) and I rely on the power of the internets to being able to resurrect my data in case of a crash.

This system clearly is not suited for an amateur video cutter, but in my and similar cases, where there's a few magnitudes of difference between the volume of irreplaceable and internet-enabled-reconstructable data, this method should work pretty good.

Re:Mozy is good, but they don't encrypt filenames

[alanfairless]

FYI, even if you use your own key, Mozy only encrypts the contents, not the filenames. That's rather insufficient. A court could subpoena them for a list of your files, establish that particular files exists, and require you to produce them. See http://michaelshadle.com/2007/05/07/mozy-the-backup-client-damn-close-but-still-no-cigar/

Plug: In 2006 I founded https://spideroak.com/ specifically to provide a zero-knowledge approach to online backups. We don't know anything about your data, including your file and foldernames. On the servers we just see sequentially numbered data blocks. It's written in Python and C and we've always supported Linux and OS X (and Windows if that's what you're into.) SpiderOak keeps historical versions of your files and deleted files forever (or until you decide to remove them) and will sync folders for you across several computers. Some reviews are http://www.linuxplanet.com/linuxplanet/reviews/6644/1/ and http://www.maclife.com/article/reviews/online_storage_battle_which_cloud_backup_service_reigns_supreme

I still like my old idea...

[Lazy Jones]

... of backing up stuff by e-mailing it to a forwarding loop using a bunch of free e-mail accounts/aliases.

Re:Do we have to bring this up over and over again

[Larryish]

We use rsync between the various machines in our home office. Every machine runs Linux except my daughter's, which runs XP (her Care Bears game doesn't like WINE).

I have a laptop machine, a desktop machine, and a headless fileserver for my personal use, and my wife has a laptop. That is 4 machines that are used to back up her work directory (she works from home in recruitment for a franchise company) as well as photos. I use them to back up code, emails, and local copies of websites. Everything is scheduled via rsync and cron to run in the wee hours.

I keep my ebooks collection on all 3 of my machines, I guess that qualifies as the medium category.

Unimportant stuff like music and video are kept strictly on the fileserver.

Periodically I zip up the work-related crap and upload it to a cheap VPS that also serves as a secondary DNS server. The VPS is less than $8 per month and comes with 2.5 gigs of storage and 100 gigs of transfer.

Multiple redundant nightly backups using existing hardware, with offsite backup for $8 per month. Can't beat it for SOHO.

to ext. usb hard drive /w partimage || pybackpack

[keneng]

I'm not a crazy hard drive filler like a lot of people but here's my attempt to make my life simple. Try not to fill your personal hard-drive with media you didn't create yourself: all sorts of movies, books, and mp3's. If you truly must have this stuff in archives, then put it on another hard-drive not mixed up with your personal stuff.
Your personal stuff:
-----------------------
1)do keep your emails in a dedicated directory named with the date of creation i.e. from20July2009email
2)do keep your personal pictures in another directory...ditto
3)do keep your personal videos in another directory
4)do keep your work related projects in another directory

There now we've got it simplified down to 4 main directories to backup.
I've successfully backup to external hard drive by simple directory copy because I don't have that many directories to deal with.
It takes about 20 minutes to backup while attending and observing it closely.
I've tried "pybackpack" recently which is in the ubuntu repositories. It's a gui for selecting the directories and keeping them in a backup project file. It's worth a look.

Now with regards to backup of your computer's customized "just for you" settings, well from what I've seen for Linux recently is partimage
http://en.wikipedia.org/wiki/Partimage
They've even got an article recently published a couple of weeks ago in the Chinese weekly computer newspaper called "Dian Nao Bao" which has dedicated half-page to Ubuntu every week. I can't understand a word of it except they do publish the english buzzwords in it like 3g, android, partimage, wardriving, and mplayer.

prefixsuffix is cool. It's a little tool that changes groups of file names quickly. It might be useful for your backups.

Re:Do we have to bring this up over and over again

[jtheisen]

Ok, that covers the <5% of users who can set up and maintain a backup systems based on rsync. What about the other 95%?

As an interface to set up a backup system for a moderately adept geek with sufficient focus to set up and maintain a recurring rsync backup, an above average grasp of the layout of their filesystem, and the presence of mind to alter their rsync script as their computer changes over time, rsync is extremely powerful. For everyone else, it's next to useless.

So can anybody say something to rsync-like features in other backup software? Surely there should be loads of commercial ones that do this differential copying thing?

SimpleNet ?

[Thad Zurich]

Yahoo News : " On one end of the power-brick-shaped device are two USB connections, and on the other end sits a single 10/100 Ethernet port. Attach that side of the device to your network and you can hotswap any number of USB-based storage devices (be they hard drives or flash-based) to the two connectors on the other side." Apparently the product has an unfortunate conflict with an existing web hosting service. Product page here.

Re:Do we have to bring this up over and over again

[jtheisen]

So can anybody say something to rsync-like features in other backup software? Surely there should be loads of commercial ones that do this differential copying thing?

Just googled for it and at least found DeltaCopy, a Windows wrapper around rsync incorporating a scheduler. If that's done properly, it should be alright for the average Windows user.

Re:Do we have to bring this up over and over again

[afabbro]

And honestly I'm not all that worried about backing up with modern operating systems.

Modern operating systems don't protect you from:

• Oops. Didn't mean to delete that.
• Oops, my wife/kids didn't mean to delete that.
• A bug in the new release of Gnomovision ate my existing Gnomovision files.
• Break-ins, electronic or otherwise.
• Your hard drive eats itself.
• Fire, flood, etc.

Best thing at the moment for home backup is to mount an encrypted external hard drive and copy to it, then take it off-site. If you think that sounds over the top, then I predict one day you'll be sitting at your terminal saying "aw, shit".

Floppies

[DaveAtFraud]

*LOTS* of floppies. You might be surprised at how cheap they are now, too. :D

Cheers,
Dave

Re:Floppies

[Aaron B Lingwood]

I recently purchased a floppy drive for $5 after discovering a couple hundred old disks that I had misplaced a decade ago. They contained, among other things, scans of certificates that I have lost the origins to in a cyclone.

Got the drive home, opened the box and found there was no connector on the motherboard *sigh*

--

Posted via Blackberry

Automated, and live.

[m.dillon]

It has to be automated or it won't work. If your backup scheme requires physical intervention to function normally, like swapping drives, it is destined to be a failure. It also has to provide live, directly addressable snapshots. Archival formats just don't work very well these days because it takes way, way too long to get to the data you might need to get to. Sure, a prudent company will always have some sort of archival or off-line storage for extreme emergencies, but the primary means of backup and restore today is live access. Only a really stupid person uses tape in the first two layers of backup.

One has to consider what one can do with the ridiculous amount of storage available on today's drives. Just a few weeks ago I slapped together a 10-Terrabyte 'test' filesystem with 2TB drives and a port multiplier enclosure. One E-SATA cable, 10TB. Its ridiculous how cheap this stuff has gotten over the last few years. But in terms of using all that storage, one of the goals of HAMMER (and to some extent ZFS) is to integrate live-snapshots into the filesystem itself. In the case of HAMMER's transactional storage (on DragonFlyBSD), you get fine-grained history, trivialized snapshot management, and bandwidth-controlled streaming mirroring for backup.

So the setup I have now is each box running HAMMER (on DragonFlyBSD) stores about 60-days worth of daily snapshots and one day's worth of fine-grained history on the production filesystem itself. Some of my boxes don't run HAMMER yet, and those I just cpdup/rdist over to the LAN backup box. Then there is a daily backup from the boxes on the LAN to a HAMMER-based LAN backup machine, and a weekly off-site backup from the LAN backup box to the off-site box. As I convert more of the machines over to HAMMER I can use the far more efficient streaming mirroring feature for HAMMER-HAMMER backups, and even make the daily backup a near real-time stream (and then cut to a daily snapshot each day), but particularly between the dedicated HAMMER filesystem on the LAN backup box and on the off-site box.

The big advantage of having the ability to maintain hundreds of snapshots is of course that you do not need to use the horrible cruft-multiplying 'hardlink' trick to maintain each snapshot. My LAN backup box can maintain about 120 daily snapshots of all my machines and my off-site box can store over a year's worth of weekly snapshots. My base backup set is well over 200GB now. Each day adds a few gigs, so a single terrabyte drive can hold a considerable number of snapshots.

And in terms of taking drives off-line for storing on a shelf, which is a reasonable thing to do as long as NOT doing so doesn't interfere with normal backup operations... it is far easier to do that with fewer larger drives then to do that with drive arrays. If you are taking physical possession you really only want to have to deal with one drive. In the case of HAMMER I just have a second mirroring stream going to a dedicate easily-pullable drive (via an E-Sata hot swap enclosure) which does not interfere with the nominal automated chain backups.

ZFS's raid-z is quite nice, but not really suitable for a backup system unless your backup needs exceeds a few terrabytes. It's a waste of power and money to construct redundancy for a single filesystem when 2TB drives can be bought and one can construct redundancy by creating multiple, independant backups. Ultimately it does get large enough to warrant at least hard-mirroring so things continue to click along if a drive failure occurs, but for most people its just wasteful to do that. If one already has a chain of backups (and thus redundancy in the form of completely independant copies), the occasional failure of a portion of the chain isn't that big a deal.

Where ZFS really works well is on the production system in a non-clustered environment. HAMMER doesn't really do in-filesystem redundancy (i.e. we'd have to depend on a hardware or software RAID layer), but HAMMER's goal is ultimately (as in the ultimate goal of the project) to integrate into a cluster of independantly-run copies of the same filesystem and in that sort of topology one does not actually want each individual copy to itself be redundant... it would just be wasted storage.

-Matt

Re:Automated, and live.

[value_added]

It has to be automated or it won't work. If your backup scheme requires physical intervention to function normally, like swapping drives, it is destined to be a failure.

More simply "If it isn't automated, it won't get done."

I do wonder why when the subject of backups comes up, we're subjected to countless posts confusing RAID with backup, suggestions for trivial use case scenarios, endless back and forth about the reliability of a given medium, and advocacy for someone's favourite utility.

What would be more useful (or at least productive) is writing what you wrote, or having some do a Frist Post containing a set of crontab entries, and a summary description of they're implemented. That would make clear that what typically gets posted is neither relevant nor applicable.

rdiff-backup to an external server and/or ZFS

[Anonymous Coward]

rdiff-backup to an external server. Retain as many incremental versions of the data as needed. I keep 90 days for my HOME and 30 days for every server (currently 6 running in VMs)

Use a ZFS-based NAS or server storage in a RAIDz config and take snapshots constantly. Use `zfs send` to remotely mirror your data to an offsite server - Amazon supports OpenSolaris servers and ZFS storage CHEAP.

Oh, if you're on windows, there is an rdiff-backup version, but it's only good for data. Use Clonezilla to backup your apps and OS.

Re:rdiff-backup to an external server and/or ZFS

[Anonymous Coward]

We used rdiff-backup to store one our client's data off-site...and kept the entire two years of backups without it getting over 6 Gb.

Personally, I can't sing the praises of an off-site over-the-net rdiff-backup enough. It goes over ssh, it only took about 10 minutes to update the changes, and it is a mirror+reverse solution...meaning if you pull out everything but the rdiff-backup-data folders, you have a current copy.

If I were to get smart enough on encrypting the backups with a client private key, I'd probably consider offering our customers with more sensitive information that service. A few might go for it...at least, enough to pay for our broadband.

I like backupc

[cohomology]

I run backuppc on a free Pentium III running linux. It backs up itself (tar), 2 window machines (SMB), a macbook (ssh and tar), and my website (ssh and tar). It is fully automatic, and does full, differential, and incremental backups. My wife can browse backups and restore individual files herself. If a file appears in more than one backup, even if from different machines, a single copy is stored. I now have 20 full backups, and 93 incrementals using just 36 GB. For our document folders, we do incrementals twice a day.

I make an occasional offsite backup by imaging the disk to another one (craigslist!) giving it to a family member for storage.

CONS: doesn't know about Mac resource forks, windows multiple data streams, ntfs permissions, ...
Configuration required condsiderable tinkering. A full restore would require reinstalling the operating system, then putting user files in their proper places.

Backup Signal

[Anonymous Coward]

I am always annoyed by the repetitive alert beeps coming from the hard drive when I back it up.

"Now" is correct

[registrar]

The best home backup strategy is "now, securely, and with understanding". Get 99% of the benefit by doing it now.

The best way to improve "now" is "secure". Secure your backup by tying it down, and taking a copy to a remote location.

If you do it "now" you will have a chance of recovery when disaster strikes. If you do it "securely" then you will have a chance of recovery when a bigger disaster strikes. If you understand how the backup process works and what you are backing up, you have a good chance of being able to do the recovery yourself.

3 drives minimum

[syousef]

If you want to protect against a house fire or theft 3 drives is the minimum.

1st drive is your main drive, which you want to back up

2nd drive is your primary on-site backup. Preferably only switched on for backups, which are performed regularly (at least weekly)

3rd drive is your offsite backup. It's there in case your first 2 drives are corrupted, stolen or lost. It is your backup of last resort and lives at a trusted relative's house that lives more than 10 minutes away but no more than about an hour. Backup to this drive every month or two.

Remember also that your exposure is limited to however often you back up your drives.

Some more tips, if your data is really important and you have the money to do it:

NEVER overwrite existing data on a backup drive if you can help it. Incremental backups are best. Once you fill a backup drive, stow it and get a new one. Only do incremental backups to the drive. Anything you overwrite may be overwritten with something that has become corrupted.

If you want to get more secure, rotate your offsite backup drive. You'll end up with multiple offsite archive drives.

Once a drive is more than 5 years old, copy it to a new drive. Do not dispose of or overwrite the old one. Hold onto it until it dies.

Re:3 drives minimum

[smash]

UH.... re: only doing incrementals. Incrementals SUCK for restore time/effort. Do a full backup on a semi regular basis, too. And I'd be doing differentials instead of incrementals. Nothing worse than having to get up and running by doing a full restore, then multiple incrementals to get back up. plus, if one of your incrementals gets damaged, you're fucked. With diffs, you can have all your previous incrementals for the week corrupted, and so long as you have a current differential, you're sweet.

Re:3 drives minimum

[syousef]

I used the wrong terminology. I meant differential. I am a Windows user and I only use robocopy or windows explorer. (I imagine I'd use rsync if I were using unix)

You're right in terms of backups relying on previous increment resulting in total failure.

This approach also doesn't sufficiently mitigate against a virus doing damage but lurking undiscovered for some time since hooking up the backup drive exposes it to any virus. However only write once media will mitigate and I've found CDs and DVDs way too unreliable.

Re:3 drives minimum

[Anonymous Coward]

A number of backup programs support what is known as a synthetic full backup. You do a backup that backs up everything (akin to a level 0 dump). Subsequent backups are just incrementals. No worries about when to fire off an incremental or differential. The "magic" comes in when restoring -- just the original backup set is needed and one restore (as the backup program calculates what would be on the recent image), no need to go through the full restore, then a ton of incrementals to get to a known point in time. Add to this some type of grooming (using Retrospect's terminology) to pull out backup images that are expired to save room, and it is a good advance. Initially, only TSM (previously ADSM) has this functionality, but in the past several years, Backup Exec, Networker, and Retrospect have this either as a default, or as an option.

If you do have a backup program that relies on fulls/incrementals/differentials, I always recommend doing fulls every so often, incremenetals daily, and differentials weekly. This way a person is not far from a current system image.

Re:Do we have to bring this up over and over again

[RichardJenkins]

Best thing at the moment for home backup is to mount an encrypted external hard drive and copy to it, then take it off-site. If you think that sounds over the top, then I predict one day you'll be sitting at your terminal saying "aw, shit".

Replace that external hard drive with a service like Amazon S3 and a piece of software like Time Vault and you have a real contender for a viable Home Backup strategy (albeit one for geeks). Expecting people to perform off-site tape backups from home is a bit much.

Disk-based, local and remote

[FridayBob]

As someone with a fair amount of experience with backups in corporate environments, I realized years ago that achieving the same thing at home with tapes was simply not practical. I'll assume you're using something like BSD or Linux like I am, so I'll ignore commercial backup software, but even then there's plenty left to be confused about.

First of all: hardware. Years ago, I gave tapes a try, but the only affordable drives I could find, from Onstream, kept breaking down on me and the tapes were not too cheap. I would have preferred DLT, but that technology was prohibitively expensive for home use. So, that left disk-based backups. The good thing is that disks are fast and the space is cheap these days.

Software. Mainly I use a backup package called faubackup, which I think is simple and easy to use. It's very economical with disk space because subsequent backups consist largely of hard links to files that were backed up previously and have have not been changed since. Data from remote machines is first copied to the backup server with rsync, after which that data is backed up depending on rsync's success. I use my own scripts to coordinate faubackup and rsync.

Redundancy. I'm luckily in that I also maintain similar servers for some friends in other parts of the country and have managed to convince them of the advantages of backing up our data to each other's servers. Rsync is really good for doing this, but obviously I leave out the directories with the bulky media files when copying data across the Internet (we all have ADSL with only 1 Mb of upload bandwidth).

Disks. Until recently I've always configured servers with two disks: one for the system and data and the other for backups only. It's a cheap solution that I've always been very pleased with. Since each of the two disks has hundreds of gigabytes of space available, I can afford to maintain many more backups than with a typical 21-tape strategy. The only problem that I've encountered so far is that moving the backups to a new disk can be very time-consuming: as opposed to copying only it's contents, you're forced to copy the entire partition over because of all the hard links it contains.

Alternative disk configuration. The last server I installed uses an SSD for it's boot and root file systems (using the 'noatime' partition attribute) and houses all of its data (along with /root, /tmp and /var) on a 4-disk RAID-5 array. This system performs very nicely, but the only thing I'm perhaps not 100% comfortable with is the fact that the backup partition is also located on the array.That may seem safe enough, but now I feel that maybe I should have used an extra disk, or even a pair of mirrored disks, for the backups instead.

CD-Rs do not have pits!

[yet-another-lobbyist]

CDs have pressed pits, in CD-Rs the information is written into an organic dye layer bye the laser -- no pits, no pressing.

Re:CD-Rs do not have pits!

[commodore64_love]

I wish people would learn to read. The grandparent post did not say CD-Rs or DVD-Rs have pressed pits.

Alex

[Anonymous Coward]

better backup solution? It is simple :)
slice backup files to 750MB, name it as XXX.SuperPorno.avi, and share it into any P2P network :)

You let your poor parents...

[yet-another-lobbyist]

... pay the electric bill for your server backup project?

FreeNAS

[methuselah]

I find that FreeNAS and a pair of mirrored raid1 drives is suitable.

No, RAID is not a backup....

[Darth_brooks]

Storing data on a single RAID array, on your daily use machine is not a backup.

Storing your data on a RAID array, then copying that data onto another RAID array, on another machine, *is* a backup.

(of course, it's not a spectacular, super awesome backup that will protect your data from flood, fire, and Jerry Bruckhiemer, but it is a backup, and it's a lot more than most people and many businesses ever think to do.)

Cheap, Easy, Good. Pick two. A network aware 1TB SATA NAS should set you back less than $400 shipped.

Wow that is cheaper than rsync.net

[Anonymous Coward]

Someone was just telling me yesterday about the virtues of rsync.net, which is around 65 cents/gb under their best discount plan. I felt that was reasonable for backing up important documents or even my source code repository (a few gb) but certainly not for bulk backups of multi terabytes.

Installed Plant

[Anonymous Coward]

> until they run out of duct tape and bailing wire to keep their WWII-era copper plant patched together, and even then.

Yeah, um, that's FiOS.

I'm completely convinced that the reason Verizon started deploying FiOS -- particularly in Florida -- is because they've been practicing Cut-to-clear for 30 years, and their outside plant is, essentially, unrecoverable; it was cheaper to spend $10B doing FTTH than to try to regroom it.

I've never found a Verizon outside-plant type for whom that did *not* ring true. And, as a datapoint, I have 45 T-1s in 2 buildings; I lose approximately one a month because someone from Verizon stole (usually only) one pair to give to another customer.

I report it, they fix it, and presumably, some other customer then loses one pair.

bleck

[Danzigism]

Most backup solutions are silly. The common user doesn't need a massive RAID array nor would they even understand how to recover if they ever experienced a disaster. Online backup solutions are *ok* but it is more likely that your hard drive dies than your house burns down. Now this isn't the perfect solution, but it is easy and it works. I highly recommend the ShadowProtect image based backup software. Matter of fact I would recommend any type of image based backup software. Being able to restore your entire computer with all your applications and data is really freakin' nice. More importantly, if your motherboard dies, you can do a hardware independent restore with your image so that you can restore your image to a totally different machine with totally different hardware and Windows won't bluescreen all to hell. All thanks to hardware abstraction layer. Yes we all wish we had off-site fireproof backups, but I trust my data with myself and only myself. So a couple external HD's with a backup image I think would be plenty sufficient for most people "in-home". And who said tapes were unreliable? Tapes are the longest lasting form of media ever in existence. hard drives die, and ink-based CDs and DVDs have a 5 year life expectancy.

Hammer Storage NAS

[bytethese]

I have a 2TB Hammer Storage array, configured as RAID-1 at home to backup my laptops. Do I need to backup daily? Nope. I tend to copy on occasion to the backup array. If one of the drives goes, the other is still readable. Seems to work fine for me. :)

Re "RAID is not backup"... and some solutions.

[MikShapi]

Whoa cowboy.

Sure, "RAID is not backup" in the sense that your primary storage, if RAIDed, is only protected from disk failure, not file deletion or filesystem blowout.

RAID very mych is a backup if you've set up a second seperate box (maybe on a seperate power grid, better yet in another building, or perhaps on two sets of removable hard-drives that get swapped between your home and an offsite loaction (your drawer in the office? your mum's place?) routinely.

If you're willing to go the extra mile, here's my take on it:

Failure modes:
[a] Single Hard-drive failure
[b] I deleted a file accidentally, or a file got corrupted by an improper shutdown, or the filesystem got hosed.
[c] Two Hard-drive failure
[d] Machine blowout, or Major Filesystem blowout (>20% (or whatever % you put aside) of the blocks on the volume got changed),

Solutions:
---------------
Failure Mode [a] -> RAID 5
Cost: Pay for 1 extra hard-drive whose volume you can't actively use (If you wanted 4TB, you gotta buy 5 x 1TB harddrives or a blackbox that incorporates these)

Failure Mode [b] -> Run linux on your fileserver, with LVM or LVM2, allocating 10%-20% (my figures) of your volume to snapshots (depending on how much change you expect) and having an automated scheduled job perform a snapshot weekly, mount it so it can be browsed anytime, and blow out redundant historic ones (while keeping, say, one from the beginning of every month for yay many months).
Cost:
* Have to run a linux box rather than off-the-shelf storage blackbox
* Some linux work required to set up and work out what to do in a real-life failure mode.
* Some linux scripting knowhow required to automate properly
* 10-20% of your overall post-RAID volume allocated to snapshots (YMMV).

=== a bit about snapshots ===
To the uninitiated, a snapshot only "saves" the old versions of blocks that got changed when they actually got changed. It doesn't stress the machine for an hour duplicating your gazillion-terabytes at any stage, nor does it care about files or filesystems as it sits below the filesystem.
Hence, If you took a snapshot on Jan 1st, nothing would physically get copied anywhere on Jan 1st.
If, consequently, on January 4th, a single bit got changed due to corruption following an unclean shutdown in a 9GB DVD iso called /movie.iso, the entire block (4MB would be a good ballpark for an LVM block) that this naughty bit lived in will be replicated - the old (pre-corruption Jan01 copy) 4MB block will be copied onto the snapshot space (that 20% we set aside earlier), and the new (corrupted) one will be copied onto the live filesystem.

We now have the same thing we would have had without the snapshot (a corrupted file on the primary volume), except, the snapshot retains an uncorrupted version of a 9GB file, which is costing us just 4MB to retain. And even protects us from filesystem corruption.

In real-wold terms, since our Jan-1 snapshot has been read-only live-mounted since Jan01, say, under /snapshots/0101/movie.iso, we can go back to our historic snapshot once we discover the file is b0rked and copy the pre-corrupted one out of there.
=== end bit about snapshots ===

Failure Mode [c] -> RAID-6
Cost:
* have to run a linux box rather than off-the-shelf storage blackbox
* Some linux work required to set up and work out what to do in a real-life failure mode.
* Buy 2 extra hard-drives whose volume you can't actively use (If you wanted 4TB, you gotta buy 6 x 1TB harddrives)

Failure Modes [a,c,d] Full copy on secondary storage array
Cost:
* Any Secondary storage.
* Extreme inefficiency. If you're running a 4TB (post-RAID/LVM) array, you will invest in 4TB of disk and be able to retain one historic image, or pay for 8TB to retain two, etc. You can go fancy and do snapshots here too tho.

Failure Modes [a,b,c,d]
Cost:
* Linux with RAID + LVM on your primary storage. This gives me snapshots

Re:Re "RAID is not backup"... and some solutions.

[ralf1]

RAID is an availability play. Backups of data that you consider at all valuable (family photos, financial records, etc) should be on write once data or offline. Many of the posts here refer to issues around physical drive failure, but its important to consider data corruption (intentional or unintentional) All of these data replication suggestions are, like RAID, availability plays. You need to have static point in time snaps or copies; or deltas from which you can recreate points in time; to be able to recover from a data disaster. Its great that you are using rsync to replicate your kids first birthday photos to a box somewhere else, but if that folder gets corrupted you are now replicating that corruption. You are now the proud owner of two sets of useless data and the endless crying from the spouse that inevitably accompanies such.

Re:Re "RAID is not backup"... and some solutions.

[MikShapi]

Exactly, thank you for making my point.

And how exactly would offline imply uncorrupt?
Do other means available to a home user - DVD, cheap tape, hard-drive-on-a-shelf pose a solution to this problem?

I have some 3-4TB of old data backed up on optical media at home. Important stuff (photos, scanned documents) I burn in triplicate because of how un-dang-reliable optical media is 5 years after you pull it off a shelf, never mind decades, and have already had the dubious pleasure of having to copy one half of a file from one disk (where its second half is corrupted), and the other from another (where its first is corrupted).

Unless you can come up with something like home-affordable LTO tapes (which have data reliability problems of their own, never mind head-broke-where-the-fuck-do-I-get-a-new-head-for-less-than-4000$-so-I-can-recover-my-archives issue), offline media is just as bad.

If you're smart, you can do something simple akin (forgive obvious double-matching bugs:)

Have your rsync (or own hacked job) do the analogue of this:

for x in `find /`; do
if [ `md5sum $x` eq `grep "$x" /last-md5sums|cut -d: -f2` ]; then
  cp $x /backup/
else
  cp /backup/$x $x
done

See? Online media offers a solution to the very real problem whereas offline media does not.

Philosophy re offline backups is great. You just need to somehow tie it into real-world(tm) solutions.

Windows Home Server Works For Me

[pcx]

I'm using a windows home server (HP but Asus started making them too). Basically it's a little box that's barely taller than the 4 hard drives it can hold. It's quiet and power efficient. It can centralize and store media files making them easy to share throughout the house but it also backs up the other computers in the house -- in fact it will even wake up a sleeping or hibernating laptop in the middle of the night to do the backup and then put it back to sleep when its done (something that just blew my mind when I first realized what was happening).

WHS doesn't have a concept of "raid" but you can flag folders to be duplicated and WHS will ensure that if it's possible they will be duplicated across hard drives. Adding hard drives is fairly simple. When you add a new one you can chose to add it to the storage pool or to use it to back up the WHS itself. The storage pool is pretty cool, there really aren't letter drives, WHS just maintains a giant pool of storage made up from all the connected hard drives so if you're running low on server space just add another hard drive.

Generally I'm happy letting the WHS backup my laptop and game machine. If anything happens to them the WHS can rebuild them. If anything happens to a drive on the WHS the folder redundancy will keep things healthy. I'd basically have to have a simultaneous, catastrophic loss of two or more hard drives on the WHS to start losing data and the odds on that are pretty long. One of these days I might build a small raid to backup all the media on the WHS but that's low priority right now.

For businesses Windows server 2008 has much the same capacity although it supposedly does image based backups (ie norton ghost, but WHS will do ghost backups as well with service pack 3). And of course apple's solution is time machine.

Re:Windows Home Server Works For Me

[Shados]

While WHS doesn't do ghost backups, it does come with an ISO in one of the built in folders, that you burn to a CD, and in the event of a catastrophe, you boot the hosed machine from that CD, it finds the WHS, and will reinstall the machine back to working condition from any one of the backups. Don't even need to reinstall Windows yourself.

May not be a ghost, but its damn close.

The 3-2-1 rule

[plazman30]

I heard this on a podcast somewhere. I don't remember which one....

The 3-2-1 rule.

3 copies of your data

on 2 different types of media

and 1 copy offsite.

Personally I use Macs, so my strategy involves Time Machine and an external HD AND a copy of Mozy for online/offsite backup.

On the LInux side you could use an external drive and either rsync, or any number of Time Machine clones, and for your offsite backups, you could use Jungle Disk to do online backups to Amazon S3.

Re:Do we have to bring this up over and over again

[iamhassi]

"Replace that external hard drive with a service like Amazon S3"

I don't know about you, but I'd be a old man before my internet connection would upload a terabyte worth of data.

If it's only a few gigabyte and you might get away with uploading to Mozy. Anything larger than that and you're back to hard drives.

but RAID *can* be backup...

[mooboy]

I'm using a backup scenario which uses swapping out mirrors (RAID 1) in combination with Windows Volume Shadow Copy services. Twice per day my system (Server 2003 or Server 2008) takes a Volume Shadow snapshot (VSS?), which is how you get that "oops" protection - its similar to having an always available, instant restore tape library built right into the filesystem (BTW, Novell had this feature 10 or more years before Microsoft, but I digress). You just need to make sure you have plenty of free space on the drive to accommodate the snapshots, but the algorithm is very efficient since it only grows when files have changed.
Then, once per week, for the off-site disaster protection, I swap the external eSATA software mirror drive out, remove the broken mirror under disk manager, import the foreign disk from last week and recreate the mirror. Bingo - just a single drive to keep up with. I have a hot on-site mirror and an offsite mirror no more than a week old. Its quick and convenient and performance is excellent (and cost is minimal since the features are built-in to Windows)

I'm not sure if there's an equivalent to Volume Shadow Copies under Linux, but the software mirroring is there and works quite well.

Exactly what I use myself: Acronis & HDD

[Anonymous Coward]

"my current backup system is to have 2 equally sized 500 GB drives and I use Acronis on a schedule" - by junglebeast (1497399) on Sunday July 19, @03:44PM (#28749481)

See subject-line, because I agree with, & use, your mixture & solution here - HDD's are "the way", & don't co$t too much (a 40gb unit WD JB series 7,200 rpm unit of EIDE PATA variety - these are CHEAP to find nowadays, in used parts/mom & pop shops for computer parts)... I use FULL BACKUPS though, only, & I use "maximum compression", & it fits a 150gb image down under 30mb in size iirc...

HDD's (slightly older & smaller ones) are very fast compared to other forms of media also (for faster restores & backups)...

APK

P.S.=> I backup once a month only, after MS patch Tuesday, & until then, I use a RAID - 1 mirror (not depending on it here for backup, just uptime moreso (but, some folks consider that a form of backup (I do, & I don't - mixed bag on THAT account))... apk

Better Oblig. Futurama Quote

[Culture20]

"I want to make a ham sandwich. Conventionally these contain bread and ham, but I'm an idiot so I want to make it from dog hair and epoxy resin".

Leela: And that sandwich you're eating is made of old discarded sandwiches. Nothing just gets thrown away.
Fry: The future is disgusting!

Re:Better Oblig. Futurama Quote

[Miphnik]

Murray: What do you got?
Oscar Madison: I got, uh, brown sandwiches and, uh, green sandwiches. Which one do you want?
Murray: What's the green?
Oscar Madison: It's either very new cheese or very old meat.
Murray: I'll take the brown.

USB Flash

[wonkavader]

A thumb drive is not a reliable backup, but it's a crucial backup, if all your other plans keep your data at home.

If you've got some crucial data which you absolutely cannot lose, or which you'd need in a hurry if a tragedy happened, you should have that up to date on your keyring. (Encryption is your friend -- use it.)

Why? Because when your house burns down, you lose all your backups except those you've move off site.

So yes, back up to another HD. Back up to disk. Mail those disk somewhere. etc. But also make a copy of the truly crucial stuff you need right away, and key in on your hip when you leave the house.

Voodoo

[Nekomusume]

Once a month I set my computer up in a circle of lit candles, spray a bit of booze around, and behead a live chicken.

Re:Do we have to bring this up over and over again

[Anonymous Coward]

Ok, that covers the <5% of users who can set up and maintain a backup systems based on rsync. What about the other 95%?

They should use Time Machine.

Which requires OS X Leopard.

So you're saying that all of those users who don't own a Mac with the latest version of OS X should either replace their non-Mac PCs with new Macs (Mac mini starts at $600) or upgrade to OS X Leopard ($130)?

Insightful indeed.

ZFS

[Samah]

I currently have two raptor drives with a mirrored ZFS setup that I use only for backups. I take snapshots regularly so that if a file gets deleted or overwritten I can just roll it back. Again, this is just for backups. I work on my desktop PC and copy documents, etc. over to my server once a week or so.
It does the job and the redundancy is comforting.

External HD + Online Backup

[Anonymous Coward]

A hybrid approach:

An external HD in whatever format you choose (USB drive, NAS device, etc.) is an excellent start since it's fast and local. However, it doesn't handle the "Holy fucking shit!" scenario (e.g. flood, theft, earthquake). In that case, you should use an online backup service like Mozy and let them handle the "Holy fucking shit!" scenario since it's their job.

Re:Do we have to bring this up over and over again

[iamhassi]

"Modern operating systems don't protect you from: Oops. Didn't mean to delete that."

Honestly, I've never accidentally deleted something that couldn't be recovered.

"Oops, my wife/kids didn't mean to delete that."

Best solution for that is a second PC. I bought a used 2.4ghz IBM for $65 and a slightly damaged 19" LCD for $50 on Craigslist. That's probably about the same price as the external 1 terabyte drive and far more convenient than taking it off-site, and you could just backup files across the network to that PC.

"A bug in the new release of Gnomovision ate my existing Gnomovision files."

Virtual PC

"Break-ins, electronic or otherwise. Your hard drive eats itself. Fire, flood, etc."

All very rare, but there's always that chance...

Honestly my dream backup solution has always been a low-power PC on a UPS inside a fire-proof cabinet in my attic. Unfortunately I didn't think 802.11g was up to the task with only 54mbps, but now that wireless-N's 300mbps is practically here I think this might be the year.

RAID + external drive

[smash]

Disk is cheap nowadays.

Use RAID and snapshotting (previous versions, shadow copies, whatever your OS of choice calls it) to take care of the "oh my disk broke and I need to get this TPS report done" factor (staying up and working).

Add an external disk for offsite backup ability.

For corporate use, there are other ways and means, but for home use that would be my method of choice at the moment.

mnb Re:Differential + hard drive - online

[Anonymous Coward]

Due to bandwidth restrictions, it would take years just to make a backup of a typical user's hard drive, and they don't offer enough space (seriously)

Translation: I can't keep my tracker ratio if I waste my parent's upstream backing up their financial records and my baby pictures to Mozy. I downloaded all my movies and have no physical pressed DVD as backup.

If the differentials start to get too large, I'll do a new full backup and start doing differentials from there again.

Yea, when diffs are large it totally makes sense to just do the whole thing instead. \o/

I just wrote my own backup script

[prometheus123abc]

It's pretty cool if you ask me. I just invoke the script, and it automatically rsyncs a few directories with a local mirror. rsync is very nice because it only copies the changes from the folders that it's syncing. It's very efficient. while the local mirror is being updated with all of the latest changes, the script uses ssh to create a newly timestamped copy of the last mirror I uploaded to my server. The script then compares the local mirror and the remote mirror, and only sends the changes. This way I can back up all of my important files locally and remotely while using as little bandwidth and local processing power as possible.

Re:Do we have to bring this up over and over again

[CrashNBrn]

Well its been almost a month. Technology changes very rapidly, and JollyRoger wants to know "Is there any better advice these days?" Since, you know, last month. Lets see there was Gbridge, PathSync, DirSync Pro, Jake, SyncToy v2.0, Unison File Synchronizer, cron run rsync's, and JBOD in a Storage Tower from Addonics.
Since you know technology changes so rapidly, you'd definitely want to trust your backups to something bleeding new. And "these days" are so much better than "those days".

RAID with removable disks

[time961]

I tackled this problem with an encrypted (dm-crypt) RAID-1 server equipped with easy-removal SATA cages.

I have a four-drive RAID-1 configuration (that is, every bit gets written to four different drives).

Every two weeks or so, I run a script that dismounts the file system briefly (to render the on-disk state consistent), and pop out two of the RAID drives, leaving two working.

I then mail the two removed drives to my two friends who hold on to my off-site backups (priority mail flat rate box $4.95) and they mail back the drives I sent them previously (I include prepaid labels).

In the meantime, while I'm waiting for my friends to mail drives back, I take the two drives I'd previously gotten back in the mail and put them back in the server. The server recognizes them as out of date and starts rebuilding, and a day or so later, I'm back to having four identical drives.

Lather, rinse, repeat.

There are usually four drives in the server (the two permanent ones, plus the two that will make the next backup), and four drives "out" (two being sent to the undisclosed locations, and two being sent back). So in the event of subtle corruption, I have two complete redundant backup sets: one two weeks old, and one four weeks old.

This works for me because my storage needs are modest enough that a single 1-TB drive holds them comfortably. Next server I build like this will be 2-TB, and that will hold me even longer. Astonishingly, drive capacity is growing faster than my ability to consume it (I used to have multiple boxes of external drives).

It's a lot of drives: eight drives doing the work of one. But drives are cheap: the current sweet spot seems to be $130 for 1.5TB, or about $1000 for the disks and another $500 or so for the rest of the server (those SATA cages are surprisingly expensive).

You could use fewer drives: maybe only single drives as backups, rather than pairs in different locations, and maybe just wait for drives to return from the field, rather than having a shadow set always there and spinning. But the extras are pretty cheap as cheap insurance goes. And, of course, you don't have to mail them (and actually I hand-deliver one of each pair, because one friend is local, but it's comforting to know that my other shadow is 2500 miles away).

If I wanted, I could just pop in an extra drive (or dissimilar pair) for a day and create archival snapshots to store permanently off-site but I haven't really had that need.

I haven't lost a drive in the mail yet, and the rebuild process is a pretty good proof that the drives are working.

I made a previous version of this idea with external USB disks, but SATA drives are MUCH more convenient (and cheaper to mail). All those USB enclosures and their inefficient little power supplies and twisty cables made for a very clumsy (and toasty) configuration. Now, I have just a single full-tower chassis and the cooling is efficient and built in.

This solution is entirely based on commodity parts. Anything that breaks, I can replace with something new that's as good or better. I deliberately made my RAID pairs from dissimilar drives (Seagate and WD), to stave off concurrent failure modes, and even sourced my Seagates from two vendors so I got two made in Thailand and two in China (I think). I used to do tape backup, and man, was that a pain: drives go obsolete, formats disappear, media always gets more expensive, and it's slow and linear. Of course, if I had 100 GB of enterprise data to protect, I might be more interested in a tape-based solution--but it would be a lot more expensive. Disk is clearly where the cost-effectiveness is being pushed most heavily.

Encryption is essential. It would be absurd to be mailing around cleartext drives. But a strong password means that the whole-disk encryption is effective against any plausible attack. I keep the password well-protected and never change it, although it would be easy enough to do that with dm-crypt.

I w

I like rebit

[PinchDuck]

An external HDD that also archives individual files. It is very cool.

"backup" vs "back up"

[Anonymous Coward]

Back up to a honking flash drive?

As shentino (perhaps unconsciously) points out, the proper verb form is "back up", not "backup". The word "backup" is a noun, not a verb. (This is true of most compound words ending with "up" or "down", such as "makeup", "shutdown", "letdown", "screwup", etc.) So, in the summary, the word "backup" should be "back up" in the following phrases: "too large to backup via conventional means", "the best way to backup hard drives", and "Backup your internal HDD". The word "backup" was correctly used (as a noun) in the following phrases: "Best Home Backup Strategy Now?" and "RAID is not backup".

home users

[Zork the Almighty]

Backup your internal HDD to an external one, and if your data is really important, have two externals and swap one off-site once a week.

For the vast majority of home users this is still the best advice.

RAID is not a backup...

[Eskarel]

but backing data up to another storage source is.

When people say that raid is not a backup, they mean that simply putting raid on your existing storage will not give you backup, not that something using raid cannot be a backup. Anything which results in two or ideally more independent copies of something is a perfectly valid backup mechanism, regardless of whether the two copies are both on raid devices or neither is. The biggest question with any backup mechanism is what disasters will wipe out your backup and your original, and whether you can live with them. A copy of a file on the same physical medium will protect against accidental deletion and small scale disk corruption, but it won't protect against a failure of the medium. A copy on an external drive constantly connected to the PC will protect you against drive failure, but may or may not protect you against a power surge(depending on what causes the surge and a certain element of luck). A copy on a NAS device in the same building will protect you against the destruction of your computer, and some surges, but may or may not protect you against damage from a lightening strike, fire or other natural disaster. A copy in another building offers additional levels of protection.

Of course as you move out along that scale, cost increases sometimes dramatically. An external copy is more expensive than a NAS device is substantially more expensive than an external hard drive, offsite backup is generally substantially more expensive than a NAS device, etc. It all really depends on how important the data you're trying to back up is to you, and how difficult/expensive it would be to replace. For regular home users, most stuff isn't really all that important and any archived copy will probably do. There are some exceptions of course, family photos, wedding videos, a lot of the things which actually don't get seriously considered in backup routines, but most stuff really just isn't all that important. Even for those things though, it's probably enough to keep multiple dvd copies with a few stored at another location, and reasonably frequent new copies to prevent bitrot.

DROBO

[countach]

Because hard drive technology has outpaced tapes etc, I'm afraid they seem to be the only solution as far as I can tell. And if your backup requirements exceed the size of a single drive, yes I think Drobo is the way to go.

Sure I wish there was a tape solution that could do the job, but unless you have major corporate sized budget, I can't find a product that can do it.
So go DROBO as far as I can see. On the other hand, if you've got that much data you probably are already on Drobo, and now need a 2nd one to back it up.

Drobo is good.

rsync to a good friend

[datadefender]

Paradigm: 1. backup daily as disks do fail. 2. store offsite in case you have a burglary or fire

If you have a broadband and a flatrate:
Mirror your Linux box with the one of a trusted friend using rsync - and he mirrors to your box.
"rsync -avz --delete -e "ssh -i /home/myaccount/myrsa.key" /myvolume myaccount@myfriend.dydndsn.org:/myvolume"

If you have no broadband or flatrate - use 2 USB disks (Truecrypted) - one of them stored at friends or at work.
Do daily backup using rsync (linux) or robocopy (windows).
Swap the 2 disks in regular intervals - say weekly or monthly.
use www.mozy.com to backup your "crown jewels" that change daily - 5GB are free there and its all encrypted.

Time Machine

[bgspence]

But, not Apple's.

If I have a disk crash I use my time machine to go back in time, buy a backup disk and back up the one that's going to crash.

And as a bonus, sometimes doing this alters the future enough so my original disk runs just fine without crashing!

Again, first page of replies is fucking useless

[Master of Transhuman]

Except one guy who had it right. Buy a fucking docking station, buy 2 fucking 1-terabyte drives (times however many fucking terabytes of fucking porn you own), back up the fucking shit on each fucking drive, take one off fucking site.

That's fucking IT! There IS NO better fucking solution.

Now if you only have 100 fucking MB of fucking email, you can find a better fucking solution than fucking 1TB drives, I'm sure. But there IS NO one fucking solution that fits every fucking situation.

Now - WHAT THE FUCK DON'T YOU UNDERSTAND?

Sorry, been listening to the Christian Bale tape again...

Says who?

[jotaeleemeese]

Sun is putting SSDs on their storage devices.

Those are enterprise quality devices, not pr0n backup servers ....

Re:Says who?

[Killjoy_NL]

Being one thing doesn't have to exclude it to be the other :)

Most likely you have 1.9TB of crap.

[jotaeleemeese]

That is the real problem that few people want to address: that we don't need to backup everything that we have.

Re:Most likely you have 1.9TB of crap.

[ifrag]

Exactly, this is why my backup only takes up 1.8TB instead of 1.9TB, because I was able to properly identify the stuff that really needed backed up.

Yep, there is.

[jotaeleemeese]

To drive home the inadequacy of online backups in the *cloud" (Web 2.0 music here please).

Hourly incrementals? In a home computer?

[jotaeleemeese]

In which planet do you live???

Home backup

[Anonymous Coward]

Use a caddy/hdd - keep it secure and safe from fire.

Test your backups

Don't backup. I am serious.

[jotaeleemeese]

This is a home computer isn't it?

What is so vital that you need to do organized backups?

Movies? How many movies do you have there? Will you ever watch them all?

Ditto for music, holiday photographs and videos (be honest, most of them are eminently forgettable).

Important stuff is on paper or off site in a safe place (contracts of any kind, etc).

Are you a developer? Your code is in your company's servers. OSS developer? Get an account in Sourceforge.

If you really must (most likely you don't) just buy another hard disk, make an exact copy of what is in your computer, and forget about it for 6 months.

I personally think that most people need only the above strategy. If you as an individual *think* you need to backup 2TB of data reliably I will say you have far too much crap in your hard drive. I humbly suggest that you get a life.

Analyze your real needs

[dbbd]

The biggest mistake, which directly leads to the original question above, is that you need to back up everything.

If you spend some time analyzing your data, you'll find out that not all data is equal. Loosely your data is split among:

1. actual work (spreadsheets, documents, etc.)

2. media

3. programs

4. OS There are different attributes to those different types. Your work files are most likely the most important, they are also the least in size, and the hardest to re-create. Media does not change over time. New is added, but whatever is there, never changes. Programs are easily re-installed, same as OS.

What I'm saying is you need different backup strategies. For me, work files are backed up online via a service. Music media is backed up on DVDs, incrementally, so when new is added, only the new is backed up (and for me media is music only - for movies I keep the original DVD media, or if downloaded, I don't bother. They are re-downloadable. Same for programs - either I have the original, of if downloaded, I create a DVD media for it.

For my pictures collection I use a different scheme. I use online redundant synchronized copies. I have 3 copies. If one disk dies, after I fix the computer, I have 2 more copies to re-sync from.

I think that like any big problem - cutting into smaller ones gives simpler if different solutions to the parts of the problem.

Re:Different kinds of backups for different failur

[Anonymous Coward]

5a. You might have been foolish enough to install a videogame on your PC, and the copy-protection routines might have sabotaged your backups.

Re:Different kinds of backups for different failur

[hab136]

Great summary, with one quibble:

4. We would like to protect against natural disasters. For someone living in New Orleans, it would be nice to have a backup somewhere outside the path of Hurricane Katrina. Remote backups may be pretty much the only way to accomplish this, unless you're a frequent traveler and can hand-deliver backup media to remote locations.

If you actually have a remote site, UPS/Fedex will carry the tapes for you. You don't have to send your own human along. I imagine that with most home users the issue is that they don't have a reliable remote site to send tapes to, not that they can't get the tapes there.

RAID is not back up

[_pi-away]

The expression "RAID is not back up" is meant to illustrate that if you're only copy of something (ergo you're working-copy) is on RAID, that is not enough. Significant hardware failure, file corruption through application or OS, or file deletion by accident or a virus - any of these will destroy your local data even if it is RAID'd.

If your working copy is on computer A, however, and you're backing up to a RAID on computer B (or NAS B, whatever), that is back up. It is not fantastic back up (offsite, multiple layers of redundancy, encrypted, yada yada yada), but it is back up.

In short, "RAID is not back up" should be rephrased as "RAID alone is not back up."

Best Mac backup solution ever

[Anonymous Coward]

Best Mac backup solution ever = external HDD + SuperDuper

What sort of nurturing?

[Anonymous Coward]

> First, they were born/nurtured in such a way to have above average technical aptitude.

Yeah, but that "nurturing" was when I had no choice but to RTFM and figure it out on my own... You do have to be interested enough to sit through explanations you don't understand and break them down piece by piece until you finally understand them, though. And implicit in that, you have to be able to remember things you don't understand at all accurately for long enough to digest them.

Don't overdo it for homeuse

[houghi]

Just pop in another HD and use a script like http://storebackup.org/ to do the incremential backups. I do not zip my files, so the restore is the most importand part. What I use it for is if I did something stupid, more then being afraid of hardware failure.

The main things I backup are settings and some files.
Things I do not backup: CDs and DVDs I ripped. Those are on another HD that is read-only. Website stuff, as that is already backed up with my hoster.
Scripts and such I have are with my webhoster as well.

I often see that people do a complete overkill with backups. I understand that if you are a geek, it is fun to play with it. If you are a standard user, first lok at how you want the restore to be happening.

So what I do is have the backup HD mounted as read-only and read-write by root when doing the actual backup. Yes, I have lost data, but as it was for my home, who cares if I lost 3 years from a mailing list? In all that time I have never really missed it. I now even do a yearly cleanup of all my mails. I just delete them, as I never look at them anyway.

Just because I can do backups does not mean I must.

Re:Different kinds of backups for different failur

[jabuzz]

1.5 can be done with SAN storage and/or virtualization in a cluster. With the SAN solution, you just point the LUN's at another similar box and start it up.

2,3,4,5 can be done with IBM's TSM

use neighbours

[stiller]

Install a SFF system (Mac Mini, Shuttle, whatever) at a neighbours' house (across the street, preferably), but within reach of your wlan. Using your preferred method (cronned rsync, time machine, whatever) automate the creation of daily, hourly, whatever backups. Offer free internet access, PC maintenance, whatever in return. Make sure it's internet accessible for those trips.
Done: Free, fast, regular offsite backups.

Alternatively, use very long CAT cable.

Re:Mozy is good, but they don't encrypt filenames

[tompeach]

Looks good, will you support OpenSolaris??

Verify Your Backups!

[Anonymous Coward]

Whatever strategy you use, verify your backups! there is nothing worse than having backups that don't work or which are stuck in some proprietary format you can no longer access for some reason.

I don't take orders from fat AC cunts

[Hognoxious]

It last came up about a month ago. Even singularitards don't think things change that quick.

Cheap windows way

[Anonymous Coward]

My setup:

1 FreeNAS which can hold 500GB of data, running 24/7: cost about $100
1 Western Digital External HDD which can hold 650GB of data, some of it important: cost about $60
2 Windows XP client PC's with some important data

My software:

DSynchronise (http://dimio.altervista.org/eng/) runs on each client as a real time service copying all stuff from selected directories to the NAS, keeping 2 changes: cost $0
Unstoppable Copier (http://www.roadkil.net/program.php?ProgramID=29) manually runs on one client once a month to copy everything from the NAS to the Western Digital External HDD: cost $0

Benefits:

Data on client goes directly to NAS
Real time backup with 2 changes directly available on NAS via Samba
Very improbable that one client and NAS and month old backup on WD get trashed
Pretty cheap 2 way backup

Bad:

WD External HDD should not be in the same house as the NAS in case of fire.
Don't forget to do WD HDD process otherwise failover situation gets older and older each month.

Tip: Give family pictures and movies to as much people possible. In case of manual retrieval you have lots of sources.

Re:Do we have to bring this up over and over again

[pAnkRat]

Pfff, a full backup of my fotolibrary (17Gig) to my rackserver takes about 2-3 Days,
incrementals are done in minutes, or 2 Hours max (daily increments)
Hardly time to get to old, but good for me, for not getting to much gray hair.

DSL with good upload rates are not really expensive here in germany.

if you're a mac fanboi

[v1]

you can just plug in an external HD and click the time machine button and forget about it.

For things important enough to be offsite you can just save them on your MobileMe drive and let apple mirror it for you.

Irony here is I am one but do neither. I use rsync all over the place here and don't presently have an offsite.
I just got done downloading and building Rsync 3.06 and need to figure out how to build it for a different architecture, as I only have dev tools on my intel and I need to get rsync updated on the ppc's. (I know only just enough to be dangerous with fink/dev tools/etc and follow instructions for downloading and compiling builds)

Ken's Offsite Laws

[kenp2002]

"The safety of a offsite facility is inversely proportional to the height of the builds around it." - Population density tends to correlate to the height of building making these location statistically prone to issues. .""The safety of offsite storage increase with distance from the source of the data"." - Environmental catastrophy, the further the distance from home the more likely to avoid the same catastrophy. .""The cost of offsite storage increases with distance from the source of the data"."-
Usually this is the result of shipping costs or fuel to get the backup from home to storage. .""The convienence of offsite storage increases with distance from the source of the data"." -
The closer the storage, the easier to drop tapes\disks off and to pick them up. .""Never use offsite storage facilities near Denver Co. or damn near anywhere in the Dakotas."." - Primary nuclear targets. Simply not an option for risk. .""Never store data in a city with a population greater then four times the median city size for your state nor in the state's capital city"." - While mirroring the first rule, the basis for this specific rule is in the event of calamity traffic in\out, power, riots, etc. All those population driven risks come into play. The larger the city the more risk involved. Now with terrorists factoring into risk analysis the larger cities are bigger targets.

and the last golden rule I offer: .""Just because you backup online doesn't absolve you from due dilligence on the physical location where that online backup ends up."." - Online backups aren't magical. 'Where does the data end up and how "safe" is that location?' are questions that still need to be asked.

Offsite and Automatic Backups are Best

[scottm52]

I keep a couple USB Drives too. But online backup just keeps it all offsite, automatic (because I'd forget) and covers all my boxes.

I've been using OPENRSM CloudBackup for my systems. It works well (even on my crappy DSL connection).

Their software client works on Win MAC and Linux so I even run it on my VPS and Dedicated Servers besides the home systems. And they let me do all those systems (6 and growing) with my one account.

I'm sure there's others. But this one works for me.

Wasting time

[frozentier]

You folks are wasting time backing up data. It's a well known fact that hard drive do NOT fail. Data showing otherwise has just been fabricated by manufacturers of data backup products and services.

External + NTI Shadow

[Saracenus]

I have set my technology challenged mom's system up on a 500 GB USB external drive and have NTI Shadow running in the background. Shadow copies whatever file you are working on to the external drive in realtime. If you accidentally delete a file, it keeps the last copy you have for a restore left on the disk. For mission critical files (family business) I copy a set to my laptop 1/week.

I really should ghost her machine if I need to rebuild after a crash.

it pays to be nice.

[Organic Brain Damage]

People pay you to help them. To make their computers work. And you want them to do what you've done, read all about computers and fix it themselves....and then why would they keep your mean angry little self around?

I choose cheap & reliable...

[Organic Brain Damage]

Which NOT easy method do you recommend?

Plus: rich-client backup and real server platform

[SplatMan_DK]

There is also the option to install push-agents on each windows client, for daily backup.

The client-backup repository is based on Windows Server 2003 principles. It creates an initial image of the host, and then does sector-based differential backup. This greatly reduces the time needed to backup individual clients.

The backups can be used to restore complete images of the host harddrive (in case of complete failure) or individual files. Individual files are accessed through the Windows Home Server Console where the user can simply open the backup for any day (where a backup was performed) and browse to the desired file(s) using a normal Explorer window.

I have been using WHS for quite some time now and I am extremely satisfied with its features and performance. To my knowledge there are no push-agents for non-windows operating systems so I am scripting the backups of my Mac and openSUSE mahines. And obviously there is nothing "open source" involved. But the server runs a ton things for me that no NAS device could ever do. In addition to backup and media streaming my WHS also serves as a mailserver, enhanced local DHCP, network booting, centralized antivirus control, media library for ripped DVDs (with plugins for various players and a ton of metadata for the ripped movies) a bunch of homebre web applications (i use ASPX but it is no problem to install a mySQL and Apache webserver if that is your flavor) and remote access to my files from anywhere on the internet.

I am not trying to turn this into a FOSS-vs-Microsoft flamewar. I am simply saying that WHS is actually a pretty good product regardless of your personal feelings about Microsoft.

- Jesper

RAID1 + drive rotation

[Anonymous Coward]

When I recently rebuilt my main home machine, I bought three brand new 750GB drives.

Two are in the machine at any given time, as a RAID1 mirror. The third drive is kept offsite. Once a week, I pull a drive from the mirror and swap it out with the drive carried back home from my offsite location, rebuild the mirror, and then take the freshly pulled drive back to my offsite storage location.

If something bad happens to my home or the primary RAID1 mirror, then I've still got an offsite copy of all my stuff no older than a week old. That's good enough for me.

Homebuilt? Or buy a low-power server the easy way!

[SplatMan_DK]

Disclaimer: I am in no way associated with the following companies and/or any of its representativs or employees. I am merely an very very satisfied customer.

If you don't want to build your own system you could consider buying a prebuilt system. It saves you a ton of trouble, lowers your electrical bill, reduces your carbon footprint, and has cool features that are hard to get right in a homebuilt system (such as small size, passive cooling, hot-swap bays or wall-mount kits).

I researched the market for prebuilt WHS servers and ended up with a Tranquil SQA-5H (see http://tranquilpc.co.uk/ for the full product range) and it is probably the best piece of computer hardware I have ever purchased. Room for 5 harddrives, easy to work with, and the machine itself uses only 29 W.

Similar products are available from HP (check out http://www.hp.com/united-states/campaigns/mediasmart-server/) and a lot of other vendors (like Acer, Velocity Micro, Niveus). They are all available under the "Buy" tab of the Windows Home Server website: http://www.microsoft.com/windows/products/winfamily/windowshomeserver/default.mspx

- Jesper

Re:Do we have to bring this up over and over again

[router]

I mean, google has all the answers.

http://www.ss64.com/bash/rsync.html

Zip and email

[blueforce]

Get one of those free, secure, online email accounts with unlimited storage somehwere like hotmail, gmail, yahoo, etc. Then just zip your hard drive contents and email it to yourself.

Ta-da.

Multiple methods never hurt....

[robnsara]

Okay, well, as a data protection engineer at work and a geek at home, here's my strategy (I do BOTH)
1) Near line/onsite backups: I do a multiple-times daily sync to an external USB drive for critical data. In this case, this includes things like my MP3 library, video garbage that I've collected. Basically anything that I can't replace _really_ easily.

2) Offline/Offsite backups: I use Mozy (as some other users have mentioned). $5 a month, large storage capacity. It's an always incremental backup, automatically runs when the computer is idle. It takes quite a while for the initial backup, but who cares. I don't personally allow my MP3s or some of my video junk to backup to Mozy, but the stuff I seriously care about is there.

For the stuff I'm really paranoid about, I try to make sure I've got copies dumped to my iPhone, which is always in my pocket, so it's not going to be at my home if the place burns down.

CrashPlan, JungleDisk, SpiderOak

[Anonymous Coward]

I value my photos. I keep them on RAID1 with an auto-sync to external drive. I then run CrashPlan to backup to 2 remote datacenters where I have servers co-located. I also run an occasional CrashPlan local backup to external drive which I take to the bank and place in my safe deposit box.

1 colocation box is at work, $0. 1 colocation box is in DC ($45/mo). Safe deposit box is included with my bank account ($0). CrashPlan cost me $60 or so for the software to get the advanced features.

I used to run JungleDisk but figured running my own machines would be safe enough since I have two of them at separate sites as well as my other levels of redundancy. SpiderOak looks interesting but it's not the fastest thing out there... Anyhoo, those are the 3 that I narrowed down to, you may have different tastes.

Sort Your Thinks

[Anonymous Coward]

HDD are getting fat & fatter but I can't believe people need to backup 1000 MB of personal data.

Put the important stuff (= what you created yourself, not downloaded/copied/...) somewhere and back that up using some old time solution, and some new-fangled "online" backup with revision control.

For the other 999 MB (of porn) do nothing, losing that is a godsend (= terabytes of *new* porn woohooo !!!!) .....

Personally I think it's stupid to backup/burn/keep anything that's easily found on the web (like ant kind of media), by the time you'll need it again:
  - you wont be able to find your hardcopy
  - the particular version you backed up will be obsolete
  - it'll never happened

Second HDD + Backup SW

[EmagGeek]

Whenever I build a machine, I put in a second internal hard drive to which I make a periodic image. Personally, I use Acronis, but I am sure there are a ton of decent applications out there that can do it. You could even do a bootable flash drive with Linux that just runs a dd to copy the drive over.

Hard drives are cheap.

Crashplan

[Ron Harwood]

I use crashplan - it's free as in beer... cross platform and does "to disk" backups - either to a local disk or a disk on a remote system. It does data de-duplication, compression, encryption and only backs up thing that have changed.

www.crashplan.com

Flash has a poor shelf life

[default luser]

What exactly is the title referring to? It means simply this: if you put a flash drive on a shelf, and NOTHING breaks EVER, you will still lose your data within 10 years. This is because the data on flash is less "permanent" than a hard disk drive - it is an electric charge trapped between insulating gates. Unfortunately, the insulation is imperfect, and the charge slowly leaks away. This means that every drive in existence will lose it's data, unless that data is periodically refreshed (i.e. manually rewrite all the data).

I would say that hard drives have a longer shelf-life in terms of the lifetime of magnetic moment, but at the same time the mechanical portions have poor shelf life. Assuming nothing else breaks, if you leave a hard drive on a shelf for years without powering it up, the mechanical parts will seize (corrosion, drying of lubricants, etc).

Your best best is to backup early, and often, and KNOW the weaknesses of your media. Basically, if you want to use flash, periodically clean the drive and re-write all data. If you want to use hard disks, power them up regularly.

Not impressed by Time Machine so far....

[knet99]

I have a Time Capsule and is not too impressed by time machine.
After months the backup got corrupt, and I find out that all of the
backup is stored in one single object, a sparse bundle. So Time
Machine has one gaint single point of failure.

Disk Utility failed to repair it so I bought DiskWarrior that all are raving
about, didn't even work repairing the sparsebundle itself on a remote
disk, and then of course can't repair it. With lots of trouble copied the
sparsebundle to a USB connected disk, to find that because DiskWarrior
stores all information in RAM and a backup disk the way Apple has done
the backup storage contains tons of files (inside the sparsebundle I
guess), DiskWorrior has to begin swap, and after 24 hours it had not
showed any progress at all. Guess it would require a week or more.....
(Alsoft support thinks it is no need to inform customers about the RAM
requirements with many files, even if swapping it *will* eventually do
its job weeks later so not a bug, we customers should after all
understand the internals of DiskWarrior).

Time Machine to my Time Capsule is also slow like hell and loads
my machine more than I expected.

Re: Use RAIDED media.

[ceo4techass]

Backing up to hard drives is good. Backing up to removable harddrives is better (the combination removable tray and hard drive is called "media"). But backing up to RAIDED media is the best! presently, the only company I know of which makes RAIDED media is http://www.high-rely.com. They offer a 3 drive RAID 5 media and a two drive RAID 1 (mirror) media as well as RAID 0 media. They also offer "Softwareless backup". Pretty cool stuff. Bit pricey.

Re:Homebuilt? Or buy a low-power server the easy w

[tcolberg]

Actually, because of Falcon4's comments, I am seriously considering a WHS system. Specifically, I'm looking at the HP LX195 because of the low power use. What helped push me over the line of reticence was his mention that you could run remote backups through Hamachi. This makes setting up a system of automated backups on our home's 5-7 systems so simple, I can't just leave it to chance anymore.

If this is as simple as you all are saying, all I'll have to do is make sure to do the additional backup to an external so that I can move it offsite every month.

Dedicated backup server at your parents/friends

[Anonymous Coward]

By my opinion, one of the best solution is to set up low power consuming server such as Bubba2, at offsite location (at your parent's / friend's) house. First (full) backup will take forever, but the incremental ones will be ok. There is hard disk at the other side (if it is in RAID1, even better). When using such a server you don't mind if it's turned on all the time, and you can also set it up to serve some other servers (for personal use, as it has not enough power for anything more).
http://www.excito.com/bubba/products/overview.html

If you are using Linux at the machine, that has to be backup-ed, you can use sshfs to mount the remote filesystem and rsync to sync it. Versioning can be done using hard-links. This is simple, easy to understand solution, but that's just one of the many posibilites.

If anyone know for any similar low-power linux server, please share the info.

P.S.: If you have too much time, you can create a low-power linux server using LinuxStamp and external USB hard drives (but now I am drifting away from original topic).
http://www.opencircuits.com/Linuxstamp

I laugh at your long term reliability

[sean.peters]

I painted my data onto the walls of caves... 30,000 years from now, when the rest of you are crying in your beer over your lost data, the precious pictures of my hunting trip will still be retrievable!

It's already a backup.

[Anonymous Coward]

99% of people using 1 TB of data for personal storage, are already "Backing up" their movies and music.

So what you are really asking is "what's the best way to backup my backup"?

Once we have that figured out, you are going to want to know how to backup your backup of your backup.

BTW, I use Mozy, and love it. As long as your backup isn't huge, or you are willing to wait a long time for your backup to be complete, and you have a good internet connection, you should be set. I started my backup as a small set of folders that I considered most important, and then expanded the folders as they completed. Took many nights, but I was sleeping, so I don't care.

do something stupid?

[Igarden2]

I submit that "a cigarette in my mouth" is sufficient proof of concept.

Mozy problems

[rvw]

I had a problem with Mozy as well. It turned out my osx-partition had too little space, and some big files couldn't be processed anymore. No mention of this in the status window or anything. It even reported that the backup succeeded. Just because I noticed that the backup had been uploaded too quick to be true, I checked the logs, and found an error message. Mozy support still doesn't understand that this is a bug.

So it may be that your problem was reported in the error logs. But I guess you don't trust it anymore.

I have about 100GB to upload as well, which succeeded in the end. I am still thinking about changing to S3 though.

Re:Different kinds of backups for different failur

[yakovlev]

6. Someone may break into your home/place of business and steal anything that looks remotely "technical." This requires some sort of remote backups.

Nothing beats a 1:1 copy

[Anonymous Coward]

This has been my system for years and it works. HD's are so cheap these days I just buy two at a time.

http://texturedstatic.blogspot.com/2009/02/one-approach-to-backup-your-computer.html

Anonymous

[Anonymous Coward]

A flash drive is probably the most stable technology.

I disagree. They wear down gradually, you know?
I had MP3 player once, 1GB. I made fairly regular backups on it. After 3 months or so it started to hang Vista when I put it in. It went to the drawer for next 2 years as broken.
Just recently I looked through my stuff and found it. I booted up my Linux system and made a raw copy. It stopped on 204th megabyte with permanent read error.
Reliable, don't you think?

Take my advice - never, ever do backups on your USB or store your only copy of data on it.

Re:Do we have to bring this up over and over again

[Anonymous Coward]

-1 insightful?

Re:Different kinds of backups for different failur

[WuphonsReach]

We would like to protect against accidental deletion of files, file corruption, or edits to a file that we have now reconsidered. This can be done with snapshotting. In source code, to reconsider and edit to a file is fairly common, and is the reason why most programming projects use revision control systems. Other options like nilfs or ZFS snapshots can also fill this goal. This goal is accomplished more easily if the backups area automatic and the backup device is live on the system.

The solution for this on a Linux box is FSVS paired with a Subversion backend repository.

Once you have the machine configured (ignoring directories like /tmp, /home, media folders, or other things that you really don't want versioned), every time you make a change to configuration files, you simply check in the changes:

# fsvs ci -m "blah blah blah" /path/to/filename

You could also script that to run at say 4am every morning to check-in things that you forgot to check-in.

It's a wonderful tool for tracking changes to /etc files. Or for figuring out what changed that might have broken something. Plus I can (with the proper permissions on the SVN repository) open up the repository and browse the history from another machine. Such as looking at changes from my laptop running TortoiseSVN and using the visual diff tool of TSVN.

Re:Do we have to bring this up over and over again

[WuphonsReach]

On the Windows side, try Second Copy 7.

On the Linux side, rdiff-backup has a very simple syntax.

I like your layered approach, too

[WindShadow]

Your irreplaceable pool is larger than mine, my really critical stuff fits on a pair of DL DVDs, so only about 14GB (I use "dvdisaster" software ECC in the images). However, I did spring for a Blu-Ray burner, which hasn't gone production yet but promises 20GB+ECC now, and possibly double that when/if 50GB media is available and affordable.

I have one more layer you don't, I inherited a large gun safe, and I have DVD backups in a small fireproof security box in that. That gives me hours of fire protection, and the data is in the format of encrypted ext2 filesystems, copies of loop mounted backup images.

The nice thing about DVDs is that they can be mailed to a secure location without needing a fast network connection.