Slashdot Mirror
Ask Slashdot: Best On-Site Backup Plan?
An anonymous reader writes "I know most people use backup services in the cloud now, off-site, but does anyone have good ideas on how to best protect data without it leaving the site? I'm a photographer and, I shoot 32GB to 64GB in a couple of hours. I've accumulated about 8TB of images over the past decade and just can't imagine paying to host them somewhere off-site. I don't make enough money as it is. Currently I just redundantly back them up to hard drives in different rooms of my house, but that's a total crapshoot — if there's a fire, I'd be out of luck. Does anyone keep a hard disk or NAS inside a fireproof safe? In a bunker in the cellar? In the detached garage? It's so much data that even doing routine backups bogs the system down for days. I'd love suggestions, especially from gamers or videographers who have TBs of data they need to back up, on what options there are with a limited budget to maximize protection."
Make a gigant gluster? Then at least you will ony have to worry about it not catching fire (and not about how many duplicates and where do you store them)
There are offsite options besides the cloud. I shuffle hard drives between work and home. If you work from home, you could do the same at a friend's house or something.
http://iosafe.com
Check out my sci-fi/humor trilogy at PatriotsBooks.
Copy them to a few cheap external drives and mail them to a trusted friend/relative.
Find a friend with same issue and figure out a plan to place two cheap NAS servers one at each house and back up to the NAS at your friends house encrypt if you wish and problem solved.
Works like a charm
megaupload
in the short run and in the long run. Also, storing locally does nothing to protect you from flood, fire, theft, etc... Backblaze is $5/mo, unlimited storage. I'm sure there are others with similar/better deals. What's a NAS inside a fireproof safe going to cost?
"I'd rather be a lightning rod than a seismometer." -Ken Kesey
raid 0, periodically swap a third drive out and put in a safe deposit box.
if you earn revenue from it, pay for backups
if it has sentimental value then think about paying for backups
hard drives go bad all the time so if you're going to back up to hard disk and its important buy a few external ones and keep them in different locations
Are you insane? I think you mean RAID 1.
You could backup to take. Cycle two tapes. Keep one at home and one in a deposit box. Switch monthly. In the event you lost everything at home you would at most lose one months worth of data.
Adjust to a shorter cycle if a month is too long.
I'm a professional photographer and I keep backups of my network drives in a safe deposit box at my bank. I have a couple terabyte drives and then once a month I update the drives at the bank. This may not be very sophisticated but it's simple and it works for me.
Just a note about fireproof safes. They are fireproof in the sense that they will protect paper from getting hot enough to catch fire and burn. However, they will still get hot enough in a fire to destroy any electronics or plastic items stored inside.
First, going strictly by your requirements, I would suggest either a fireproof safe or fireproof drive enclosure. I don't have experience with the enclosures, but the safe itself should be able to handle your normal everyday fire and protect your data.
However, I'd suggest that you don't store your safe at your location at all. Surely you have a friend or someone you know that would let you borrow a few square feet of their basement for the safe. This would create a physcial barrier that would enhance your securiy if not always convenient. I'd also recommend a second copy somewhere else if this data is that important to you.
Remember that as with (almost) anything else, there is a cost-benefit tradeoff. I'm not convinced that a "cloud" based solution is your best bet anyway. But a simple, low tech solution seems to be what you need anyhow.
See my journal for slashdot ID's by year. Mine created in 2005. http://slashdot.org/journal/289875/slashdot-ids-by-year
Backing up to portable hard drives is fine. Better if you have at least two copies of the data, on different drives of course
Keeping your backups on site is great if you are concerned about user error, but offers nothing for disaster recovery. Put them in a safe deposit box. Store them at a friend's house. Put them in a Zip-lock and bury them in the woods. Do something, but get them out of the same building where your primary storage is if you really want to have a useful backup for disaster recovery.
This post comes with a double-your-money-back guarantee!
Any offense taken to this post is at your sole discretion.
Have you thought of getting a safety deposit box at a bank? Usually they're in a fire resistant box inside a fire resistant room.
Store backup copies of disks in there, and swap them out, similar to tape backup strategies.
Crashplan is wickedly cheap, and for Unlimited Storage its worth the buy
You could put some backup drives in a safe deposit box. With as much as you're storing, it may be beneficial to store just the bare drives.
A friend and I just keep external HD's for each other. Every now and again we go to the other's house and swap them. They are small... and fit easily on a shelf in an unused closet....
If you don't trust your friend enough you can even encrypt the HD....
Cheap, effective, works.
Any pro photographer will tell you that 95% of what you shoot is crap. Prune it mercilessly.
Give me Classic Slashdot or give me death!
Get a couple external 3TB drives. They are under $150 each and keep them in a safe deposit box. I store 2 at my local bank and switch them out from time to time. Depending on your bank and balance you can get them for free.
Safe deposit boxes aren't expensive and they're not a bad offsite location to store copies of you data on external hard drives. I don't really like using hard disks for long-term archiving but it's one of the lowest cost practical solutions. A tape drive would be something to look into but they're not cheap.
Why not get a firesafe?
Some of them are rated for higher temperatures than house fires usually attain, and the response time of your fire department should give you an idea of how long they need to hold out for.
If you get one that has a decent lock you can keep your gun and your pot in there without the kids playing with them.
Do not meddle in the affairs of geeks for they are subtle and quick to anger
I have two USB hard drives that I use for backing up. They have the same data on them. I keep one at home and the other in a safety deposit box at the bank. I backup about every week or two, but once a month I will do a backup, then take that drive to the bank and swap it with the other, bring it home and back up to that drive.
I use crashplan plus backup for my online backup as well as local backup solutions. Crashplan Plus is only $5/mo for unlimited data on one computer or $10/mo for an entire family of computers and unlimited data. That is a lot cheaper than buying 8TB or backup drives. Software is easy to set up and recover from . You can use the same software to do a local backup and a cloud backup.
Steve C
Apple Certified Technical Coordinator 10.6-10.7
www.promacs.com
Take your current 8 TB of backed up data (which is on redundant drives you said), store them in your mom's basement... now you have an offsite backup. All new data from now on, store them in Dropbox or some other cloud service. Bite the bullet and pay the $5 a month.
I don't know if you'll ever find online solutions that will store 8TB of data for free/cheap, so I suggest just loading up on the harddrives and putting them in a lockbox. This could be at a friends house, work, a hidden panel in your car, the bank, etc. There are tons of places you could put it, but if you find a cheap online solution that'd obviously be excellent.
I have redundant drives at home, but what about a fire, flood, or theft? Two options:
(1) Put the data on an encrypted hard drive and bring it to work. This is what I do. It is safe in my desk and even if someone at work broke into my desk, they wouldn't get past the TrueCrypt.
(2) Same as number 1 but use a SSD and put it in the trunk of your car if you don't work in an office. (SSD is less sensitive to vibration)
-d
"Here Lies Philip J. Fry, named for his uncle, to carry on his spirit"
If the workplace burns-down, I have the home copy. And if the home burns-down, I have the work copy. The most-important files (resume, government clearance) and small-sized text files (ebooks) I have a triple-backup through Google Drive. 5 gigabytes free of charge.
My AC stalker: " I personally agree with your posts most of the time, but that won't keep me from modding you troll"
'Simultaneous' on-site and offsite backup. Cheap - and they are committed to unlimited storage... I have just over 1TB with them - encrypted with my own key - backed up to their data center and to two separate local locations.
http://www.crashplan.com/consumer/crashplan-plus.html
This looks like it might work for you and I've been investigating it lately. I can't recommend it, given I haven't used it, but it's an option if you have the bandwidth to upload 8TB of files.
http://www.backblaze.com/
Backup your data and rotate drives off-site to a friend or family members house. Storage is cheap, and eSATA works like a goddamn champion.
Due to large changes in amount of data, delta type backups won't be nearly as effective, so just use something like ViceVersa or Robocopy to duplicate your new data to one of your off-site drives, and then store it off-site for free. No need for complex RAID or costly hardware because your goal is to prevent against loss of data due to catastrophic failure or disaster at your residence.
eSATA or USB3.0 (though I haven't tried USB3 yet so I can't say).
Create bogus identities, create accounts, upload the max and get as many free prints as you can. Rinse, repeat. You may need a couple of proxies just in case they're smart enough to monitor the IP address.
Stores all those prints in filing cabinets.
I think it'll only take a few months, 10 hours a day.
But it'll be FREE!
http://www.crashplan.com/
Unlimited backup for $5/mo to the cloud. FREE backup to other computers using their software which is cross platform on Windows, Mac, and Linux. I'd purchase an external HD(s), backup to it then get a friend to put it at their house. You can adopt the backup on their computer and then backup to their computer (FREE) and to your external HD(s) with their software automatically from your own computer.
Or you can just sync it to the cloud, but 8TB might take a while to get everything up there.
FunOne
There's no such thing as fireproof , there's only fire-resistant. Fire will burn through anything given enough time , and safes are rated for the number of hourse they can withstand.
AC because i'm at work but i feel like this shold be known by you all.
Why bother with other storage technologies that degrade overtime. You already have access to cheap (but slow... in this application that doesn't really matter) solid state storage that doesn't degrade. If your objective is to simply archive the content, buy as many SDXC cards as you need to archive your content, and when you go on shoots charge them for the cost of an extra sdxc card to archive the content on and throw it into your safe.
The SentrySafe QE5541 has a USB pass-through that lets you store a bus-powered USB hard drive right in the safe. I've got this hooked up to my primary machine, which pushed my backup right into the fire-protected drive.
Go to your bank, and for around $100 a year you can keep your drives in their vault. You should be able to fit 4 2GB external drives in the smallest-size box (but bring them with you to make sure).
In addition to your "hard drives in different rooms" strategy, consider keeping a copy offsite in a bank safe deposit box.
Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
I just ran a network cable to my shed and run a second NAS unit from there.
1) Buy a few 3TB drives, backup then store in a Bank Safety deposit box
2) Find a friend who also has similar needs, and over your internet pipes give yourselves access to each other's storage drives (I recommend a small home NAS for this with with some level of RAID mirroring). This solution requires twice the storage space you need just for yourself (one set of drives for your stuff at home + X number of drives to store 100% of the required space for your friend). that said, it would likely still be cheaper than option one due to safety deposit rental charges + buying extra drives anyway. For this solution, I'd recommend you do the initial backup of all you have today over 'sneaker net', but then all subsequent backups can be done over your home internet pipe. Of course, this also requires a) A decent internet pipe with good upload speed (a rarity in many parts of North America) + b) Unlimited bandwidth capacity (i.e. a 100G limit sounds like it would last you all of a couple of days).
Good luck!
"does anyone have good ideas on how to best protect data without it leaving the site?"
If it's not leaving the site, you aren't protecting it. At the very least, get a few portable drives and rotate backups to a relative's house.
Onsite-only backup is risky.
Offsite backup is intrinsically expensive, since it involves the costs of a 2nd site.
To roll your own offsite backup you would have to, for example, buy a second house to use as the other site, or rent some space in a warehouse or something.
If you can't afford that, you could always use a cloud backup service, but the bandwidth will be expensive.
If you can't afford that either, then maybe your data isn't as valuable as you think it is.
Of course, the other side of the coin is that you could try to use the space you have more efficiently. Even for a professional photographer, you probably don't need every photo to be in 10+MP RAW format. JPEG with high quality settings is actually quite good. If you used compression appropriately instead of senselessly hoarding every last subpixel of sensor data, you would never have had a problem in the first place.
And on the coin's third side, Moore's law says the longer you hold out before getting a real backup system, the cheaper it will be when you do. Maybe you should just accept risk for a while, and then deal with it later.
final note: Going by cost-per-gigabyte, the cheapest backup media, in order of increasing cost, excluding tape drives and other such nonsense:
$30/TB Single-Layer BD-R
$50/TB 3GB Hard Drives
$70/TB Dual-Layer DVD-R and Single-Layer DVD-R
Buy enough hard drives to make two backups of your data. Keep one copy of your data onsite, ready for a quick restore if necessary . Keep the other copy offsite, in case there is a fire, flood or burglary at your main location(i.e., you really DO want one copy of your data to leave your site). You don't need to pay for a fancy offsite backup service -- just keep them at a friend's house and be sure to rotate your back drives regularly. This is what I do with my 3TB of data and it works very well. Use backup software that automatically copies only the changed data. Otherwise, you will not be able to do a full backup of 8TB every day. On Windows, you can do this cheaply with Robocopy. On Linux or Mac, rsync does the same thing (or use Time Machine on a Mac). You can schedule the backup script to run automatically every night.
If you need to keep all 8TBs spinning all the time, you'll probably want to look at getting hardware RAID devices. If you only need to keep the most recent data instantly available, RAID might be unnecessary.
Drop a box at a friend or relatives home, preferably one with a big pipe. Setup two storage systems, I would look at ZFS on FreeBSD (FreeNAS perhaps). And then setup async replication rules. You write to one, it will eventually catch the other one up, constantly. If yours fails, go get the other one, put them on a local highspeed network and clone again. You can increase reliability with more georedundancy. More boxes, more places. Since you are not replicating the entire dataset daily, only the change state, it wont bog things down badly.
Any company that I've ever worked for that had money to spend did tape backups and stored them in a vault offsite. Tapes get verified as they're written, and don't have parts that fail like hard drives do. They have a 30-year shelf life, and you'll always be able to find a way to read them in the future. Go to ebay, buy a used LTO3 or LTO4 drive, (400GB and 800GB uncompressed, respectively). Tapes are about $25/ea for LTO3. Then put a backup somewhere safe.
I see this all the time with photographers. Bottom line: your photographs are not all that valuable. Some are, yes. Most are not. Pare them down. Delete the bad ones, the failures, the misfocussed, the bad exposures. The greatest photographers the world has ever known are only known for a few dozen photos at best. Do you really need an 8 TB photographic archive? Who's going to ever look at them all? Save the best. Delete the rest.
Speaking as someone who has to manage the backing up of a good few hundred TB of data each day, as a home solution I would use something like Amanda with a cheap tape drive off eBay onto LTO then stick the tapes in a safe deposit box or under a rock somewhere. It's a cheap method of duplicating how its done professionaly
Not to sound like a shill (I'm a fanboy, which, while different, will sound somewhat similar in practice), Crashplan has a free option available where you and a friend can both run it and can use it to back up to each other. If you have a photographer friend (ideally in a place far enough away that you won't be hit by the same natural disaster), this can be a pretty good option. It'll likely take awhile to do the backups, however, and you'll also need to have adequate hard drives on hand to store not only your own work, but also your friend's, which may get in the way of going cheap.
That said, for $140 (the price of a hard drive or two) you can get a 4-year subscription for their cloud hosting with an unlimited backup size. The company I work at uses their business-level product, and I recently started using Crashplan+ at home for my own computers. While it does take awhile to back up, it's painless to do so. At least so far, I prefer it quite a bit over Carbonite, which is what I was previously using at home.
Give your mom a box of backups and ask her to hold on to it, it is "stuff you made"
She'll never get rid of it.
and if the house catches fire, it will be the first thing she grabs when she runs out.
I like microcars
Set up a RAID 6 array at a friends or relatives house. Do an initial dump of all of your data to it before you bring it over to them. Offer to pay their Internet bill in exchange. Set up a VPN and run rsync between your place and theirs.
That has got to be about the cheapest and simplest off site back up you can possible have. You can even write off the cost of their Internet as a necessary business expense if you can get a receipt (since you are a photographer for a living and not a hobby).
You have to ultimately ask yourself a few questions:
1) How accessible does this data have to be.
Do you need quick access to it any time of day? Will there ever be an emergency need for access?
2) How secure does it have to be.
Should other people be restricted from accessing it? If someone stole it, what would the ramifications be?
3) How redundant should the copies be?
Raid levels? Mirroring? Multiple copies?
There will always be compromises between these issues, and you'll have to come up with a solution that makes you the most comfortable. Is there a 100% secure way to guarantee against data loss? Nope. The whole point is to mitigate against casual loss and/or minor catastrophe. If a meteor comes out of the sky which is the size of Texas, well, no backup plan is going to save that data.
So some options: Find a friend who has broadband and put together a NAS (FreeNAS ZFS box with multi TB drives) and see if your friend will host it. You can transfer your data across the net to it, and it gives you 24/7 access to the data. Then, as a secondary backup some WD 3TB USB drives in a safe deposit box. This gives you multiple locations, fire/theft proof backups, and hardware redundancy. Might cost a few hundred bucks to put together, and some time each week or so (depending on your shoot schedule) to maintain, but it gives you security and flexibility and redundancy. Once the system is set up though, it should be pretty simple to keep going.
I suspect that what Hatta said is completely true. I doubt that 100% of what you have is really worth keeping forever. I take photos when traveling and some are mistakes or just didn't turn out that well. One of things I like is that if I take enough photos, some will turn out to be really good. Note that I said "some" not "all".
If you have a friend who doesn't mind you could make a backup to the biggest hard drives you can afford and have your friend store them. I'm an IT guy and on a previous job I actually was authorized by my company to keep a set of backup tapes for our development servers at my house so we didn't have to pay an offsite storage company to store them.
I have a question - are you really going to argue that your photos are so valuable that you can't let them go into the cloud for fear that others might get copies? I'm in no position to judge whether that is true or not and even if you say it is, the rest of us may disagree. I know that some professional photographers have some really warped ideas about their own work and think that the wedding video they just got hired to shoot belongs to them and the paying customer is some kind of leech they'll deign to give one copy to under duress. If you got oodles of old wedding photos, for example, that you were hired 10 years ago to take, I doubt that it's really critical that you keep copies. I participate on a video forum and we see people all the time who want to watermark stupid crap like videos of their kids playing tee ball* because they are so delusional that they think the whole world wants to see their kid. I get that these photos may be just incredibly important to you, but do you really think that others are just waiting to steal them if they go on the cloud? Cloud backup would be your most cost effective solution.
* For the non-North Americans, "tee ball" is version of baseball that small children play in organized leagues. They hit the ball off a tee, hence the name.
Not your next door neighbor (particularly if you are in an apartment). You need a friend who is far enough away that the same disaster that hits your home will probably not hit your friend's home.
You can ignore this advice in cases of asteroid impact, zombie apocalypse, super volcano or robot uprising.
Who can you trust? Yourself! Burn them onto double sided blue ray disks and keep them in two of your locations in a fireproof safe.
The purpose of existence is to make money.
If you're going to put things in a fire rated container, there are a few things to consider. Those containers are not "fire proof" by any means. Get one whose rating is reasonably high as they will buy you some time.
Most house fires are either a basic 'room and contents' or a much more involved fire where whole floors are exposed (and largely consumed) by flame.
When you put your fire rated container somewhere, consider that fire burns upwards, and the thermal difference from floor to ceiling is around 400 degrees F on average. Before you put the container in the basement corner, remember that firefighters use water to put out fires. Lots of water. 150-200GPM per handline and 1000-2000GPM for the big pipes on the ladder trucks. Most of the damage in a house fire is from water. You'll get us much as 6-12 inches of flooding per floor (until the firefighters cut holes in the floor to drain it so the floors don't collapse.
Also should the roof or ceiling collapse, the best places to have things are near the corners of the load bearing walls.
This is my long way of saying store your fire rated container on a solid hardwood (not particle board) or metal shelf, about knee height on a low floor near the corner by load bearing walls. This way in the event the whole house is a write off, you still have a reasonable chance of saving some of your data and personal effects.
"Omnis tuus capsa sunt inesse nos"
I use a backup service called Crashplan. They have clients for Linux/Mac/Windows and support backups either to your local network (free), "friends" machines in a p2p type configuration (free) or to their servers (paid). Everything's encrypted locally and the client app is pretty decent IMHO. Best of all is that the paid plans are pretty reasonable - I have the unlimited plan for something like $100 a year, and it really is unlimited (well, they claim it is and I have no reason to doubt them). I currently have about 3TB up there so I don't see why you'd have an issue.
The way I have it configured is that all the machines on my network backup to both my local fileserver and to their cloud. The local backup has a higher priority so any changes get pushed over the lan immediately and then batched up and sent offsite over the slower link. Speedwise I can't saturate my uplink when uploading to them but I get a pretty consistent 1-2MB/s, so figure maybe 100GB a day? I think my initial seed took a couple of weeks. I've done a couple of small test restores and download speeds were similar (although in all but complete disasters I'd be restoring from my local fileserver which is obviously far faster).
Disclaimer - Not related to the company in any way, just a very happy customer.
---- Den ene knappen er powerknapp, den andre er Bender voice knapp "Bite My Shiny Metal Ass"
I use MyCleanPC along with a custom hosts file to redirect my documents to my online backup service. I'm mostly backing up my GameMaker projects.
I use backblaze http://www.backblaze.com/ for off-site backups. $50/year for unlimited storage is more than reasonable. I currently have about 2.5TB backed up there.
`fortune -o`
Look for a used LTO3/LTO4 tape drive, then bulk-buy tapes.
Write each set of content to two tapes, ideally of different brands, and store in different places if you're really concerned.
I've been backing up to LTO3 tapes for ~3 years now, i've got 50+ tapes, mostly in my safety deposit box at the bank (cost $75/year)
LTO4 based on eBay prices right now would be an initial expenditure of ~$1k for the drive, and $25-30 per 800GB of storage.
The cloud options aren't really feasible for me, as the upload time & bandwidth cost is horrendous.
ICQ# : 30269588
"I used to be an idealist, but I got mugged by reality."
What's a good way of ensuring data integrity (and possibly repairing any corruption) that might happen? Is two copies and a checksum enough to be able to reasonably repair a (not too) corrupt file?
Nothing sucks like a Vax, nothing blows like a PowerMac G4
Since the vast majority if your files aren't changing frequently, use daily an incremental backup. There's options out there which will let you run it at the end of the working day and turn off your machine once its done.
Can you throw some cabling into the garage? Else WiFi would probably do. Stick a NAS in there, configure the software, make a once-monthly entry into your calendar to check the backups are viable and you can forget about it. Well, until winter maybe.
Run a second backup say over the weekend and swap drives at a family/friend house. Safety deposit box if necessary.
After each shoot, be realistic about what photographs you keep. I'd wager a smallish fraction of what you're keeping now. Remember there is some overhead cost in retaining those files, nevermind in hard drives but in your time sorting through it.
Take your data, store it on several independent HDD's (not RAID).
On thanksgiving or christmas when you visit your family, take the drives with you and store the drives there.
Be selective. Back the good ones up - to a USB key.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
I started this long, convincing post about why cloud backups are so much better in terms of durability and availability. Then I looked at the cost.
Holy smoke! According to Amazon's handy cost calculator, your 8 TB of data would cost $915.86 per month to store in Amazon's cloud. I would argue that kind of cost may be acceptable to back up your entire livelihood, but that really depends on your cash flow, doesn't it?
My new recommendation is to burn your pictures to DVDs or blue-ray discs and bury them in your back yard. :-) Oh wait, I mean keep the discs in a large-size safe deposit box at your neighborhood bank.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
You don't need online backup unless you have to be able to get the data back instantly. In your situation you don't need that--a few days latency is fine.
Just get yourself a tape drive or blue-ray burner, and get a relative in another state to set aside a cardboard box in their closet for you (pay them for this if necessary). Back up to tape or blue-ray and once a week or so, mail the tape to your relative, who tosses it into the box. This is your last resort backup--of course you should also have normal backups at home.
If something happens where you need the remote backup, then arrange with your relative for her to ship the box to you with you paying the freight bill, again not all that complicated.
As a photographer myself (Though only around 4TB of photos at current) here's my setup:
Onsite backup: Drobo to Drobo.
Offsite backup: Backblaze. I pay $4/mo (2 years prepaid). This is a secondary backup. It still has everything, but I rely on the local backup to retrieve something should my primary storage fail, and the offsite is for when things burn to the ground or someone steals my stuff or lightning takes everything out. Really it's cheap, it's just that initial backup which takes an eternity and might get you in trouble with your ISP. Fortunately I was able to use the local university 1gbit connection to reduce my initial backup time to just 18 days (straight).
Zing!
I am betting that you can recover that from your friends.
A lot of people are telling you that you're wrong, that you need "offsite backup", etc, etc. But that's not what you asked. You've got an external Hard drive already it sounds like. Most banks will rent a safe deposit box for fairly cheap. This will give you that "Fireproof" that you need. You're either going to need a computer with USB 3.0 or external hard drive(s) with ethernet capability. My suggestion would be something like a DROBO, spend the extra money to get two sets of hard drives (say a grand total of 8 3TB hard drives for the Drobo). Then do the following:
1. Mark 4 of the hard drives as "Week 1 (Drive 1,2,3,4)" or "Week A (Drive 1, 2, 3, 4)"
2. Mark the other 4 as "Week 2 (Drive 1, 2, 3, 4)" or "Week B (Drive 1, 2, 3, 4)"
3. make sure your computer has a gigabit network card in it (The drobo has a gigabit card) and get a consumer grade gigabit switch/hub.
4. Whatever week you get all this set up is Week 1, on that Thursday or Friday start a backup of the data to the Drobo, there is a Command Line command that will compare the current folder to the drobo and only copy changes (it's been a while since I've used it, I want to say robocopy, but do some googling to find it and all the variables)
5. That Saturday (assuming the backup is done) drop all 4 disks off at the bank in the safe deposit box, load the other 4 drives into the Drobo.
6. The following Thursday or Friday run the copy command again. That Saturday go pick up the drives from the bank and drop off the other drives.
7. Repeat weekly.
Depending on how often you add more files to the system and how secure you want to be/how much of a hassle it is to get to the bank, this could become an every 2 week or every month thing. Conversely it could become a every 3 day thing. It depends on how much data you are willing to lose in the event of a total catastrophic loss (tornado, fire, earthquake, etc) vs. the cost of going to the bank, cost of hard drives, etc. The initial costs will likely be heavy, and there will be a yearly charge from the bank for the safe deposit box, as well as the cost of gas for a trip to the bank every week and other costs I might not have figured into the equation, so you really do need to do a full cost/benefit analysis and decide if this, a cheap colo + Drobo, or renting space on an existing DR company's server would be the best use of your time/money, but there is an honest answer of how to do a self-ran DR set up.
Hope this gives you some things to think about.
I'm a photographer too (albeit semi-pro) and don't have that much data, about 2TB at last count. I use CrashPlan, they offer both cloud ($6/m unlimited data), local and shared (backing up to a family/friends computer). You can seed you initial backup to the cloud with physical disks sent to CrashPlan.
Take a look at high-rely (dot) com or tapesucks (dot) com. They have pretty large removable drives that mirror themselves for additional redundancy.
This is one of the few cases where external drives can be a good idea. The data doesn't changes much so there is no need for a daily/weekly backups. Get four 2TB drives (or three 3TB drives) for the old pictures and store them off-site(friends/relatives place?).
Then you only have to worry about the new stuff, and that can be backuped daily on-site and biweekly/montly/whatever off-site.
Upload your photos. All of them. To flickr, facebook, whatever. The good ones will survive. The great ones will be shared. The ones you're ashamed of will go viral and you'll never get rid of them again, no matter how much you want them to.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
There is no "safe" on-site storage option. My brother had an expensive expensive fire-proof gun safe at his house - it weighed over 500 lbs.... After his house literally burnt to the ground (rural area, no one home to alert the first department, by the time the neighbor down the road saw the fire and called the fire department, all they could do was put out the fire in the surrounding brush (and cars)), he couldn't even identify any remaining pieces of the safe or the guns that were in it in the debris.
If you want your data safe, store it off site, preferably in another part of the country so it's not subject to the same local disasters as your house. Mailing snapshots of your data to an out of state relative is probably best.
I'm also a photographer, don't make a lot of money on it, and I also backup to hard disk (because nothing else is big enough. Blu-ray? It is to laugh.)
My strong (really strong) recommendation: Keep at least one backup set offsite. This is a really really strong recommendation. As a photographer, your images are absolutely the most valuable commodity you own. If your equipment is stolen, burnt or dropped off a bridge, it would really suck, but you could replace it. You can't replace your photos.
This protects your original content from catastrophes up to geological level. If your house burns down or washes off a cliff, you'll lose your most recent photos, but the bulk of your work will still exist.
To protect against geo-scale catastrophies, keep a set at a remote relative's house, swapping in a newer backup each time you visit. I have relatives at 200 and 600 miles away, not really far enough for widespread catastrophes, but better than nothing. I bet you could think of some prospects.
In summary, I don't depend on fire proof safes or detached garages. For protection from natural disaster, nothing beats your data not being there when it happens.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
big ass-magnets
Fireboxes are designed to keep paper from bursting into flame. Sure, your hard drive is mostly metal, but that PCB board on the back could melt, warp, lose solder, etc.
I guess the part I don't understand is how any of the solutions involving buying another hard drive (or three) and securing them are somehow cheaper than a tool like www.crashplan.com that is $50 a year for unlimited storage. Sure, the upload of 8TB will take a while.. so what? a single T1 (1.5Mbs) line can upload a few TB a month.
Then its all offsite. Worst case, just download the crashplan sofware, and have a buddy install it too, and use the free person to person backup feature (only pay to store on their servers) to have an offsite backup. Have him bring his PC over for a night, to sync up the initial copy.. then you will just send changes over the internet.
What are we going to do tonight Brain?
You don't necessarily have to go "cloud" to go cloud.
The company I work for has rack space in several colo centers. Some customers just want space for their own servers, which we're happy to rent them. You could buy your own RAID storage server, and buddy up to a local IT company for a bit of rack space. Saves a lot of trouble over swapping & carrying drives around. You wouldn't need high-reliability stuff like redundant power supplies or blade processing -- it's just storage.
Heck, you could go super cheap, and get one of those rack-mounted lock boxes and just lock your drives in it, inside a locked cage, inside the data center. Climate controlled, fire safe storage. Bonus to that over the bank safe deposit is 24-hour access.
I know it's off-site, which isn't what you were looking for, but just throwing that out there. That's what I'd do... I'm not impressed with virtualized cloud stuff. And I work with it!
That I'm right, and you don't like it, doesn't mean I'm a troll.
Hackaday recently featured a similar question. You might want to go look at the responses there as well:
http://hackaday.com/2012/08/04/ask-hackaday-whats-your-backup-solution/#comments
I use backblaze -- $47.50/year for a two year term and unlimited storage. For the mathematically challenged among us that's $3.96/month. Skip a couple cups of coffee a month and sleep better in more ways than one. As a bonus they show you how to build one of their 135TB storage pods here http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/ .
I have been using Mac OS X with ZFS ( i have a 6 Tb /media partition that has music, mp3, DVD, Bluray, HDmusic, work, subdirectories. I have a 4 disk Esata container that holds the 3 3Tb drives that make up the raidz1 partition. I have purchased 2 more Esata containers, there is a raidz1 backup partition that backs up using rsync weekly (I just ripped 2 new CD's, an entire backup took 3 whole minutes). Backup A is rotated to my offsite secure storage location, and backup B returned and is remounted. This takes 9 3Tb hard drives altogether, but I have spent $$$$ on the CDs, DVDs, and BluRays and wouldn't care to have to rerip them!
These questions pop up a couple times a year, and the ultimate answer for cost effectiveness and minimal complexity is always the same.
Buy storage in groups of two or three (I'll explain the different options shortly) drives at a time. So let's say your 8 TB can be covered by three 3TB USB HDDs. So you buy a set of three 3 TB HDDs and that's your master copy that you keep at home and work from. Now duplicate that exact same setup and that's your backup copy. This gives you protection against human error (accidental deletion, overwriting, etc), hardware failure and filesystem corruption. Now buy a third set of drives. This is your offsite backup which could live in a safe deposit box, a friends house, etc. This gives you protection against events such as fire, flood, power surge, theft, etc which cause both the primary and onsite backup sets to be lost. The onsite and offsite backups should never be at your house at the same time. Periodically (you can detemine the best schedule for your situation) you should take the onsite backup and move that offsite, and conversely bring the offsite backup onsite. The schedule at which you do this would determine your RPO (amount of data you would lose if your house burned down).
The initial sync will be painful, but afterward you can use just about any sync software (there are many freebies, as well as many commercial) to keep the two backup sets in sync with the primary storage. Just let it run overnight while you're not using the system.
If you get to a point where you need to add a 4th drive, buy three identical drives (one for each storage set). Periodically you'll want to refresh the drives to stay current with technology and minimize the risk of hardware failure.
As mentioned, this approach does have the drawback of a slow initial backup, but EVERY solution is going to have that and HDD to HDD backup is likely going to be the fastest and least suck method of doing it. This is also just about the cheapest solution, and doesn't rely on any special hardware or software.
At the densities of today's HDs bitrot is not a question of IF but WHEN.
Don't get a 4 drive drobo, they do NOT have scrubbing of data, but the 5 bay and up ones do. Without scrubbing once a bit flips your picture is ruined.
Whatever solution you go with, either the file system or the hardware needs to have data scrubbing built in.
Cwm, fjord-bank glyphs vext quiz
First, you should have a DAM (Digital Asset Management) software such as Apple's Aperture (awesome!) or Adobe's Lightroom (meh). As other have said, you can trim/delete the shots you don't need. Then store your DAM (okay to use "damn" here to) library on the NAS and share via NFSv4 (OS X or 'nix) or CIFS ('doze). Now consider converting some RAW to only JPG or PNG or TIF to save some disk space.
As others have stated, you can, and should, have some cloud storage. You can setup your DAM to have high-quality previews or the RAW files, then only backup to cloud the previews. The previews even at high quality might only be 1-2% of the library size with RAW files. This provides cloud storage at smaller data requirements.
As for offsite copies with massive storage, ZFS makes it easy to break a mirror and take offsite. Then you just bring it back, resilver, and presto, you're up to date again on both mirrors.
Some details really glossed over, but I'm sure you get the idea.
I hate the cloud, and the reason is that someone can get into it period!
If you have questions about this then i suggest you keep thinking about it for a few more times until it makes sense!
I'm here for the experience, not the Hyperbole.
Basically you need to look one of the many answers that were suggested yesterday:
"Ask Slashdot: Simple Way To Backup 24TB of Data Onto USB HDDs?"
http://hardware.slashdot.org/story/12/08/10/026256/
First off, your initial sync to any storage medium will take a long time. Once it's done, automated (!) backups/incrementals should be less of a big deal.
As for me, depending on the budget and/or physical space constraints (which you don't list for brevity's sake (?)):
Basically get an enclosure of some kind and stuff a whole bunch of hard drives in it. Connect it to a machine via eSATA and use MD/LVM/ZFS/Btrfs to create a backup volume. Install some backup software on the machine (Bacula, AMANDA, rsyncd/rsnapshot, CrashPlan, Time Machine) and point all the other machines in the house to it. Make sure it has a GigE NIC (or multiple in LACP).
The initial sync will be a pain. Let things run for about a month with incremental backups being done overnight in the background.
After a month (or when you get a decent amount of cash), buy the same enclosure and fill it with drives. Unmount the first one and attach and set up the second as the first. The initial sync will be a pain again.
Take the first drive enclosure to a (geek) friend's or relative's place. After 1-3 months swap them again. Keep doing this forever.
You may want to add a third or fourth enclosure over time so you get some "wear-leveling" out of the drives, and perhaps to distribute the enclosure over more locations. Add encryption if you're paranoid.
Even if you don't want it in the cloud, CrashPlan has a good PC to PC software (Free) that's very friendly and allows version control. Either way, you are a fool if you don't have an offsite. Most thefts or things worthy of an insurance claim would destroy or make unavailable your backups. Find a family member or friend with an acceptable internet connection, trade them external hard drives and allow reciprocal backups over the wire. Reality though, maintaining multiple TB drives is more expensive per year than paying a cloud service.
tar zcf - photos1/ | gpg -e | curl -X PUT --data-binary - http://me.s3.amazon.com/photos1.tar.gz.gpg
... or whatever the appropriate command/url is.
Oops, just checked the pricing. It will actually cost about $900 per month for 8TB. NM.
...from Robert X. Cringely's blog, August 8th, 2012.
Post URL: http://www.cringely.com/belt-suspenders-cloud-storage-122387
He talks about his setup using ClearOS: http://www.clearfoundation.com/software/overview.html
Backup "Best Practices" Checklist
* Stable / Works Every Time
* Automatic
* Different Storage Media
* Fast
* Efficient
* Secure
* Versioned
* Local AND Offsite / Remote
* Restore Tested
For data that is really important, having just 2 copies is not enough. You want more copies.
I'm shocked with people "backup" their data to a USB drive, then delete it from their main laptop or desktop. If there is 1 copy, then it isn't a "backup."
RAID is not a backup. Learn that, know that, love that. RAID just means that your corrupt data is stored twice AND is still corrupted.
So if you are looking for a any backup total solution, you want all those things. Off the top of my head, I know that duplicity-based solutions cover those things.
Safe. Deposit. Box.
...just make a copy of the file in the same directory... anything else "on site" can't really be much better than this
:)
having said that, i was stupid enough to by a netgear rnd4000 nas with a usb drive plugged into the backup port on the front panel with backup schedules at 12am every day
the backup usb drive sits on top of the nas, and the nas sits beside my workstation
if i were smart i would get another usb backup drive and rotate them with one kept offsite at work or something, but nas covers hard disk failure, the usb backup covers acidental file deletion (losing only a day's work at most)... if there was a fire that would be bad, but so many problems in the world are only resolved after a major catastrophe forces the hand of those responsible, so who am i to buck the trend
Crashplan is likely to be a reasonable option, including the free version, particularly if you're working from an office that is separate from your home, or if you have a friend, relative, or given the size of your daily data increase perhaps just a neighbor who's willing to accept a machine for you. Notes on that at the end.
What you'll want to do is set up a backup machine with plenty of disk space. This will likely need to be a single monster volume and you probably want to ensure that it's expandable; I can't help with the details of that but others here certainly can. What are the good expandable RAID options?
The backup machine will receive the initial backup sitting next to the source machine, but will then be moved offsite. Given the volume of data you described, creating a "seed" backup and just moving that would likely be more hassle than it's worth.
All machines involved should have Gigabit Ethernet, and I strongly recommend investing in a Gigabit network switch or better, an ABGN router with Gigabit ports. If you can find one, I like the WNDR3700v2 (that v2 is REALLY IMPORTANT and hard to find these days) running OpenWRT, but there are plenty of other options. You're still going to need an Internet connection for the machines to identify each other (I believe this uses Crashplan's servers even if you're not backing up to them).
Once you have all that set up, you're just going to install Crashplan on both machines, run your backup, and move the backup machine to its new home. Crashplan assigns unique IDs to machines and uses those for coordinating the backups between them, so the fact that the backup server has moved won't cause a problem.
Finally for the note about "just a neighbor" - if you're creating tens or hundreds of gigs of new data per month, just hosting your offsite backup may be a problem for many people (think bandwidth caps), and transmitting it may be just as big a problem for you. If you have or can find a reasonably close neighbor that you'll trust with an encrypted copy of your data and who's willing to host your machine, I'm going to strongly suggest that you set up the machine with a good wireless connector and have it in their house but on your wireless. Ideally this person would not be in the same building you're in (assuming condo/apartment). Wireless-N at 2.4GHz is likely to be your best bet for range, though it's more prone to interference than 5.5GHz.
fencepost
just a little off
Get a safety deposit box at a local bank. They don't cost much and if you drop a backup HD or tape in it once a month you should be pretty good.
I would suggest storing the data on a RAID drive set for redundancy first of all. See:
http://en.wikipedia.org/wiki/RAID
Then I would suggest having a Wi-Fi connection to another RAID redundancy drive set in a building separate from your house. This could be done with Ethernet cable or such instead but I like the Wi-Fi because it leaves no wires to pickup lightning blasts. We get a lot of lightning. Wi-Fi is essentially immune to the lightning in the real world test lab here on our mountain. Ethernet cabling not so even with tons of surge suppression. Of course, implement necessary encryption and security.
Then setup a remote backup program that copies changed files from the primary RAID drive set to the remote RAID drive set every night.
Then if you really care enough to do the very best, setup a third drive that gets backed up to weekly. And if you're paranoid, enough, then another for monthly backups. Then yearly. Then century...
All, of course, surrounded by high tensile electrified fencing and 300 Ninja guard pigs.
We installed a wireless link to my mother-in-law's place. We haven't done so yet, but the intention is to have some form of fileserver for backup there. We should expect around 4GB an hour transfer rate.
https://plus.google.com/photos/103933303525261507105/albums/5692174876583065521
Photos of wireless link
http://dpbestflow.org/links/39
All you need is a waterproof box, a shovel and some accurate GPS co-ordinates. It's waterproof so you should be safe from a flood. It's outside so you should be safe from an house fire and it's underground, so you'll be safe from any break-in's!
Simple: ( if you are using Linux that is)
Use 2 USB hard drives.
Store all your files to #1 /usr/bin/rsync -avz /media/my_drive1/ /media/my_drive2/
Using cron, take an Hourly snapshot of #1 and copy to #2 using RSYNC (very very fast), this syncs drive 1 over to drive 2, only moving the changes.
0 * * * *
Monthly, use the delete option to delete any files on 2 which have been removed from drive 1. This gives you some time to recover files accidentally deleted. /usr/bin/rsync /usr/bin/rsync -avz --delete /media/my_drive1/ /media/my_drive2/
30 1 1 * *
Expand this to a 3rd drive if you want, and sync it, then remove it offsite, to a safe, or whatever.
good luck!!
Jim
Right now, your best option is tape backup, buried in a weather-proof box. Fire won't harm it under the ground - only moisture. And nosy neighbours who reckon you've buried cash.
A couple of 'burial mounds', with tapes rotated out every year, should be about right.
As technology improves, the ideal backup solution would have the weatherproof box contain SSDs sitting in cheap USB3-SATA caddies. USB cables would come out of a weatherproofed junction and rise to the surface through a snorkel. There, a small solar panel would power the SATA caddies as the USB3 equivalent of a Zigbee (wireless USB) chip. Hey, presto - onsite but still "remote", online backup! Or the data links become fiber-optic, you could probably string a long USB and Thunderbolt directly to the snorkel.
I have tried CrashPlan a couple of times on Windows and Linux, and had to give up. It would either fail to connect, or make very slow progress. It's not my broadband, since Mozy (Windows and Mac only) is fine. I also found that on Linux it would really hammer the system when backing up (4GB dual-core system) so it was barely usable.
Possibly CrashPlan's cloud service is the problem, but I'm not very impressed with the software.
For Linux and Mac backups, it's worth using something like rsnapshot, which is rsync-based and works very well to back up over 1 TB of data. It doesn't do block-level incremental backup, and it makes complete copies of files (rsync plus hard links) but it works incredibly well without writing shell scripts. It can do hourly, daily, weekly, monthly and yearly backups, and has automatic retention i.e. purges the oldest backups in a predictable way (say after 6 months or 5 years).
rsnapshot works well for Linux and (I believe) for Mac, as long as you don't need fully bootable backups, and it should work really well for photos as 99% of them won't change after being created.
rsnapshot is very similar in concept to Time Machine on Mac, but without the nice GUI (in fact, without any GUI). Your files end up in a big file tree and can be restored with any file-copy tool.
http://rsnapshot.org/
64 GIGABYTES A DAY? Unless you are doing NASA imagery that will be analyzed for the next years, you are doing it wrong. Seriously.
If you storing it uncompressed, you are most likely wasting space. Even very slight compression will be invisible in practice* and can cut down the file sizes dramatically. You can easily get down to 25-50% with no loss in visual quality. If you don't believe me, do try it.
(*) - note that what matters to people looking at the photos is practice, not theory. What matters is also that any manipulation of them is 3-4 faster. I am not talking about blocky overcompression like we had in the nineties (and still see today in crappily compressed video streams). I'm a talking about compression that you cannot see, like high-rate lossy audio compression that is indistinguishable to the originals in double-blind tests by audio experts.
Make an encrypted archive and upload them on some newsgroup server.
It's probably the most reliable and cheapest solution there is. Retention is also really good.
Hi,
Many people suggest using hard disks and storing them in remote locations like bank deposit boxes. Don't do it. The day you need the backup you'll find those hard disks don't work anymore. Electronics die even when not powered. I know people who suffered this kind of problems.
You must use a powered RAID-1/5/6 setup so that you know when a hard disk fails, and can replace it before data-loss is fatal. That is the only way to know your offsite backup remains in a working, readable state. I think a powered NAS in a friend's home with email notifications should do it pretty well, and won't draw much power.
Cheers
Enlar
That seems like and awful lot of images to be capturing. Yes digital 'film' is cheap, but your time isn't.
I suggest learning how to use your camera better, how to compose the image better, think about your image before capturing it, and generally shooting less. Out of all those images how many are really usable? how many are cream of the crop? Use some of the techniques above will actually give you a much higher percentage of awesome pictures, and help reduce your work flow as you go thru and picking out only the few images that are the very best. If you aspire to be a pro, you should really only be showing your very best images.
The one exception I can think if is for sports, or some other high-action activity where you'd need to continuous shoot to get the image, but even then 3-5 should suffice.
People have been doing backup right for decades. I suggest you pick up the o'reilly backup and recovery book: that's how to do it right.
I recommend tape. They might not be as convenient as some complex NAS solution, but the data assurance is there. LTO drives verify data as it is written to the tape and rewrite errors as they go. For LTO the advertised unrecoverable bit error rate is 1 in 10^17. Desktop hard drives are advertised at around 1 in 10^14. Tapes are tough and are designed with travel in mind. That isn't the same thing as taking a hard disk that was designed to sit in a desktop and slapping an external case with a usb port. Tapes last for decades...
Erm, sorry. I get a little wound up about these damn kids and their new fangled backup to disk in 'the cloud' who think RAID saves all the world's children.
In your situation the data don't change much, so you can really wind up the incremental backups to 6 months or a year. Take them down the street to a friends house and bring a case of beer. Or better yet, send them to your grandma in New Jersey. She'll appreciate the attention! Best: do both! Your karma will go up so much you'll never have to use your backups and eventually you'll wonder why the hell you ever worried about backup in the first place. Then you'll stop the regimen and lose all your data.
Go to the library. Pick up the book and read it to some kid. Save the children. Like Levar says... "you don't have to take my word for it".
Why not, you make backups on tape and store them in a safe deposit box or fireproof safe. It's a great way to store data. If it's in a good environment it'll last 30 years.
As far as I know, BackBlaze.com lets you backup as many things as you want for as low as 5$/month (but you can save on longer periods).
Consider this:
- $3.96 for 2 years, $95 in total. It's less expensive that buying blu-ray dvd recorder and blu-ray dvds. Plus, it won't get obsolete: it's web based.
You may want to get a faster internet connection: here in italy, for example, a very well know ISP lets users increase upload bandwidth to 4 mbps for 4 more €/month. It should take some time (maybe months, but hey, they're 8TB!!) to upload, but once done you can upload all your new data by night.
Set up a NAS with 9 3TB SATA drives. rsync your data to it.
The NAS should be a RAID 01 or a RAID15 or similar. The idea would be to have two sets of three drives. e.g. sdc sdd sde make up md3 and sdf sdg sdh make up md4. Make a RAID-1 of md3 and md4 (call it md5). Format your md5 with ext4 and do your rsync. When it's time to move your data offsite, break fail md4 out of md5 and remove it. Dissassemble md4 and pull the drives. Replace the drives and rebuild md4. Put md4 back into md5 and wait a long time for it to resync. Take your md4 drives and bring them offsite (to a friend's house or a bank safe deposit box).
For security, make a LUKS volume on top of md5 and mount that. You could do all of the above with zfsonlinux in a couple steps if you want to do that (though it has some beta-ish edges at the moment).
For really budget get external eSATA adapters and an eSATA card with multiple connectors. The freestanding units where you just slide in a drive are nice. But, that's really ghetto and you'll be happier with a hotswappable setup, if you can afford it at this point (or go ghetto now and upgrade later with no changes to the drives).
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Use Backblaze: www.backblaze.com $5/month per system, you just need a beefy internet connection
Use CrashPlan Pro: www.crashplan.com $8/month per system, allows local backup as well
or finally
Use Synology NAS, replicate to secondary device and place in a datacenter or another location
All these require big internet connections though.
I am a photographer myself, this is what I do: I store all my photos (both RAW+JPEG) in a Subversion repository (I guess Git could be used as well, however, I started doing this way back, before Git existed). I have a workstation on which I do post-processing of my photos. The photos I work with are in a Subversion Working Copy. I "commit" the photos to the Subversion Repository which runs on a small server with some external USB 3 harddrives on my local network. I also have a two spare external hard drives that I periodically copy the Subversion Repository onto. One of these drives is always stored at my parents' place. My parents live a few hundred miles away. Every time I visit them, I bring the other extra hard drive with me and switch it for the one at my parents. This way, I always have an off-site backup at my parents'.
My workflow is this:
This gives me several advantages:
I notice the following disadvantages:
thomasdamgaard.dk.
The OP mentions limited budget and huge volumes of data to back up locally. Working within these constraints, a solution comes into focus:
FreeNAS + rsnapshot. Both are free ($0) and accomplish essentially what Time Machine does for Macs, but to/from almost any hardware. Bear in mind that any solution offering any semblance of security for 8TB of data won't be cheap.
Probably the biggest investment will be the NAS box itself, and of that, the HDDs will most likely cost the most. 8TB of RAIDed storage will easily cost >$500. The other hardware need not be much, it just needs enough SATA ports and power to run the HDDs, plus a GigE NIC.
rsnapshot keeps very intelligent backups, only recording diffs between backups using hardlinks. It's not too difficult to set up, and it's totally automated. The net result is a bunch of, well, snapshots, going back a few hours, days, months, even years if you like (and have storage for).
There's just no way to offer a fireproof solution for this much data without investing serious cash. Definitely store the NAS as physically separate as possible from the original data. The NAS only needs power and a single Cat6 cord, so it's conceivable to place it in a detached building or something. Keep in mind, heat and humidity will become an issue in extremes.