Slashdot Mirror
Ask Slashdot: Simple Backups To a Neighbor?
First time accepted submitter renzema writes "I'm looking for a way to do near-site backups — backups that are not on my physical property, but with a hard drive still accessible should I need to do a restore (let's face it — this is where cloud backup services are really weak — 1 TB at 3-4mb downloads just doesn't cut it). I've tried crashplan, but that requires that someone has a computer on all the time and they don't ship hard drives to Sweden. What I want is to be able to back up my Windows and Mac to both a local disk and to a disk that I own that is not on site. I don't want a computer running 24x7 to support this — just a router or NAS. I would even be happy with a local disk that is somehow mirrored to a remote location. I haven't found anything out there that makes this simple. Any ideas?" What, besides "walk over a disk once in a while," would you advise?
A colocation center? Do the initial backup locally then use something to replicate changes in the future?
I hate sigs.
If so, any remotely accessible computer (*nix box) with a wifi card will work.
Nobodies Prefect
Tidbits for Techs Technology Blog
A neighbour? Why not hook up an external panel antenna to the side of your place aimed at their place and have a NAS with wifi on it (may need external antenna for your NAS as well but maybe not). Then you dont even have any wires to worry about and its still on your network...encrypt the NAS in case of possible break and enters..
I mean that would fit the bill in terms of being a fairly easy automatic setup. Just rsync your machine to the remote backup at midnight every day, or you can even do it ever hour or ever 5 minutes if you want. Obviously any scheme can run into "you have too much data to deal with RIGHT NOW" but there's no cure for that. I guess the other option is sneakernet. You might swing something with a neighbor that involves using wireless. If the guy next door can pick up the signal from your router you could locate a NAS box in his place, etc. This of course presumes you really trust your neighbor...
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
I use Crashplan - it doesn't need to be on all the time, and your neighbours computer doesn't need to be on all the time (the one that has your USB disk plugged into it).
Crashplan just works!
parkour disk over once in a while.
Crashplan certainly does the "neighbour backup" quite well, and I think it is smart enough to wait around until both machines are online at the same time to do its magic, if you don't want to have the "destination machine" having to be running 24/7. You can use it to do the initial backup to an external drive and then walk that drive over to the neighbour's place for the subsequent incremental backups. One used to be able to buy a "Crashplan+" license which had a few more features like multiple backup sets for different destinations, but I don't see any way to get that type of license without signing up for a cloud backup subscription. Perhaps if you sign up for a few months and then cancel the cloud backup subscription part, your software might retain the "+" features.
Make sure you can trust your neighbor.
1) Convince your neighbour to be a part of this;
2) Dig a small trench stretching from your garden to your neighbour's;
3) Lay a properly protected CAT6 ethernet cable;
4) profit?
Should 1) fail, do it surreptitiously.
I would use carbon copy cloner for the mac. as long as the remote drive appears mounted on your computer every once and a while, it will do the backup. You can configure it to automatically fire when the drive is mounted (also after the designated time period), so the not-always-on thing isn't an issue.
Also not sure about the low-bandwidth restore. maybe you walk over for that one instance. Hopefully it's rare!
I used to be obsessive about having mirrored backups to an external drive. Over the years, I realized that my personal computer is not a business server and doesn't need to be treated like one. So my backup plan keeps only the most critical files in Drop Box, and the less critical stuff (things that can be re-downloaded, re-installed, or remade) gets packed up once a week in Windows backup.
So my suggestion is have two external hard drives for large format media and keep one in a safe deposit box at the bank, and rotate them out any time you need to back up something new (e.g. new install file, new batch of movies, or whatever.) For small files that get updated regularly, like documents or photos, then a cloud solution is still best.
Occasionally living proof of the Ballmer peak.
Assuming it is a next door neighbor it sounds like something a neighbor and i did during highschool:
-Merge the 2 LANS with a fibre switch at both ends.
http://www.netgear.com.au/business/products/switches/smart-switches/smart-switches/GS110TP.aspx
-Remove the internet account at one property and increase the bandwidth at the other property so there is only one gateway OR statically assign IP addresses for both properties so there are still 2 networks and two gateways
-Put a NAS on both properties with dual LAN cards and multi-home them so that each NAS is on Both networks at the same time.
Routers and NAS Devices are computers that you leave on all the time.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
If you can convince a friend to swap out his/her wifi router for a new one with the port you need, then all you need is to hook up a hard drive to the usb port on the back. Put a replacement OS on the router for additional features and you can use rsync over ssh. Since the wifi router will be an always on device, it would make a good backup target. Use dynamic dns or some homegrown ping system to find the router if it changes IP address.
Of course, your friend better stay that way if he has all your data.
Amazon has AWS Import where you can send them your hard drive and they upload it to Amazon S3. They also operate in Europe. It's pretty pricey, though.
1) If you're having your backups that close why not just put another drive in your computer and sync to that?
2) Buffalo NAS
3) Apple Airport Time Capsule
4) If you want a cheap hackable solution: Raspberry Pi NAS
5) Any other NAS
Probably the challenging bit is how you will sync to the storage. Here are some options for that:
1) Rsync
2) "Owncloud
3) Unison
You'll probably need to wrap a bit of scripting around it.
I have a pair of QNAP NAS boxes. One at the home office and one at the commercial office. Each location backs up the local machines to the respective NAS and each night the 2 NAS boxes sync to each other. I use encrypted backups for all sensitive data. This has been working well for two years so far.
Occasionally when I had an extended network outage at my office I physically transferred that NAS to the house to sync at gigabit speeds.
On a given day I may gave a delta of up to 100G but usually it is much less and we'll within the network uplink capacity (2.5Gbps) to finish before the morning rush starts
Bitorrent sync is a very simple way to go if you don't want to be too worried about backup administration. Just set up a read-only share for directories on the remote machine and put password protected encryption on the remote share.
That will give you at least some measure of protection from the remote server owner reading your files and they won't be able to nuke your local copies. Btsync is the most no-fuss, transparent backup solution I've used so far. I've got 4 personal machines that it's syncing right now and aside from a couple minor issues in earlier releases, it's been reliable, fast and has a minimal amount of administration you have to deal with.
Where's the challenge? What's the piece you can't figure out?
A DD-WRT compatible WiFi router with USB port goes for $30, and draws all of 2W of power.
https://www.amazon.com/dp/B009AO64E8
Connect a USB hard drive, enable mass storage, and SSH access. Use sdparm to set it to spin-down after 30 minutes of inactivity. Install rsync. Give it a free dyndns address (or some other service that screws free customers less).
Stick this contraption in a datacenter, under your desk in your office, in a friends/neighbor's house, etc. If you can't get them to open a port on their firewall, then you'll need to do "reverse SSH" tunneling, but it'll still work just a bit slower.
Hell, if you can find a location to put it that's under a KM from your home, you could even skip the internet requirement, and use WiFi for connectivity. You could even do without the power grid, setting up a modest solar panel to charge a 12V battery... My USB HDD enclosure runs on 12V directly, and a $5 car cell phone charger can provide the 5V@2A the listed router needs:
https://www.amazon.com/dp/B0079BLTPS
In any case, you'd just need to figure out the rsync command-line options to run on your home computers to copy the differences over the wire with the minimal overhead.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
What I do is make incremental backups to a set of 3 hard drives (which I just recently upgraded to USB 3.0 and 2TB each). I rotate them to/from my work location (but you could do this with a friend's or family member's house). I take one to work, and bring the other one that was at work back with me at the end of the day, and run the backup to it that night or the next day or two. I rotate about twice a week since usually a few days of lost data due to, say, my house burning down and destroying the backup drive, too, would be the least of my worries. So there is always at least one at home and at least one at work. If you are more paranoid, get 5 drives and do it more often. Or maybe use 2 sites away from home. If you work for the NSA ... uh ... nevermind.
I use a black one, a red one, and a blue one. I did not get the titanium one.
now we need to go OSS in diesel cars
That's what I did, now I have an off-site 3TB backup system.
I doubt Anonymous Coward has a job. He posts here several times an hour, every hours, 24 by 7. Unless his job is spamming out these ripoff web sites.
now we need to go OSS in diesel cars
"Walk over a disk once in a while." Seriously, just once a week trade a drive with your neighbour. I know this is /. where complexity wins, but jebus.
Wifi and BitTorrent Sync?
I personally love BitTorrent Sync...
You can tell how powerful someone is by the magnitude of the crime they can commit and be able to get away with.
Get a fire safe. Put your backups inside. Probably safer than having them in your neighbour's house as anything that could happen to yours could also engulf theirs. Get two drives so you can have one attached and the other in the safe. That's what we do at work as well as having drives go off site. Can't have too many backups. Of course, it does require some work but you can't really get away without some effort. Heck, if you have a basement you could locate your drives down there. Anything short of a meteor strike should be protected against assuming you're also protecting against water.
"I have the attention span of a strobe lit goldfish, please get to the point quickly!"
I have three options I'll present for you. One matches your headline, one is cheap, and one is really, really solid.
The option that most matches your headline would be to use a WIFI NAS at the next door neighbor's house. Use any of the many good backup software packages. More on what a "good" backup system is in a moment.
Something I used to do was have two external drives. On Mondays, I'd switch out the drive in the house for the one in the car, which would go to work with me. The drawback to that is it's not fully automatic, so sometimes I'd forget or be in a rush. That leads us to the attributes of a good backup system:
Backups must be fully automatic, otherwise you'll stop doing them regularly.
Backups should be rotated. A midnight backup is useless if you are hacked at 11:55 PM, or discover a problem 2 days later. You must have access to older backups.
Backups must be offsite. Fires and burglars will take your backup if it is on site.
Backups must be accessible. As you said, spending two weeks downloading your data isn't acceptable.
Backups must be tested. Our experience with web servers indicates that approximately 60% of backups provided by hosting providers don't actually work when you try to restore them
To meet all of the above requirements, we use an enterprise grade system. It may be overkill for your needs, but then again the $8 / month version may be just what you want. It provides several offsite backups from different points in time and they are BOOTABLE. You can pull down a file or two, run a program or service remotely, or restore a full system.
3-4 Mbps to transfer 1TB is no good, as you said, but you actually have 200 Mbps available if you use the system we use. If you need the entire 1 TB, not just a small part of it, the whole 1TB bootable drive will be delivered to your front door within 12 hours. You may know the old saying "never underestimate the bandwidth of a station wagon full if tapes.". With a 1TB drive, the bandwidth of FedEX is over 200 Mbps.
What we use is called Clonebox. It's designed more for business, but it may either work for you, or give you some ideas.
You can use the free version, which only backs up once every 24 hours (but you can trigger manual backups). It'll tell you via several methods if it's failed to back up for more than a day or two, but I don't believe it'll keep trying until the destination happens to be online - if it can't back up at its 24-hour window, it'll fail.
What you probably want is their Crashplan+ 10GB plan (~33/year in the US), even if you never use the 10GB of online storage. By getting the paid plan you also get the features of better encryption and more frequent backups. On the paid version they also also enable having multiple backup sets, so you can back up some files to the online storage (e.g. "Documents") while backing up "everything" to another computer. The "perpetual" licenses ended back in 2010 and mostly predated the online cloud backup options.
If you're not going to have direct network access between the computers (e.g. shared wifi), you'll also probably want to do the initial backup with both on the same network, then separate them. Crashplan's ID-based backup system should handle that change with no problems.
fencepost
just a little off
As a hosting provider that went broke found out, it's not really a backup if a delete on one leads to a delete on another. Forget mirrors - go for something like amanda or many of the others and put the output on USB disk or whatever.
As an added bonus, Crashplan encrypts the data before it leaves the/your client, and so many privacy concerns can be put the rest.
Most other solutions will not have scrambling of data (e.g., straight rsync), or you're rolling your own protocol/system and so it may not be as secure as you think it is.
I backup my iMac to a Time Capsule over WiFi. It happens to be located in my home, but it could just as well be next door, wouldn't make a difference. So if your neighbour is what we city dwellers think "neighbour" means and not "the next ranch ten miles down the river", that might work.
So basically, get a WiFi-enabled harddrive. Or a WiFi router with a USB port. Initial backup via USB or whatever, and incremental updates are usually small enough that they can happen in the background. On the Mac that's built-in, I'm sure there's software for Linux and maybe that hobby OS from Redmond a few people here use.
Assorted stuff I do sometimes: Lemuria.org
"rsync at midnight". At 8:00 AM, discover that your filesystem got hosed at 10:00 PM, so you now have two copies of garbage.
Do not just sync periodically. Approximately everyone I've seen try that method got screwed in the end. They'd discover that they got rooted two weeks before, they'd overwritten an important file two days before, etc. You must ROTATE and then sync to be doing anything more than pretending that you have a backup.
me.
The attributes of a good backup system:
Backups must be fully automatic, otherwise you'll stop doing them regularly.
Backups should be rotated. A midnight backup is useless if you are hacked at 11:55 PM, or discover a problem 2 days later. You must have access to older backups.
Backups must be offsite. Fires and burglars will take your backup if it is on site.
Backups must be accessible. As OP said, spending two weeks downloading your data isn't acceptable.
Backups must be tested. Our experience with web servers indicates that approximately 60% of backups provided by hosting providers don't actually work when you try to restore them
To meet all of the above requirements, we use an enterprise grade system called Clonebox. Other systems may be more applicable for home use.
Why bother? The National Security Agency already backs everything up for you.
Even though the OP made a cryptic comment about Crashplan not shipping drives to Sweden, the free version works perfectly for just this task. I created the initial backup volume on a portable hard drive to avoid sending the initial backup over a relatively slow ADSL line. After that, I sneakernetted it to a remote machine, copied the backup set onto it and it does the incremental backups once every 24 hours. If the remote machine isn't on, the software is smart enough to wait until it appears on the network and then start the backup. If I need to restore from the remote drive after my house is obliterated by an alien invasion (hopefully without us in it), all I need to do is copy the backup image onto a portable drive and physically attach it to my new machine.
Buy 2 VPN routers, set up a VPN to another location, Locate a NAS at the other location.
Most ISP's allow 2 ore more IP addresses, so you could have this setup parallel to any network that is currently running at the second location
unless you live in moms basement, mom would never notice this running at her house
Freedom of Information Act, they have everything on backup
Gently reply
But what are the odds of both being hit at the same time? For severe weather events, then obviously correlation will be pretty close to 1, but for burglary, correlation might actually be negative, if the burglar wants to move in, take some stuff, and get out of the area as quickly as possible before being caught, then they will probably pick only one house in the neighbourhood, and if they pick yours, that means they won't pick your neighbours.
Suppose it takes 1 week to recover from a burglary, and the probability of being burgled in any particular week is 1% - that is about 1 burglary every two years which is pretty high. If the correlation between you getting burgled and your neighbour getting burgled is 0, then the probability of both getting burgled in the same week is 0.01% or once every 200 years.
Isn't a NAS basically a computer you let run all the time? Sure it might be a bit lower power, but still...
How and what are you backing up? Continual structural disk images? Virtual RAID? Autonomous files with periodic synchronization?
If it's the first two, how close is your neighbor? Close enough for a good 300mpbs 802.11n signal? Maybe with a WDS/client bridge or two? Would you be able to bury some conduit piping and feed through a CAT6 Ethernet?
If it's the latter, a good software client with speed limit enforcement and pause/resume support should be adequate to run it over your Internet connection.
In any case, create a full backup locally first and walk it over to your neighbor, then continue your incrementals or synchronizations afterwards.
802.11ac offers gigabit wifi capability with a sufficient setup. We're talking some serious antennae arrangements, but even mutltiple Terabyte drives should be back-upable with 802.11ac and line-of-sight situations (like to a neighbor's house).
Them look for a usb 3 one and hang a hard drive from the router usb nothing to it, open up a rout in.
Probably a only suitable for a pure Linux solution, but I have this setup using a raspberry pie, a usb harddrive (that has it's own AC adaptor), and rsnapshot. I mount a scp share that uses a truecrypt file volume. This way I get point in time snap shots, using all free software, and the remote computer doesn't need to know the encryption keys.
You might look into retrospect (http://retrospect.com/). The have clients for macs and PC (and some flavors of Linux) and it's pretty easy to use. You can back up remotely (on schedule or on demand) and could restore locally of the hard drive. You & your neighbor can also back up locally onto a 2nd hard drive. The program has been around for 20+ years, it's reasonably price and the support is slightly above average. They have a free trial.
I wasn't thinking of burglary as a problem in need of the backup, but instead, your data getting into the hands of a stranger. You're basically doubling your risk of that happening, assuming your link is secure, and your neighbor is solidly honest and has a secure PC.
Instead, you could drop your encrypted disk in a safety deposit box and have physical security, encrypted data-at-rest and no network exposure during the backup process, no need to wonder how secure your neighbor's network/PC is.
Here in Atlanta, we had a 500-year flood at the same time California was experiencing massive wildfires.
Mudslides, earthquakes, floods, tsunamis, neighborhood-flattening gas leaks, the SWAT team trying to take out James DiMaggio/MOVE/David Koresh, or even the dipshit down the street burning leaves in a drought could take out both your houses.
The likelihood is small, but the consequences of failure are large. These are black swans. They don't usually happen...but they don't never happen either.
Y'know what's another huge risk, probably way more likely? Your backup failing. I remember being part of an IT project where an ancient SCA data drive went down on a database server. They had to get a replacement for the legacy hardware from the manufacturer's parts inventory, and spent 36 hours rebuilding the drive from backup tapes before figuring out that the database data itself had never been part of the server backup, because you'd have to disconnect the database to do that.
Fun.
Beg/borrow/steal an old laptop that isn't be used anymore. Those only draw about 10-15 Watts under normal use, so are about the same as a router or NAS if you leave it on all the time. Plug an external USB drive into it if its internal HDD isn't big enough for your purposes. You can use Windows, Linux, or even OS X; but the common network sharing system everyone uses seems to be SMB and CIFS which is Windows native. That's not to diss the Samba guys - I think they've done a great job reverse engineering a proprietary Microsoft protocol. But when I test it side by side, native Windows shares always seem to be a smidge faster. (Samba/Unix is much better if you're setting up multiple accounts and permissions though. Windows doesn't seem to like that unless you buy the server version).
Just be sure to do the initial 1TB full backup over the wired network. Subsequent backups should be incremental or differential backups, and shouldn't take long over wifi (unless you're generating or downloading a ton of new files every day). Every month or two you can drop by next door, plug into the wired network, and make a new full backup.
This is essentially what I set up for my dad, to backup his laptop to a NAS I put in his house. His laptop does the backup silently, in the background, over wifi. He is none the wiser, except the few times he's accidentally deleted a file I've been able to recover it from the previous day's backup. I just VPNed in and it looks like his differential backups are only about 10-50 MB each day. Based on that file size, I could probably just have his laptop do a second backup to my file server over the Internet/VPN for an off-site backup.
You've already mentioned that you tried the Crashplan approach. If having a computer turned on all the time is the only drawback that you see, then just buy a Raspberry Pi and use that to run Crashplan on. It will be cheaper than buying a NAS, and might even give you good portability because you can use a USB drive that can easily be used to recover data from in the future.
Since Crashplan allows you to generate your own keypair for encrypting backups, that should take care of the security aspect as well. Using Crashplan also lets you make multiple backup sets, define frequencies and retentions.
If you run out of storage, growing it will be as simple as adding a new hard disk to the logical volume. Of course, if you want to keep it simple, you could always just copy data to the new, larger disk and continue backing up too. You could even go fancy and do RAIDs on this, but then it might be overkill depending on your situation.
Overall, with this approach you use software that works well and have quite the minimal investment, i.e. RPi + USB hard disk. Using Crashplan for backups to destinations that are not on their servers is free.
What about keeping a drive/Time Capsule in a locking mailbox at the curb? Of course, power might be an issue, but maybe slip a 12V line up into the box from the yard lighting to power it, or even some solar cell set up although you don't want to attract too much attention that something electronic is happening in there. If nothing else, it could be a convenient place to swap or grab backups as needed that is relatively 'off site'.
That's a higher quality response than often found here on /. when someone refutes a post, thank you.
You've brought up some interesting topics. I bet you could ask some really good questions regarding the topics you mentioned. Instead, you chose to make ludicrous assumptions, assuming answers to the questions you could have asked.
Do you have any reason to make any of those assumptions? What makes you think some companies, such as a certain insurance company, don't run Clonebox on-site, mirroring between their facilities? Are you familiar with the datacenter choices available with Clonebox, including the underground nuclear bunker?
Do you think it's even POSSIBLE to offer that service at that price WITHOUT deduplication?
Do you somehow think you know anything at all about the media used?
You can use Storagecraft's ShadowProtect software for backups, it supports the off-site function, as do many full fledged backup sets. From there, you can setup a NAS on the other side, your office for example.
I'll store it in climate controlled, above ground storage. I'll return it to you by fedex overnight or 2 day shipping if you need it back to restore a borked computer. I'll charge $100 per year which includes up to 2 shipping cycles sending the drive back to you.
There are several major reasons for backup:
1) computer/drive/house is destroyed
2) files were accidently deleted
3) historic archive
For #1 you have to find a way to restore onto new hardware which might have problems if you don't have the crypto keys off site too.
For #2 a local drive is the best and most automatic but it has to be able to store several versions of files (why didn't Linux get the VMS file preservation option?)
For #3 the best option is clone the hard drive every few months/years. Or better yet, remove old drive and install a new one and restore your data so you can check your data is still intact.
I tend to use backup servers which use rsync and linked files to allow several versions and deal with some data deduplication. About once a week I copy that data set to another disk and then get that off site. The result is I can restore screwups in about 5 minutes and if the place burns down, I am out out a week of data. Sicne the disks keep getting bigger, I also have backups from years ago should the tax man want to know about something specific 5 years ago if the paper records were to be unreadable. My typical backup server build is a freebsd box booting off a flash stick card and then lots of disk. The result is I only need a copy of the flash stick and and a disk and any generic PC to restore the data.
This is why I would look into solutions being referred to as zero-knowledge solutions.
Most people have friends or relatives. It's pretty cheap to create a zero-knowledge implementation where the only thing required is for the other party to have somewhat reliable power and an Internet connection.
Two or more parties would be involved in the initial exchange where you distribute the costs equally and receive an equal sized TrueCrypt container that is, as you would wish, encrypted data-at-rest.
Only when your machine is on does it initiate routine rsync jobs that incrementally sync your TrueCrypt container using a Shadow Copy service (If that's not possible, automatically unmounting and remounting the container does work). I find this works fairly reliably, but is rather process intensive when the containers are large. It's not unusual to take 30 minutes just to figure out what will be sent in an incremental update.
The only real advantage in this setup is encrypted data-at-rest capabilities and restoring is as simple as downloading the whole container, or driving over and copying the container locally. Anything farther than local and you are right back to where you started, which is relying on services limited by Internet bandwidth.
I've been interested in doing this for some time now. A couple of 3TB hard drives in a cheap Linux based enclosure that is plug-n-play 24/7 would allow you to create 1-10GB containers (more efficient for syncing and organization of data) that are present locally and in two remote locations. You could not have any files larger than 10GB, but do you really need to have a backup of a Bluray disc?
I figure it would allow you to restore locally very fast by copying the TrueCrypt container back to your main system and just mounting them. An added benefit is that with a half dozen people participating you could easily have 1 local/2 remote backups with a decent number of containers that are supported.
He posts here several times an hour, every hours, 24 by 7. Unless his job is spamming out these ripoff web sites.
Dude... that's spooky. I just went and signed up to receive my first payment to my PayPal account for $78.93!
All I have to do is copy, paste, and post the same thing.
Of course I had to pay an initial $49.99 on a Russian website with questionable security, but I'm fully expecting those payments any time soon.
Btw, go to http://jobs95.com/ please. Please?
I forgot to mess up the domain name before I posted, so for the sarcastically challenged here, don't click the link
One of the nicest things about btrfs (or any copy-on-write filesystem, really) is the ease of snapshotting. Just add a step in your cron job where after the rsync is done, a snapshot is taken. Then if you never send corrupted data over rsync, great! You can just ignore those snapshots and it isn't any more hassle to manage. But if you do have issues, you can walk back through the snapshots and pull out uncorrupted data from before whatever wiped out your source did so.
You could adjust this for rotation, but rotation is for chumps who don't buy enough storage ;) But no, more seriously, leveraging the filesystem itself for such versioning/rotation (the latter only if you really don't have the space) makes a lot more sense in this day and age.
I remember sigs. Oh, a simpler time!
OP: I don't want to use Crashplan because (incorrect assumption)
Commenters: use Crashplan fool.
There's a lot of talk about wifi. That's just dumb. It's your neighbour. Run one nice ethernet cable. Put it into a proper conduit. Bury it in a small trench 18" deep. We're talking about 50' of ethernet cable.
As for your neighbour's house burning down, it's crazy unlikely for two houses to burn at the same time. Assuming you have a fire department, they focus on keeping the fire from spreading. It's often way too late for them to save the first house, but very easy for them to save the second. When was the last time you saw two houses burn together, in any reasonably-sized city?
So yeah, full-speed ethernet, to a NAS -- neighbour accessible storage. Or hell, it can just be an external hard drive sitting in a window, and you can run eSATA or USB in your conduit.
If you live in a neighbourhood like mine, you could probably run it across the entire street of backyards, across twenty neighbours -- we built the fences together -- and solve the problem twenty times in a row.
Get two synology nas boxes. They support nas to nas backups. http://www.synology.com/support/tutorials_show.php?q_id=461
My Hello World is 512 bytes. But it's also a valid Fat12 boot sector, Fat12 file reader, and Pmode routine.
You have everything you need in your description...just dont use his computer!
Buy a simple single disk NAS box. Then simply ask your neighbor, best friend, etc. (or relative across town) to plug it into their router (and tweak the port forwarding) so you can access it. Then use Crashplan (and something like DynDns.org) to make sure you can always resolve to that share.
I'd also suggest buying your data host a "thank you" gift of a brand new router that supports VPN (and remote admin so you can tweak) so you can encrypt your data tunnel and access it reliably.
Yes, its more complex than that, but you are already on the right track. Just add your own autonomous hardware at the remote site (and maybe a new router for their trouble) and you are all set.
Even better if you can reciprocate for your friend and host their backups.
I will add one cautionary warning. If you want TRUE disaster recovery, DONT use your neighbor. Flood? Tornado? Wildfire? Choose your remote host wisely. Remember if you need to recover your data, driving up to a couple hours to retrieve your backup wont be a big deal in the heat of the moment. Simply having it safe will give you warm fuzzies in your car on the trip to get the recovery media. But to be standing there saying "YES! I backed up off site!" only to realize your offsite backup across the street was also destroyed by the same disaster doesnt really help matters. (and will only put you deeper into despair) Just google "2008 Columbus, IN Flood" or "Richmond hill explosion pictures" for examples.
Ok, the way I do it is via sneakernet (or truck-net if you will). I have one 'a' those hard drive plug-in devices, and every once in awhile, I plug in a terabyte drive, do a backup of my stuff, and drive it over to a friend five miles away, who has a fire safe. Swap the drive with the one already in there, and bring it back.
This isn't the same as an automatic process that backs up everything, but it protects the stuff I care about. As I write this, I wonder if Ghost can't be rigged to do automatic backups to the dropbox folder.
In any case, I can absolutely sympathize with what you're trying to do. I do photography professionally, and have just under a terabyte of my own work online. A house fire would completely wipe me out. So now I have small scale geo-backup, and it'd take quite the natural disaster to get both copies.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
Snapshots can certainly be part of a backup system, and the btrfs variety can be a convenient local "undo" step, similar to the undo in your editor. The way they function is not appropriate for offsite backups, though, not without a solid validation regime anyway.
The problem with btrfs snapshots is that if data is corrupted at any point, that data silently remains corrupted in future snapshots because they are copy-on-write. Suppose, for example, that your backup of your httpd.conf file gets corrupted at any point. If you take a snapshot, the "new" version is actually the old version - still corrupted. You then rsync from the source. Rsync looks at the mtime and because the file isn't marked as changed on the source, it doesn't get synced. All backups are still corrupted. For a litle while, you might have a copy from before the corruption, but eventually that ages out and you're left with nothing but corrupted data. I'm not theorizing on this point. I learned this lesson the hard way.
If you don't have an intermediary server, you'll only be able to backup when the receptacle is running. This isn't unique to crashplan, but rather is the nature of direct computer to computer communications. Honestly, Crashplan is the easiest I've worked with.
Alternatively, you can setup an FTP server on their network reasonably easily, and backup to it regularly. This could be a special piece of hardware, like a $200 Synology Diskstation, or a Raspberry Pi with an attached USB drive. Or it could just be a regular computer, though crashplan might be a better option in that case.
You could also do something cheeky if your friend is within wifi range, like giving them a USB drive attached to a router that connects to YOUR Wifi network. Then back up to it as if it were local to your house. But, of course, you're not as protected in that circumstance.
The ______ Agenda
That is what encryption is for. If the burgler steals the backup device, they get some sweet hardware - but without the keys, no useable data.
Git Annex ( http://git-annex.branchable.com/ ) is awesome at doing it's syncing when it's possible and has a somewhat nice web UI over great CLI tools. It can also do encrypted backups which may be important if you don't fully trust your neighbour.
This is what I am looking for. The problem is how do you access the remote disk via crashplan? Every type of NAS disk that I have seen only supports access files via the internet side via the web - no options for remote SMB links. What I really want is a remote disk that I can access via a windows share. This would solve my problems. I just can't find one.
We are very lucky here in Sweden - I have 100/100, as do my parents in law (via a local exchange point), where I hope to place the disk.
I have Crashplan setup with my computers and a closer neighbor (who has 1000/1000 via the same broadband provider as I do) and a 3TB drive in his 24x7 server, but it has not been reliable at all. I can hit his port and crashplan responds (when I check it via telnet), but the services don't talk to each other. It can go weeks at a time with no successful backups (I also have Crashplan+, and the cloud backups are working just fine, so I don't worry too much, but I am concerned about the potential restore speeds). Maybe I just need to reinstall crashplan on his machine...
The problem with hosting locally is the way most ISPs are setup...
You typically have pitiful upstream bandwidth, making uploading painful, and if your neighbors use a different isp they could actually be a large number of hops away, via whatever peering centers the isps both use. Even if they use the same isp, your connection could terminate far away causing a several hundred mile round trip before it reaches the guy next door.
Ideally you'd have local peering so you can get gigabit speeds or faster to those in your local area, and only depend on the isp for things further away.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Torrent sync will sync when connected and you can set limits on bandwidth usage.
I use it to back ground sync tens of GB over multiple sites
Slashdot Beta should die a painful death.
Just been researching this topic over the last week. So far settled on Areca Backup ( http://www.areca-backup.org/ ), open source rsync algo based tool + offsite WD Mybook Live NAS. Other Rsync solutions looked too much of a hassle to use on Windows. Also was hoping the WD MyBook Live NAS to have something out of the box but all the tools added were useless for backups. At least the NAS has ssh and FTP, this allows most of the backuping solutions to work on it. Will take a look at BTSync now, thanks.
I use an additional router with USB drives in my garden shed, 15 yards away from my house.
As additional measure I send 1-3Terabyte disks to myself 'poste restante' (general delivery) that my postoffice will send back to me after 3 months if I don't claim them.
It's the cheapest way I found and I can drop the disks in any mailbox on the way to work.
Use Wifi, or bury a Cat7 cable. Kinda surprised you didn't think of that yourself.
For those who have a neighbour across the street or a few houses down, we use a Nanostation as a wifi link. Easily get 40-60 Mbps. We used to use WDS on DD-WRT but it would consistently drop packets and lose connection. Our connection is so solid now that I actually canceled my internet service and we share a better package with faster speeds.
nft.
...run a little fiber along the ditch (or across the lawn - you can bury it a couple of inches to avoid the lawn mower). Get a couple of gigabit switches and fiber converters, and you can wail on a NAS from either house.
I live on my family's 'compound', and have priced running the few km of fiber needed - still a little expensive for 4 houses on 50 acres, but very do-able across a lawn.
Tiller's Rule: Never use a word in written form that you've only heard and never read. You will end up looking foolish.
Crashplan is exactly what the OP is looking for.
Computers don't have to be on constantly (they'll back up when they both happen to be on).
With the first link, the chain is forged.
Check the Buffalo units. they are awesome. http://www.buffalotech.com/products/network-storage. Many of their products allow for access from about every system out there, SMB, FTP, etc.
You could also be running into port blocking issues at the ISP. That is where my suggestion of buying your friend a new router with VPN support. At that point you setup a VPN from your workstation to the NAS via the new router and you can have full access to the disk across the wire.
Its tricky, but in theory it should work.
Bother to read the docs, and then use Crashplan.
You can use CrashPlan to backup to their cloud (for $), your own computer (free) or a friend's computer (free).
- Your computer does *not* have to be on 24/7
- The backup machine does *not* have to be on 24/7
Too bad that your lack of reading comprehension is going to give you a flood of rediculously-complex solutions.
Indeed btrfs checksums with a custom scrub job could detect some types of corruption, corruption that occurs after the backup.
By default, the btrfs scrub will try to recover the bad block when it's read. For this use case, you need it to delete or at least touch() the file.
That would be the "not without a solid validation regime anyway" part of my post.
Write to HD locally, mail it to yourself.
If you always have one in transit, you're safe.
The Kruger Dunning explains most post on
What you're after is possible. CrashPlan can be configured to run headless on a NAS. However, it is unsupported by Code42. People have done it, though; the CrashPlan user community has some really clever folks, and we engineered it to accommodate customization.
Communications Manager Code42
You make some good points. I think this statement deserves some clarification because after you said it, you mentioned why it's not wholly true:
> This is not a problem with using btrfs snapshots to perform incremental backups -- it's a problem with incremental backups.
It's a problem with SOME approaches to incremental backups. As you also said:
> then do an rsync into that incremental backup with --ignore-times
That's one way to do incremental without the "corrupt forever" problem, it works as one type of validation.
So incremental can be done without that problem.
> fakery ... to make each incremental backup *look* like a full backup.
Indeed, btrfs snapshots and some similar approaches *look* more comprehensive than they are. It's a trap for the unwary.
Contrary to the claim, crashplan doesnt require 24/7 power on. The initial seed does, after that it does incrementals daily (for the free version) which generally will happen whenever they need to. If theyre scheduled for 3am and your computer misses the schedule, it will simply postpone till the next time things are on.
If you pony up for one of their paid versions (CP+, Pro, or ProE), you get backups every 15 minutes of tiny deltas.
There may be other solutions, but you may want to take a deeper look at crashplan.
Now that you can easily fit 3-6TB in an external enclosure, you can do some pretty flexible things with backups.
Here's my system.
Local 3TB drive in system, mirrored to 2nd internal 3TB drive
Nightly, I rsync that data to a 3TB mirrored NAS
Weekly I rsync that data to a 2nd 3TB mirrored NAS
Monthly, I rsync to an external 3TB enclosure via USB
When I go to the bank to deposit checks every month or two, I swap the 3TB external USB enclosure with an identical one in my safe deposit box.
Only costs me $50 a year for the safe deposit box, and I don't have to worry about my neighbors breaking anything.
Also, I have a 2nd manual version of my backup scripts featuring --delete for when storage starts to fill.
-- Dave
up 12 days, 22:30, 2 users, load averages: 993.20, 994.21, 994.56
*makes note to limit user processes...
> Backups are rotated on two drives at least, and they shouldn't be both online at any time, in case of power surge or lightning, or even hacking of the backup server.
> The only way I see to achieve this, with one backup site, is to have backup drive B physically disconnected while backup drive A is plugged.
> How could it be fully automated ?
If your backup facility isn't protected from lightning and you can only have one facility, that does make it harder.
Given those requirements, I suppose one could use a relay on the power line. For the data line, I guess you'd need something
that includes chips like http://www.mouser.com/Semiconductors/Switch-ICs/Analog-Switch-ICs/_/N-7590c/ and I good alternate path to ground.
Oddly enough I just caught an episode of The Linux Action Show last night that included a product that seems to have exactly what you'd want -- although it sure ain't cheap. Look for: DiskStation vs FreeNAS | LAS s29e03.
I'm at work so I can't get the details for you right now, but they did a brief review of a 4-bay NAS running Linux, which has some "app-store" style functionality...and it's a full linux system. There's an app that'll mirror the entire NAS to Amazon Glacier, although then you've got your network speed bottleneck (though having a one-button restore may help there) but there's no reason you couldn't set one of these up with your neighbor or wherever and access it via whatever network. It's very high speed, very low power, but it runs an Atom processor and a Linux distro so you could probably just toss your favorite PC backup solution on there. Could probably also grab that Glacier or a similar app, set it up to constantly *restore* nightly, and push the backups out from your machine...so you're only pushing incremental changes which shouldn't be bad, and the drive syncs those up to a local copy every night.
If you've got rsync and bash you can run shootsnap. It's dead simple.
http://typinganimal.net/code/
Oh, wait, you said Windows. You'll have to spend money, then ;)
The only real advantage in this setup is encrypted data-at-rest capabilities and restoring is as simple as downloading the whole container, or driving over and copying the container locally. Anything farther than local and you are right back to where you started, which is relying on services limited by Internet bandwidth.
Recently had my dad send me his laptop for some repairs. $40 to ship an entire, way over-packed laptop about 1000 miles, and it arrived in two days via UPS standard ground. If your backup strategy is to give the drive to a friend, there's no reason you couldn't ask them to drop it in the mail when you need to recover.
If you're recovering so often that the cost or hassle of mailing it is too much, then you have some bigger problems to look at. Keep a local backup for everything except 'my house no longer exists' type events...and if your house is ceasing to exist every month or every year, *what the hell*??? Seriously though, if you need to use your offsite backups, you're almost certainly gonna have thousands or more in repair bills so even if you've gotta have them overnight some massive 4-bay NAS, it's still gonna be a very minor cost in comparison.
I've been a happy user of buddybackup (.com) for years. It backs up to multiple friends computers and vice-versa (and a local copy). Pretty much runs in the background without any intrusiveness or adware. No mac version though.
--checksum sounds like a winner. I may need to review man rsync, then see if we should be using that anywhere.
>OK, I haven't actually used btrfs (or any other fs with similar snapshot/CoW capabilities
Neither have I, but I don't think we're supposed to admit that on /. I think we're supposed to act as though we're experts on things we've never seen before, since this is Slashdot.
I have read the code for copy-on-write snapshots used by lvm and my understanding is that it's essentially similar.
This software is really cool, I use it with great success everyday and it is encrypted. It works over a local network or the internet so it can make use of a gigabit network. If you were talking about your next door neighbor then there is no reason you couldn't just set up a wired connection or wireless bridge between your houses and let it go to town. This can also run off a raspberry pi so it will use minimal power. The raspi wont run at gigabit speeds but it is low power. Give it a shot its free!
I know you said you've tried crash plan but why not use it? Simply seed the initial backups, move the drive to your neighbor's, add the seed to their Crashplan and the schedule it to run x days berweenny times? I do it and it's flawless. Yes it requires a computer but other than that it's great. Another thing I use is Bitcasa $99 a year but unlimited and offers mirroring.
NAS is really a "computer" on all the time sorta however. If neither that or physically walking a external USB over are not acceptable, your best bet is to pray to the voodoo gods that you never have a failure.
There are cloud services you can pay for, which I would never use if you don't want to share your data with the NSA. Apart from that, there are expensive tape drives, that are not all that different from external usb HD backup really.
If you set it up right, you could simply install some extra internal HD and just do a backup there. You could even do a RAID mirror or something if you like. Won't help if your house burns down however. Then only offsite will do, which is basically sneaker-net, or cloud.
However if you backed up to your neighbor, and your neighbor to you, unless both houses burn down you are safe!