Slashdot Mirror
NetBSD - Live Network Backup
dvl writes "It is possible but inconvenient to manually clone a hard disk
drive remotely, using dd and netcat. der Mouse, a Montreal-based NetBSD
developer, has developed tools
that allow for automated, remote partition-level cloning to occur
automatically on an opportunistic basis. A high-level description of the system has been posted at KernelTrap. This facility can be used to
maintain complete duplicates of remote client laptop drives to a server
system. This network mirroring facility will be presented at BSDCAN 2005 in Ottawa, ON on May 13-15."
I'm not up on my xBSD's, so can someone explain how hard this would be to port to the Mac? This would be perfect for cloning my son's Mac Mini.
...when you get that idiot (and EVERY company has at least 1 of these guys) who calls you up asking if it's OK to defrag their hard-drive after downloading a virus or installing spyware. Then, when you tell them "NO", they just tell you that they did it anyways.
Now we can just hit a button and restore everything, a few thousand miles away.
The only thing left is to write code to block stupid people from reproducing.
IGB: More fun than eating oatmeal!
I, too, immediately thought of German when I saw "der Mouse" (although in German it would be "die Maus", since Maus is feminine). Since they're located in Montreal, however, it seems unlikely that they'd be inclined to use German, and would be more likely to go for a French reference. So I ask, where does the "der" come from?
Ben Hocking
Need a professional organizer?
While this is cool, as I thought when I saw it on KernelTrap, disk mirroring is useful in situations where the hardware is less reliable than the transaction. If you have e.g., an application-level way to back out of a write (an "undo" feature), then disk mirroring is your huckleberry.
/.Recycler, whatever). Obviously the underlying routine would have to do its own garhage collection, deleting trash files by some FIFO or largest-older-first algorithm.
Most (all) of my quick restore needs result from users deleting or overwriting files - the hardware is more reliable than the transaction. I do have on-disk backups of the most important stuff, but sometimes they surprise me.
I'd like a system library that would modify the rename(2), truncate(2), unlink(2), and write(2) calls to move the deleted stuff to some private directory (/.Trash,
Just a thought.
sigs, as if you care.
Novell Zenworks has had this capability for sometime in production environments. It also integrates with their management tools so it is easy to use on an entire network. To say this technology is newly discovered is a far cry from the truth. They also use Linux on the back end of the client to move the data to the server.
It is nice though to have something like this in the open source world though. Competition is good.
not the same?
http://www.feyrer.de/g4u/
Use rsync and hardlinked snapshots. There are lots of examples out there. I rolled my own a while back, but if you want something relatively nicely polished and based on that idea, check out dirvish (I didn't find that until after I already had my system set up).
I really like having several months worth of nightly snapshots, all conveniently accessible just like any other filesystem, and just taking up slightly more than the space of the changed files.
From the article, it sounds like they are using a custom kernel module to intercept all output to the drive. This would keep things from getting corrupted, yes?
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Image backups certainly have their place for people who can understand their limitations. However, a good, automatic, versioning file backup is almost certainly a higher priority for most computer users.
Great. Now, could you please enlighten us as to what a good, automatic, versioning file-based backup system might consist of?
AFAICT, this doesn't seem to exist. It doesn't matter how much sense it makes, or how perfect the idea is. It is simply unavailable.
In fact, the glaring lack of such a capable system almost seems to indicate that it is a victim of the "pick any two" rule.
So where is it?
(And, no. A few programs tied together with a ream of Perl or shell script that needs modified in order to function does not constitute a working system, and nor does a HOWTO with instructions on coding one.
Non-programmers, believe it or not, often have important data to back up, too, and being able to code should not be a prerequisite for keeping important stuff backed up. That is, unless you programmers really do think that it'd be no big deal if your loan officer lost your mortgage just hours before closing, or when the accountant's machine trashes your financials.)
Kid-proof tablet..
rsync is not scalable to large numbers of files. We set up a backuppc machine awhile ago, tried to rsync the entire backup set over to another machine... It was a miserable failure. Even if we didn't check for hardlinks, (which we have to, backuppc uses tons of hardlinks,) the rsync process completely saturated a gig of RAM before it even started syncing.
Now, rsync would have been fine if we'd unmounted the filesystem and done it on the raw partition. But there's a couple of problems with that:
It's not live. Not a big deal for us, since it's a backup machine to begin with, but still...
rsync doesn't do that. A couple of people have submitted patches to allow a flag for rsync to copy block devices as if they were files. They were tiny patches, but they were rejected out of a fear of users doing stupid things with them. I guess the usual Rsync Way is to duplicate the filesystem, so that devices are copied with mknod, not dd.
Don't thank God, thank a doctor!
I hear ya. We've been cloning our labs with dump/restore over the net for years. Works on everything: Solaris, *BSD, Linux. Wrapper scripts make it a one line command.
I know some Linux distros don't come with dump/restore. Maybe that's why more people don't use it.
Serve Gonk.
I can think of a few reasons. It makes time-consuming partioning/formatting unnecesary. It does not require as much work to restore the bootable partion (ie. no need to bootstrap to run "lilo", "installboot" or whatnot). But mainly, because there are just no good backup tools...
Full dumps work fine, despite the above limitations, and I've piped dumps over the network many times. However, I've had incrimentals fail to restore a few times, so I can't trust them to work, and full-dumps take much too long to do regularly. So ufsdump is a lowsy option, in my experience.
Actually, you can pretty easily solve this, though it takes quite a chunk of time.
On any unix system, just do "dd if=/dev/zero of=zerofile". After it fills your 38GBs of unused disk space, delete the zerofile. Then, your 2GBs of data, and 38GBs of zeros will compress down to a little more than 2GBs. Writing the zerofile to disk takes forever though, but it's well worth it, especially if you will be sending the image back out to multiple machines.
I used-to do this to clone Windows machines. I wrote a simple C program to do the zerofile thing, then I'd multicast the compressed drive image to about 100 similar machines simultaneously. It was incredibly fast, and made Ghost look like a joke...
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
You missed the point. Here you only need to copy the image once and then all subsequent writes are done on both images at once (the on-disk and the network one). That means that everything after the initial copy (assuming you begin doing this on an existing fs) is as efficient and real-time as possible, requiring no polling for changes or any scheduling. It is essentially RAID1 over a network. Although it doesn't do much against system crashes (since neither side will have the final syncs and umount writes) it does work very well against hard disk crashes, and it is also good to know that the same data is on another machine - so you can just boot into that system and get your server up, without needing to migrate disks over or reconfigure some things. Well, I don't know how close usual RAID1 is to that.
Sam ty sig.