Slashdot Mirror
NetBSD's Real-Time Network Backup
jschauma writes "One of NetBSD's developers, der Mouse, was interviewed by DaemonNews about his real-time network backup system (originally presented at BSDCan 2005), where changes to your local filesystem are automatically propagated to a backup server. In his interview der Mouse tells about his idea, how it works, and of course, how cool it is."
This concept has been in play for years as a commercial product for Disaster Recovery, Veritas Volume Replicator (VVR).
"Those crazy Germans."
No, that would be "der Maus"
You crazy Americans -- Hier ist der Maus!
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
Those crazy Germans.
According to the article, he's canadian.
I enjoy large posteriors and I cannot prevaricate.
How does this compare with Linux Network Block Device? Sounds very similar.
There are pretty mature commercial tools for this stuff, as well - Veritas' VVR replication comes to mind.
How is this any different from DRBD (http://www.drbd.org./
/dev/nbX). Every write is sent to the local 'lower level block device' and to the node with the device in 'secondary' state. The secondary device simply writes the data to its lower level block device. Reads are always carried out locally.
From the website:
DRBD is a block device which is designed to build high availability clusters. This is done by mirroring a whole block device via (a dedicated) network. You could see it as a network raid-1.
Each device (DRBD provides more than one of these devices) has a state, which can be 'primary' or 'secondary'. On the node with the primary device the application is supposed to run and to access the device (/dev/drbdX; used to be
If the primary node fails, heartbeat is switching the secondary device into primary state and starts the application there. (If you are using it with a non-journaling FS this involves running fsck)
If the failed node comes up again, it is a new secondary node and has to synchronise its content to the primary. This, of course, will happen whithout interruption of service in the background.
And, of course, we only will resynchronize those parts of the device that actually have been changed. DRBD has always done intelligent resynchronization when possible. Starting with the DBRD-0.7 series, you can define an "active set" of a certain size. This makes it possible to have a total resync time of 1--3 min, regardless of device size (currently up to 4TB), even after a hard crash of an active node.
As of Windows 2003 R2, there is a capability to do a VSS type of thing over the network to a remote server.
I'm a little ashamed that I know that, but it's true.
So what you have is not only a real-time backup, but also the ability to unwind any possibly damage after a break-in or other event. If your backup server is only running a journalling client, it can be made extremely secure, and also provide an excellent auditing tool.
It would also be possible to delay the streams and have arbitrarily old filesystems available. Or to use a local journal as a buffer to smooth out the IO load as it is piped off site. It could also be used to augment a non-journalling filesystem, for crash recovery purposes, assuming your filesystem provides at least some consistency guarantees. In fact Netapp does something similar by logging FS operations to NVRAM, while the filesystem only writes consistentcy points periodically.
Although I haven't read about the NetBSD work, I am sceptical that they could get the error handling to work correctly at that level. With the DragonFly journalling, there is support for transactional consistency, as well as recovering from interrupted network connections. While it is not complete, much of it is in place and functional.
See the mountctl manual page for attaching a journal to a mount, and the jscan manual page for processing the journal.
In linux a RAID array can contain any block devices, including network block devices, ramdisks, whatever.
(I read this in a linux software RAID tutorial once)
NetApp has been doing somthing similar for a very long time. A lot of people use the Sun boxes on the frontend to boot or attach to the storage appliance and let it do the backups. It saves space and saves the server from having to do it.
Sadly, no, not even to DragonflyBSD. Don't know why, but maybe it's because it uses kernel threads internally...
-- Sig down
Actually, he's been around as Der Mouse since I was in college (circa 1985). I ran the xterm replacement he wrote back then. Long, long before Theo had his hissy fit and forked off OBSD. Of course, a trivial Google would have shown that, but hey, an AC would want to miss out on an ad homen flame...