Slashdot Mirror


NetBSD's Real-Time Network Backup

jschauma writes "One of NetBSD's developers, der Mouse, was interviewed by DaemonNews about his real-time network backup system (originally presented at BSDCan 2005), where changes to your local filesystem are automatically propagated to a backup server. In his interview der Mouse tells about his idea, how it works, and of course, how cool it is."

18 of 166 comments (clear)

  1. Correct me if I'm wrong by thedletterman · · Score: 5, Interesting

    But hasn't Sun been doing this with Solaris for at least 3 years?

    --
    Any fool can criticise, condemn, and complain, and most fools do. - Benjamin Franklin
    1. Re:Correct me if I'm wrong by operagost · · Score: 4, Interesting

      OpenVMS has been doing this for even longer using volume shadowing.

      --

      Gamingmuseum.com: Give your 3D accelerator a rest.
    2. Re:Correct me if I'm wrong by vertinox · · Score: 3, Funny

      But hasn't Sun been doing this with Solaris for at least 3 years?

      Yes, but do you want to sell your children and a kidney for a Solaris server?

      --
      "I am the king of the Romans, and am superior to rules of grammar!"
      -Sigismund, Holy Roman Emperor (1368-1437)
  2. Neat. by Pig+Hogger · · Score: 3, Interesting
    This is definitely the way to go. With huge hard-disks that offer capacities beyond tape drives, it is less and less feasible to use traditionnal tape-based backup systems in many organizations, if only by the time taken by the frigging tape drive...

    Here is the idea behind the setup I am currently using: Easy Automated Snapshot-Style Backups with Linux and Rsync.

    1. Re:Neat. by Lord+of+Ironhand · · Score: 4, Interesting

      I prefer Dirvish, and I highly recommend that people looking for a good harddisk-based backup system take a look at it. I've looked long and hard for a good backup system and this is the first that seems to fit the bill for me.

  3. Re:B.S. D? by ThePiMan2003 · · Score: 5, Insightful

    I think the point is that it could be used for an off site backup. Raid does not protect you from Hurricanes, or even fires.

  4. Good idea, but there has to be a better way by TheFlyingGoat · · Score: 3, Interesting

    This idea is really cool, but implementing it by putting hooks into each device driver seems overly complicated. It also doesn't sound like they're any sort of priority setting for this or any type of data filtering.

    Personally I'd like to see something like the MS filesystem in development that allows SQL calls to be run against it (not sure if there's any other filesystems that are similar). Query every 5 minutes for changed data that fits the backup parameters (within the system dir, the user's home dir, certain filetypes) and then transfer the data as the network isn't being used.

    That would achieve the same thing, but more flexibly and without affecting normal use.

    --
    You have enemies? Good. That means you've stood up for something, sometime in your life. --Winston Churchill
    1. Re:Good idea, but there has to be a better way by jcgf · · Score: 3, Interesting
      A hook into each driver does seem like a strange way to do this, you would think that it could be done once at a higher level.

      Query every 5 minutes for changed data that fits the backup parameters (within the system dir, the user's home dir, certain filetypes) and then transfer the data as the network isn't being used.

      Then you loose the realtimeness.

    2. Re:Good idea, but there has to be a better way by ivoras · · Score: 4, Interesting
      This idea is really cool, but implementing it by putting hooks into each device driver seems overly complicated.
      FreeBSD's GEOM is solving that: http://www.bsdcan.org/2004/papers/geom.pdf

      Also, there's "GEOM gate" on FreeBSD: http://garage.freebsd.pl/GEOM_Gate.pdf
      For other cool stuff with GEOM see here and here. See also this discussion thread about ggate's limits.

      --
      -- Sig down
  5. Re:B.S. D? by Amouth · · Score: 4, Insightful

    yes you are missing the point..

    take 10 small servers that do the front end grunt work with 2-3 backup servers that keep complete working images of the servers and have access to their data..

    a front end server dies service can roll over to a backend until the front is replaced and is quickly made jsut like the orginal a backend dies and you have a second and if all the backups die then you still have the front end to recreate the backups..

    you don't normaly consider the bandwith costs as they are typicaly on a highspeed network between them and it offers you the option of replication over diffrent connections and areas..

    all redundent disks help with is if a disk dies not if ram or cpu fails

    some people have gotten too attached to their physical backups and tapes - personaly a backup is worthless if i can't have live access to it in a few min even if i am not physicaly at the point of failure..

    this isn't particulary useful for small setups but is great for mid to large scale setups and offers plenty of room to grow.

    --
    '...if only "Jumping to a Conclusion" was an event in the Olympics.'
  6. NBD? by mikeee · · Score: 3, Informative

    How does this compare with Linux Network Block Device? Sounds very similar.

    There are pretty mature commercial tools for this stuff, as well - Veritas' VVR replication comes to mind.

    1. Re:NBD? by tpgp · · Score: 3, Insightful

      How does this compare with Linux Network Block Device? Sounds very similar

      It doesn't compare at all.

      From my (quick) scan of the article - think of NBD as a replacement for NFS (well, sorta) & this as a sort of network RAID (kinda, not realtime).

      They're not really alike - for linux drbd is probably closer.

      --
      My pics.
  7. Re:B.S. D? by topical_surfactant · · Score: 5, Funny
    Raid does not protect you from Hurricanes, or even fires.

    Termites, on the other hand...

  8. Re:B.S. D? by PartialInfinity · · Score: 5, Insightful

    Why do you have to settle for one or the other? A proper backup strategy, like any security strategy, should involve more than one technology.

    Hotswappable RAID has saved my servers on more than one occasion. Likewise, the servers have also been saved by tape backups. RAID5, tape backups, and data replication all have different pros and cons.

    I think it is incorrect to say RAID5 is not acceptable in any backup strategy. The more chances you get at data redundancy, recovery, and failover, the better off your organization.

  9. Re:B.S. D? by Desert+Raven · · Score: 5, Insightful

    I don't actually run RAID, but I've gotten some interesting stories from some (more than 1) people who do.

    I'll comment on this later...

    The weakest drive fails first. Power down the RAID box to replace the bad drive...

    OK, this is where I start getting dizzy. If their data is valuable enough to have RAID, why were they such cheap bastards that they didn't get hot-swap drives? I've worked in a LOT of places that have RAID systems, and three of my own servers have RAID, yet to date, none of them were anything but hot-swap. Additionally, with a small amount of intelligence and a few extra dollars, the administrator always puts in a hot-standby drive that will automatically take over if a drive fails, allowing for the failed drive to be replaced at a more convenient time than 1:30am without sacrificing the redundancy. Sysadmins running really critical systems will often have multiple hot-standby drives.

    The stress of the power-down and restart is enough to kill the second-weakest drive.

    Now, see, here's the funny part. When you spend the bucks for SCA hot-swap drives, you actually get drives of decent enough quality that this is very rarely a problem. Even if you did have to shut the array down, which you won't because you bought proper hardware.

    enough so that they've quit using RAID as "backup"

    Further evidence of idiocy. RAID is not a backup. RAID allows you to keep running in the event of a specific type of hardware failure. But that is all it protects you from. Backups are still just as critical as they were before you had RAID. Anyone who uses a RAID array instead of proper backups deserves to have their data sacrificed to the gods of entropy, shortly followed by their own careers.

    As for my delayed comment on the first sentence... Well, I suggest you get smarter friends.

  10. Real-time accidental deletion, too. by evilviper · · Score: 3, Insightful

    This is basically RAID over the network. Personally, I can't see a lot of use for it... Just put the second drive in the machine, and use software RAID, rather than putting the second drive in a network server. Less network slowdown and congestion that way, not to mention CPU-time wasted packetizing, encrypting, etc.

    As always, RAID (and now this) is not a backup solution.

    --
    Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
  11. Re:How is this different from Windows VSS? by Anonymous+Struct · · Score: 3, Informative

    As of Windows 2003 R2, there is a capability to do a VSS type of thing over the network to a remote server.

    I'm a little ashamed that I know that, but it's true.

  12. oops! by krismon · · Score: 4, Funny

    Oh no! the rootkit got replicated to the backup server!