Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: Asynchronous RAID-1 Free Software Backup For Laptops?

Posted by timothy on from the just-the-changes-ma'am-just-the-changes dept.

First time accepted submitter ormembar writes "I have a laptop with a 1 TB hard disk. I use rsync to perform my backups (hopefully quite regularly) on an external 1 TB hard disk. But, with such a large hard disk, it takes quite some time to perform backups because rsync scans the whole disk for updates (15 minutes in average). Does it exist somewhere a kind of asynchronous RAID-1 free software that would record in a journal all the changes that I perform on the disk and replay this journal later, when I plug my external hard disk on the laptop? I guess that it would be faster than usual backup solutions (rsync, unison, you name it) that scan the whole partitions every time. Do you feel the same annoyance when backing up laptops?"

7 of 227 comments (clear)

  1. mdadm can do this by Fruit · · Score: 5, Informative

    Use mdadm -C -b internal to create a bitmap. Detach and readd the mirror at will and it will only sync the difference.

    1. Re:mdadm can do this by Anonymous Coward · · Score: 5, Informative

      Effectively you create a RAID 1 mirror. When you remove the external drive the RAID degrades. The raid bitmap keeps track of changes. When you plug the external drive in you just have to tell it to bring it up to date. Which syncs the only changes.

  2. Obligatory by Anonymous Coward · · Score: 5, Informative

    RAID is not backup.

    1. Re:Obligatory by XanC · · Score: 5, Informative

      True. I'd recommend he check out rdiff-backup, which keeps snapshots of previous syncs. Fantastic tool.

  3. Exclude directories by Anonymous Coward · · Score: 5, Informative

    Are you backing up EVERYTHING on the laptop -- OS and data included? Even if you are only backing up your home directory there is stuff you don't need to backup like the .thumbnails directory which can be quite large. Try using rysnc's exclude option to restrict the backup to only what you care about.

    DNA
    AKA mrascii

  4. COW or desync'ed RAID by phorm · · Score: 5, Informative

    In this case, it sounds like you want a fast on-demand sync rather than a RAID.

    However, you could possibly use dm-raid for this if you're a linux user.
    Have the internal disk(s) as a degraded md-raid1 partition. When you connect the backup disk, have it become part of the RAID and the disks should sync up. That said, it likely won't be any faster than rsync, quite possibly slower as it'll have to go over the entire volume.

    Alternate solutions:
    * Have a local folder that does daily syncs/backups. Move those to the external storage when it's connected.
        CAVEATS: Takes space until the external disk is available
    * Use a differential filesystem, or maybe something like a COW (copy-on-write) filesystem. Have the COW system sync over to the backup disk (when connected) and then merge it into the main filesystem tree after sync
        For example, /home is a combination of /mnt/home-ro (ro) and /mnt/home-rw (rw, COW filesystem). When external media is connected, /mnt/home-rw is synced to external media, then back over /mnt/home-ro

  5. Re:find & diff by Anonymous Coward · · Score: 5, Informative

    It's different in that you don't have to sit and wait for it and doing the backup will consist of only the actual copying

    I suggest you look again at rsync.
      - It compares changed files and copies only what has been changed. Changed files are identified by differing mtimes (by default).
      - rsync can also handle removed files with the --delete option.
      - It can do the entire filesystem tree in a single command
      - There are filter options so you can include/exclude what paths to copy (eg you don't want to copy /proc and there are some directories such as /tmp and /run which you may not care about).