Slashdot Mirror

← Back to Stories (view on slashdot.org)

How To Perform a Bare-Metal Backup On Linux LVM

Posted by timothy on from the you-may-touch-only-the-heavy-atoms dept.

perlow writes "Using the free System Rescue CD you can perform bare-metal backups and restores of many types of computer systems. In this article, ZDNet columnist Jason Perlow explains the multi-step procedure in detail on an LVM-based system."

34 comments

  1. Another good disaster recovery solution by lazy_nihilist · · Score: 3, Informative

    http://www.mondorescue.org/

    1. Re:Another good disaster recovery solution by perlow · · Score: 2, Informative

      Mondo is cool, but it is not platform independent and usable on a live CD. It requires installation and compilation on each box to build its own rescue environent.

      The objective of my procedure was to come up with a method that could back up ANY Linux distribution at a low level in a very standardized and platform independent way, without sacrificing storage or performance OR TOUCHING THE OS ON THE BOX IN ANY WAY. This works with any Linux system you boot the System Rescue CD on, it requires no package or software installs or copied files onto the system you are backing up. None of the other methods that exist right now (System Imager, Mondo, even dd) will do -all- of the things I wanted it to do. My method is klunky as it is a manual process, but some enterprising programmer could probably script this in perl or python with a simple text menuing system pretty easily and have this built into the next System Rescue CD. That's what I was hoping for.

    2. Re:Another good disaster recovery solution by Anonymous Coward · · Score: 0

      I would like to point out to you that your manual restore procedure -- http://blogs.zdnet.com/BTL/?p=8759&page=4 -- is incomplete.

      You first need to use pvcreate with the --uuid option to reinitialize the PV partition *before* you can successully do the vgcfgrestore, otherwise you'll get an error like "Couldn't find device with uuid 'abcdef-1234...'"

    3. Re:Another good disaster recovery solution by donaldm · · Score: 1

      Mondo is excellent for disaster recovery of the Linux Operating System with the advantage of producing a recovery iso without shutting down the system. While you do need to compile the source it is fairly easy to do this and the compiled binaries can easily be installed on all other Linux OS's of the same type.

      I have found that Clonezilla is an excellent tool for creating recovery or deployment images however it is not really a true disaster recovery tool since you have to create an image by shutting down the system. Still if you are happy to do this it is quite fast (I get approx 35MB/sec on my laptop drive to my backup disk or server). I have used this tool to actually image and recover Linux, MS Windows and Solaris disks and it is surprising easy to do this. Since you are backing up an image the file-system and disk management layout is not important and all data can be faithfully recovered. I suppose you could call this a "poor man's RDP solution" and having used both I prefer Clonezilla any day since IMHO it is more flexible.

      While tools like Clonezilla and Mondo are good bare metal recovery tools it is always important that a good backup strategy is used for spot recovery (eg. dump or even cpio (ugg!)), however to backup a database it is best to use the correct database backup tools preferably those that don't shut the database down. Of course the larger the amount of data you need to backup the greater the need of a good disaster recovery strategy and this is a major problem that really needs addressing from the corporate business to the home user.

      --
      There ain't no such thing as proprietary standards only proprietary formats. Standards are by definition open.
    4. Re:Another good disaster recovery solution by perlow · · Score: 1


      I'll look into this and correct if necessary. It appears to be redundant because the config backup files contain the UUID.

  2. Sure - Don't. by pla · · Score: 4, Interesting

    Step 1 - If your LVM hasn't forgotten its own configuration yet, get that sucker off an LVM as fast as you possibly can.

    When I first discovered LVM (and it had already reached a quite mature V2 at that point), I thought it the greatest thing in volume management since the invention of the RAID.

    I didn't half-ass it, I took my time to learn how it works, set up and broke a few, added a new PV and grew it to make sure it worked, backed some up, recovered from deliberately broken volumes (which works great - as long as you never try to bring the broken volume online by, say, rebooting), etc. Then I turned it on and loved it.

    Then I received a hard lesson in what it means to say that JBOD has a failure rate multiplicatively proportional to that of each individual drive. And, surprise surprise, LVM amounts to nothing more than fancy OS-level JBOD, without even the performance boost of a proper RAID controller.

    Thanks but no thanks. If I want to span a volume across multiple disks, I'll use RAID 5 or 6 from now on, ThankYouVeryMuch. I can only thank Zeus that I hadn't yet gotten around to cannibalizing my old file server when the LVM one day decided to eat itself (even if I could have salvaged something from the individual drives, I couldn't once it "fixed" its configuration to reflect the new PV set).

    1. Re:Sure - Don't. by bernz · · Score: 3, Insightful

      You make sure that all your devices block devices (at least if you're in a "Real Enterprise") are at least mirrored. Just because LVM2 writes across all disks, doesn't mean you're in a single-point-of-failure mode (or RAID0 or JBOD). You just make sure your "disks" are not just a single Physical Disk, but rather a LUN composed of a RAID6 with multiple paths anf power-supplies.

      And if you're not an Enterprise, this can be done on the "cheap" with RAID6 commodity boxes running as iSCSI targets.

      If you're ever relying on a single device for ANYTHING you're doing it wrong. But LVM is perfectly safe if you do it in a responsible way. It just isn't necessarily super-cheap.

    2. Re:Sure - Don't. by Spirilis · · Score: 1

      Yeah, that's kinda how it works. LVM is a volume management system, not a data protection system. LVM is still decent, but use underlying RAID devices as your PVs. That way you still have the flexibility of LVM without being susceptible to single-disk failures.

      --
      the real at&t mix
    3. Re:Sure - Don't. by Henry+V+.009 · · Score: 1

      LVM shouldn't be used except on drives on a redundant RAID level. As long as you take that basic precaution, it's fine and can be fairly useful.

    4. Re:Sure - Don't. by robo_mojo · · Score: 1

      Then I received a hard lesson in what it means to say that JBOD has a failure rate multiplicatively proportional to that of each individual drive. And, surprise surprise, LVM amounts to nothing more than fancy OS-level JBOD, without even the performance boost of a proper RAID controller.

      Thanks but no thanks. If I want to span a volume across multiple disks, I'll use RAID 5 or 6 from now on, ThankYouVeryMuch.

      Your problem wasn't that you used LVM, your problem was that you made it into a JBOD. Some of us use LVM without making it a JBOD.

      And of course, there is nothing stopping you from using LVM on top of that RAID. I use LVM on a RAID 1 myself.
    5. Re:Sure - Don't. by gbjbaanb · · Score: 1

      so lets get this straight, just for the record. You installed LVM and added additional drives to a single volume (in much the same way as you'd do with a RAID0 setup) and when 1 drive failed you were surprised that you lost all your data?

      Its not magic, you need RAID1 or RAID5 to get protection from drive failure, or good backups. LVM has some features that make this a good choice (eg snapshots) but it is not a RAID solution. Stick it on a redundant array and you'll be fine.

    6. Re:Sure - Don't. by Chandon+Seldon · · Score: 1, Informative

      There are rarely free lunches when it comes to computer system configuration choices - and when there are free lunches, you can be sure that the hardware / software will be using that configuration by default.

      The safest bet is this: If you aren't sure that you understand the trade-off you are making by using a tool like LVM, you should assume that it's going to come back and bite you in the ass. This isn't just true for software stuff either, hardware RAID 0 has basically the same reliability issues - but it's still sometimes the right engineering choice for someone who understands the trade-offs.

      --
      -- The act of censorship is always worse than whatever is being censored. Always.
    7. Re:Sure - Don't. by dnamaners · · Score: 1

      Of course it would behave like JBOD if you applied LVM over many non-redundant volumes. To expect redundancy protection from a tool that does not provide it by nature is like expecting blood from stones. Use LVM to best effect on whopping great raid volumes. This way and you get the effect of partitions, the flexibility to change your mind about layout. Not to mention snapshot and other useful things. The redundancy is managed by the raid method you use. Choose your tools well, use more as needed.

      If all you have is a hammer, every problem will be treated like a nail.
      Thankfully I have a wrench as well.

    8. Re:Sure - Don't. by Anonymous Coward · · Score: 0

      You're doing it wrong.

      +5 Informative!

    9. Re:Sure - Don't. by drsmithy · · Score: 2, Interesting

      I didn't half-ass it, I took my time to learn how it works, [...]
      Then I received a hard lesson in what it means to say that JBOD has a failure rate multiplicatively proportional to that of each individual drive. And, surprise surprise, LVM amounts to nothing more than fancy OS-level JBOD, without even the performance boost of a proper RAID controller.

      These two statements are in conflict.

    10. Re:Sure - Don't. by pla · · Score: 2, Insightful

      If you're ever relying on a single device for ANYTHING you're doing it wrong. But LVM is perfectly safe if you do it in a responsible way.

      I agree completely - And mostly meant that as my point (with an anecdote about myself as an otherwise-knowledgeable user making a really really stupid mistake).

      LVM does work exactly as advertised - It just shouldn't have anywhere near the popularity it currently enjoys. For example, several Linux distros (Fedora comes to mind) set up an LVM by default. Ouch.

      But yes, in the right environment (low risk of, or penalty for, failure) it solves a number of problems nicely... Adding space, something RAIDs have traditionally made painful, it makes a breeze; and volume snapshots quite simply rock the casba. But for most users (pretty much any non-enterprise level user, for that matter), I would consider it far more dangerous than helpful.

    11. Re:Sure - Don't. by pipatron · · Score: 1

      I use LVM on all my single-drive setups. I use it on this 16GB compact flash-equipped laptop. It's very convenient to be able to shrink and grow partitions on the fly, when testing new distros or using virtualization. On the machines that I actually use it across more than one physical device, I make sure to have a good backup...

      --
      c++; /* this makes c bigger but returns the old value */
    12. Re:Sure - Don't. by Saeger · · Score: 1

      You can span LVM across many individual PV points of failure (i.e. single disks), but only morons do that. The right way to use LVM is to layer it above the redundant (RAID) storage, and/or make sure you have really good backups if you're only using LVM for its added features.

      LVM itself can also do simple mirroring (and striping) internally across multiple PVs, but this is used far less commonly than layering it over s/w, h/w, or remote RAID.

      --
      Power to the Peaceful
    13. Re:Sure - Don't. by Malevolyn · · Score: 1

      Now for the important question: what's the proper word for self-cannibalism? Autocannibalism?

      --
      Your ad here.
    14. Re:Sure - Don't. by petermgreen · · Score: 2, Informative

      IMO the big thing that LVM offers is it makes reproportioning easier and safer.

      With traditional partitioning you have to slide the data along the drive (which requires you to take the system offline, is time consuming and will probablly screw things up big time if power fails while doing it).

      With LVM, I can add unallocated space to any volume I like quickly and easilly. If the filesystem supports online resizing I can even do it without unmounting it.

      I agree though that while LVM does have JBOD and striping capabilities that are sometimes usefull the ramifications of using them should be carefully considered before going ahead.

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    15. Re:Sure - Don't. by Eggplant62 · · Score: 1

      You mean you actually thought at first that the LVM was supposed to replace a RAID mirrored and striped array? Son, that's where you made your mistake. Create the raid, put the LVM on top of the raid.

      That's just dumb.

    16. Re:Sure - Don't. by tzanger · · Score: 1

      I don't know about the rest of y'all, but my filesystems are generally RAID arrays running LVM on top of them. i.e. my home fileserver has a pair of 300G drives, four 250G drives and a pair of 200G drives. I have it configured into two RAID1s and a RAID5, and the three /dev/mdx nodes are PVs which go into my VG. I can survive any single drive failure without losing the VG, and I can grow it as I want, without having to worry about my older "big" drives getting waylaid. It's perhaps not an ideal setup, but unless you're a business with the budget to just toss old but good hardware, this works just fine for me.

    17. Re:Sure - Don't. by tzanger · · Score: 2, Insightful

      LVM does work exactly as advertised - It just shouldn't have anywhere near the popularity it currently enjoys. For example, several Linux distros (Fedora comes to mind) set up an LVM by default. Ouch.

      What the heck's wrong with that? I've been doing that manually on my cheap slackware systems for ages. I don't want hard partitions, and those who do are generally too young to remember the hell of C: D: E: F: G: H: drives in the old DOS days. Take a 250G drive, put a 5G / on it and LVM the rest for /usr, /home and most importantly, /var. I don't want all that shit together, and nobody is increasing the failure rate with a single drive.

      Hell, I'm running LVM on my laptop, for christ's sake. It's just good sense.

    18. Re:Sure - Don't. by Anonymous Coward · · Score: 0

      at that point), I thought it the greatest thing in volume management since the invention of the RAID.

      I didn't half-ass it, I took my time to learn how it works apparently not. anyone with more than 1/2 an ass would have not had any trouble figuring out that the underling block device needed to be fault-tolerant. Otherwise you might as well be running what amounts to Raid0. Please go back to using Windows where your level of inanity is expected.

      Talk about *sheesh*
  3. Hmm. by gbjbaanb · · Score: 3, Informative

    other features for bare-metal backup/restore involves:

    Systemimager (dead good, and also good to restoring an image to other machines too). Restores from a cd (or PXE) boot and pulls the necessary data from the server with minimal intervention.

    Any virtualisation technology - depends on your definition of 'bare metal', but if you think of it as an easy-to-restore server, then I guess it counts.

    r1soft's (paid-for) continuous backup solution. Not tried this, but many webhost type companies have said its the dogs danglies.

  4. LVM - Sync First by Anonymous Coward · · Score: 0

    sync before making snapshots.
    That is all.

  5. No! You have to use BOTH! by khasim · · Score: 2, Insightful

    LVM is NOT the same as RAID5.

    LVM makes it easy to move space around on the disks, but it does NOTHING to prevent data loss from failed disks.

    Put LVM on top of a RAID 1 or RAID 5 subsystem. Then you can add / replace disks and grow the volumes to use the new space.

    1. Re:No! You have to use BOTH! by bigredradio · · Score: 1

      Not RAID 5, but you can map two physical extents on different disks to a single logical extent. This would be similar to RAID 1.

      --
      Flexible bare-metal recovery for Linux/UNIX
  6. Enterprise backup by Anonymous Coward · · Score: 0

    In cases where systems are using LVM and BMR is necessary, it might be wise to (heaven forbid) spend a little money and buy a product. Our company uses TSM for data and Storix for the bare-metal backups. (yes they support LVM)



    When it comes to critical stuff, I like the idea of having someone to call when all hell breaks loose.

  7. I prefer old school backup by dnamaners · · Score: 3, Interesting

    Don't get me wrong, I love centralized backups and backup server/client software (bacula for example). However I use old school methods for a bare metal restore. For simple things blanking the free space and then make a "dd" image is all you need. However you do it, keep it simple, and there may be no need to do bare metal on the whole of a file servers data set. Once I get the core of it back up I can use my favorite software to get the system up-to-the-last-snapshot current.

    LVM and Raid can be a little complex if you boot off it so ill give some tips. A nice small boot like SLAX (may favorite) or Knoppix is needed to start any bare metal restore. Pick something that supports the hardware and had LVM and or Raid. Either make a custom disk to add your backup data to the volume or have a 2nd DVD drive. Boot and restore.

    I make my backups via simple methods like rsync and tar. I use a few simple scrips to do it all. So simple that it is easy to restore by hand if need be. This makes the restore safer as it can be done a host of ways. Not to mention these unix tools are present in all distributions. Probably the biggest reason for the old school way is that on a bare metal restore often hardware may change. New drives etc, basically failure can be a reason to upgrade etc. Many utilities can choke on big hardware changes, new drive sizes and such.

    I take detailed system information and note it in the backup as human notes. Then grab your raid & LVM configuration and write a quick script to rebuild these features via the command line. Basically just a list of the actions you take to make a file system and format them the desired files system types manually. These same scripts will also re-apply boot loader and such as well. If the volumes change after a failure I can just issue updated commands (or edit the script) and make a similar layout of the filesystem. Then just open the tar balls via script or by hand and populate the new filesystem with the files you archived. With a bit of practice all of this can be automated by scripts. But if you need to vary the procedure due to hardware changes or as a way to clone a system you can.

    1. Re:I prefer old school backup by Anonymous Coward · · Score: 0

      I just have a script that snapshots the LV, then uses good old 'dump' to save it off to a CIFS mount, then remove the snapshot. Repeat for each LV. Full once per week, incrementals the other 6 days.

      Works fine and I can restore individual files as well.

    2. Re:I prefer old school backup by wertarbyte · · Score: 1

      I'm using tar to store the stuff on-the-fly to an FTP site. I even made a flexible backup script to coordinate upload, snapshotting, encryption and everything else. It's completely based on standard unix tools, so I won't have to install any special backup/restore software when recoverying my server.

      --
      Life is just nature's way of keeping meat fresh.
  8. The RedHat shop way by EvilAlphonso · · Score: 1
    Kickstart from the satellite for the OS CVS checkout in the %post section of the kickstart for /etc or use the configuration channels amrecover for the data and applications not stored on the satellite

    As long as you keep a sane filesystem plan, the entire thing can be kept in an easy to follow procedure and fully scripted (except for the tape change on the amanda server).

  9. step 1: know unix by Anonymous Coward · · Score: 0

    Um, never heard of 'dd' eh?

    try 'man dd' and be enlightened to how redundant and pointless this article is.