Volume Shadow Copy For Linux?

An anonymous reader writes "I was asked to manage a number of Linux servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I normally manage Windows servers and on those I would just use Microsoft's Volume Shadow Copy for this. I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work. I found some versioning file systems out there like ext3cow and Tux3. Those look interesting, but I need something I can use on my existing ext3 file systems. I also found the R1Soft Hot Copy command-line utility, but it does not yet support my older 2.4 Linux servers. What are you using to make snapshots on Linux?"

Comments

no offense, but what a windows mentality

[Anonymous Coward]

Linux servers don't get backed up, they only get migrated to new hardware every five years. (BSD is the same, except for the second part.)

Re:no offense, but what a windows mentality

[mikemsd]

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

Re:no offense, but what a windows mentality

[Unoti]

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

There's a mix of humor and catty vitriol here all around, but here is something that addresses the serious point made in Grandparent's statement about it being a "Windows" way of thinking.

Take a look at Infrastructures.org which describes a whole way of thinking about server reliability and configuration. Where I work we essentially use this approach. The fundamental concepts around this approach concentrate more on system configuration, ability to pick a random server and drop it out the window and have another one just like it online in moments. It's less about backups, and far more about a more comprehensive disaster recovery/prevention type of thing. The types of approaches described there are probably more easily implemented using Unix/Linux, but is probably also possible with Windows boxes.

Suck it up

[mewsenews]

You will have to migrate your servers with plain ext3 to LVM-based ext3. Short term pain for long term gain.

Re:Suck it up

[Anpheus]

Windows does it in precisely the way uncoordinated open source projects will never be able to do it. They told the NTFS team, a new team in charge of the volume snapshot service, and the team in charge of the logical disk management to work together, create and perform regression tests against each others code on every check-in and patch, and likely set up team liaisons whose sole purpose was to ensure interoperability.

If you told me a third party open source organization, without having full control of the developers and direction of both the ext filesystems and the LVM system, was going to write a service that performed the same function as the volume snapshot service, I would laugh at you. I would laugh and laugh. Open source, because of its nature, tends to attract developers who want to do something, and they want it to be the best at that something. At the same time, they don't want to tie themselves down to stable APIs because, well, it can be limiting and slow development. I totally understand why. So telling me that some third party is going to extend LVM with one API, EXT with another API, and then write a service to coordinate the two is mind-boggling. Those people would have to constantly commit code to match changes in either of the two rapidly changing projects, they'd have to fully understand the inner workings of both ext and LVM, and then they'd have to make it all work without corrupting anyone's data and ruining their reputation beyond repair.

On the other hand, you have projects like ZFS or BTRFS, which are just as monolithic, but more ambitious, and powered by the same developers I mentioned above. They want their solution to be the best. It takes a long time though because they essentially have to start from scratch and incorporate all the things that appear to be within arm's reach. But the people who start projects like BTRFS realize that it's a fool's errand to try and create interoperability between massive, disparately managed open source projects. GNOME and KDE only survive because they threw everything else out and decided to simply come with their own full suite of stuff. X is its own long story.

I don't want to diss open source, like I said, it creates magnificent pieces of software, and the developers really, truly tend to care about their projects. (Even if they can be a little defensive, sometimes.) But without a dictator forcing cooperation between different teams, you often see open source reinventing the wheel. Sure, LVM and EXT3 could theoretically work together to provide sane, fast, performant snapshots. But I'd like to meet the person who thinks they can pull that project off.

"does not yet support my older 2.4 Linux servers"

[tlambert]

"does not yet support my older 2.4 Linux servers"

So upgrade your servers to a supported release instead?

-- Terry

Re:You're confused

[pbowen]

LVM snapshots only work fine if you are using LVM. I think the OP uses "regular partitions" to mean no volume manager in use.

There's some sort of confusion here

[vadim_t]

LVM snapshots work on a block level and don't care about the filesystem. A snapshot of any data in a logical volume should work fine, even if it's not a recognized filesystem.

A nice use for this is using a read/write snapshot to try different strategies for recovering a broken filesystem.

rsnapshot is what you're looking for

[xee]

RSnapshot uses a clever blend of rsync + hard links to do what you want... you can store many incremental backups in just a little more space than a full backup. you can run rsnapshot on a backup server with lots of disk space, and all you need to expose on your target machines is SSH.

you'd create "backup" users on all the target hosts, generate a PKI key pair, and put the private key on your backup server. put the public key in the "backup" account on each target machine so the backup server can securely login without a password. then you just set up rsnapshot to log into your targets and it will use rsync-over-ssh to pull the data.

http://rsnapshot.org/

Eh?

[ledow]

If you have backups, then moving to LVM is obviously the way to go if you desire snapshots. The others options are short-term hackery, LVM was designed from the ground up to do such things. And Ext3 has nothing to do with the price of butter.

To clarify, let me rephrase your question for the other way around

"I was asked to manage a number of *Windows* servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I tried Windows Shadow Copy, but most of the servers I manage run MBR partitions with FAT file systems, so Shadow Copy will not work. I found some versioning file systems out there... Those look interesting, but I need something I can use on my existing FAT file systems. I also found --random freeware--, but it does not yet support my older Windows NT 3.5 servers. What are you using to make snapshots on Windows?"

Except, in that case, it makes more sense because the filesystem is the determining factor, not the volume management. If you have LVM, it doesn't matter what the underlying filesystem is, really. Stop faffing about - if you have a server, with backups, that you need snapshots on, take the hit and wipe the drives to a config that supports that... while you're there upgrade that damn kernel already. If nothing else, it will test that the backups you're making are actually worth the effort. It's like complaining that 95 on FAT16 doesn't support Shadow Copy. If you absolutely *can't* take those servers down, or am unable to restore your backups to another machine for testing such changes (whether because of compatibility, software licensing and/or bad backups), you have bigger problems than some random desire for a feature you don't actually *need* at the moment.

Re:Eh?

[gbjbaanb]

If you absolutely *can't* take those servers down

If you can't take those servers down, nature will be getting ready to do it for you. At a time when you least want it's "assistance".

Re:You're confused

[impaledsunset]

If they are indeed regular partitions, he can't use LVM snapshots. However, the best solution is to convert from partitions to LVM volumes. It's a little effort to do so, but switching is worth it. Second best is to wait until btrfs is more usable. As a ZFS user, I can say that filesystem-level snapshot are much nicer than LVM snapshots in lots of ways.

Another possibility is to abuse hardlinks. You can create a copy of a directory with cp -al, and then overwrite (not modify) files on the original, you'd have copy-on-write copy. If you make your backups with rsync, you can configure rsync to never write to existing files and always overwrite, then use cp -al each weak or day, to store "incremental" backups for weeks and maybe more. I personally found this solution nice, but then I installed Solaris on the backup machine and used ZFS snapshots which do the same safer, simpler and more efficiently. If the backups are stored on a separate machine, switching it to Solaris is an option.

Another thing that can be done is to keep the LVM snapshots on a separate machine, and leave the current partitions as they are. It can be done with rsync, or a drdb device can be used to sync with the server (it can be created without reformatting the partitions, but you still need to make some changes like shrinking and/or moving the data, which might destroy the data if you don't know what you're doing).

Backup != snapshot != package management

[Alex Belits]

People expect a snapshot to be immediately usable and reliable, however in practice a state of device, even if synchronized with filesystem through its transaction is not a state of all data -- some data may be in buffers, prepared to be written, and rebooting into a restored filesystem may require some cleanup of such state. In particular, SQL databases are completely unsuitable for this kind of backup (this is why they have their own backup and transaction log handling procedures), and database-like applications such as mail servers, may require reindexing.

However for purposes other than those applications, file-level backup is entirely adequate, so utilities like rdiff-backup end up providing more functionality than complicated snapshot-handling procedures -- incremental backups for subtrees, readable trees in backup media, etc.

It also should be noted that backups should not be used as a replacement of package management -- on Linux anything installed through a package manager can be uninstalled through it.

Re:Backup != snapshot != package management

[bertok]

Absolutely not! Snapshots are perfectly safe for capturing your database data.

On real server operating systems snapshot support is integrated into applications, which receive a "snapshot about to occur event" so they can quiesce their writes for a short period to make the snapshot clean.

For example, on a Windows server, a VSS snapshot is a complete restorable backup of everything, including your databases, event logs, the registry, etc... It's the standard mechanism that practically all Windows backup tools use. They take a snapshot, back it up, and then release it. The point in time that the snapshot was taken turns up in the "last backed up on" date field in SQL Server!

Even third party snapshot mechanisms integrate using plugins. If you take a snapshot with, say, VMware or your SAN, then the same quiescing mechanism is triggered.

Some real server operating systems like HP-UX appear to have LVM extensions that are similar to what VSS can do, but I can't find the equivalent in Linux. From what I can see, the closest you can get is to temporarily halt writes from the ext3 filesystem, but that's not the same thing as proper application quiescing.

Re:You're confused

[amorsen]

You currently have ext3 fs that are NOT on LVM. In the future, choose LVM.

The choice isn't that simple. LVM comes with its own complications, including a tendency to get volume offsets "wrong" so the file system data doesn't align nicely to RAID stripes. This is not good for performance.

Also, LVM has only recently acquired barrier support, and the combination of no barriers + write cache can be quite dangerous if power is lost. Even battery backed cache won't save you if you use a journalling file system (and everybody does these days) because request ordering isn't guaranteed.

I haven't touched Solaris since it had a 2 in front of its version number, but I must admit that I suffer from ZFS envy.

Re:"does not yet support my older 2.4 Linux server

[impaledsunset]

I'm troubled why people still run 2.4 server. I remember the time when I was reluctant to upgrade to 2.6, and I used prefer the older 2.4, which felt more comfortable than 2.6, regardless of how tempting the new changes sounded. But now I don't see any reason I would run this anywhere, even my router runs 2.6. Especially on newer hardware, 2.4 is really really too old.

I know there are people who probably still run Linux 2.2, but that are probably systems that are running some task well enough to require any changes, and leaving them as they are is the best. Servers are usually not like that. They need security updates, upgrades to catch up with the times, and many other changes required by the circumstances (for example, adding snapshot abilities, for which some person asked recently on Slashdot). Most production servers are not systems that you just leave running, so upgrades to the kernel are also expected and highly recommended. Not to mention that most recent distributions require 2.6.

Re:hey retard:

[Omnifarious]

In case you hadn't realized this, It is possible to tell people to migrate to LVM without calling them names.

Re:nilfs

[Chainsaw]

Nerds I'd Like to... You know what - nevermind.

Re:You're confused

[nashv]

He isn't complaining. You seem to be responding to his mentioning that "he knows how to do this on Windows" , by interpreting it as "Why is Linux so broken that it can't do a simple thing like that?" This isn't a Linux versus Windows thing. This is a Windows user, migrating to Linux and wants to know how to accomplish something. Constructive answers are more useful in such cases than getting defensive by alleging hypocrisy and double standards.