Volume Shadow Copy For Linux?

An anonymous reader writes "I was asked to manage a number of Linux servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I normally manage Windows servers and on those I would just use Microsoft's Volume Shadow Copy for this. I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work. I found some versioning file systems out there like ext3cow and Tux3. Those look interesting, but I need something I can use on my existing ext3 file systems. I also found the R1Soft Hot Copy command-line utility, but it does not yet support my older 2.4 Linux servers. What are you using to make snapshots on Linux?"

Comments

Suck it up

[mewsenews]

You will have to migrate your servers with plain ext3 to LVM-based ext3. Short term pain for long term gain.

Re:Suck it up

[Anpheus]

Windows does it in precisely the way uncoordinated open source projects will never be able to do it. They told the NTFS team, a new team in charge of the volume snapshot service, and the team in charge of the logical disk management to work together, create and perform regression tests against each others code on every check-in and patch, and likely set up team liaisons whose sole purpose was to ensure interoperability.

If you told me a third party open source organization, without having full control of the developers and direction of both the ext filesystems and the LVM system, was going to write a service that performed the same function as the volume snapshot service, I would laugh at you. I would laugh and laugh. Open source, because of its nature, tends to attract developers who want to do something, and they want it to be the best at that something. At the same time, they don't want to tie themselves down to stable APIs because, well, it can be limiting and slow development. I totally understand why. So telling me that some third party is going to extend LVM with one API, EXT with another API, and then write a service to coordinate the two is mind-boggling. Those people would have to constantly commit code to match changes in either of the two rapidly changing projects, they'd have to fully understand the inner workings of both ext and LVM, and then they'd have to make it all work without corrupting anyone's data and ruining their reputation beyond repair.

On the other hand, you have projects like ZFS or BTRFS, which are just as monolithic, but more ambitious, and powered by the same developers I mentioned above. They want their solution to be the best. It takes a long time though because they essentially have to start from scratch and incorporate all the things that appear to be within arm's reach. But the people who start projects like BTRFS realize that it's a fool's errand to try and create interoperability between massive, disparately managed open source projects. GNOME and KDE only survive because they threw everything else out and decided to simply come with their own full suite of stuff. X is its own long story.

I don't want to diss open source, like I said, it creates magnificent pieces of software, and the developers really, truly tend to care about their projects. (Even if they can be a little defensive, sometimes.) But without a dictator forcing cooperation between different teams, you often see open source reinventing the wheel. Sure, LVM and EXT3 could theoretically work together to provide sane, fast, performant snapshots. But I'd like to meet the person who thinks they can pull that project off.

Re:Suck it up

[Anonymous Coward]

Oh, you poor deluded thing.

Windows does it in precisely the way uncoordinated open source projects will never be able to do it. They told the NTFS team, a new team in charge of the volume snapshot service, and the team in charge of the logical disk management to work together, create and perform regression tests against each others code on every check-in and patch, and likely set up team liaisons whose sole purpose was to ensure interoperability.

I'm sure they did have interfaces and test suites and stuff set up, but I'm wary about any Microsoft claims of functionality. Usually, Microsoft stuff works for a few fuzzily-bounded cases, but do something interesting and it fails. And if it fails somewhere, then good luck finding someone who will own up to it and take responsibility to make sure it works.

What I love about open source is that its mechanics are completely open and accessible.

Valid point about the dictator thing, though. That's why only a few projects do exceptionally good work, such as the OpenBSD/OpenSSH project.

Re:Suck it up

[Anpheus]

Like I said, run a production system with more than a few dozen LVM snapshots and tell me you're getting good performance.

The abstraction away in one hand is a plus, it makes many things simpler, the problem though is that your disks look like this:

| ext3 volume | ... | ext3 copy on write snapshot volume 1 | ... | ext3 copy on write snapshot volume 2 | ...

With ZFS, your old and stale data is mixed in one volume, and a defragmentation operation would have the opportunity to move the stale data out and move the current data in to make a directory contiguous again.

The problem is, LVM's complete abstraction of the block devices with no means for inter-allocating volumes or making requests to the filesystem to un-allocate small blocks within a volume to assign to snapshots means that performant snapshots on Linux will have to wait until ZFS, BTRFS, NILFS, whatever.

"does not yet support my older 2.4 Linux servers"

[tlambert]

"does not yet support my older 2.4 Linux servers"

So upgrade your servers to a supported release instead?

-- Terry

There's some sort of confusion here

[vadim_t]

LVM snapshots work on a block level and don't care about the filesystem. A snapshot of any data in a logical volume should work fine, even if it's not a recognized filesystem.

A nice use for this is using a read/write snapshot to try different strategies for recovering a broken filesystem.

Re:There's some sort of confusion here

[Unoti]

Yes, and that's what he should change. Don't you think?

Re:You're confused

[nschubach]

The double standard being that the Linux servers wouldn't need updated where the Windows servers would. There's an update that has to happen to support the feature. Linux is not immune to this (though it would likely do the update without a total rebuild opposed to Windows.)

Re:hey retard:

[Omnifarious]

In case you hadn't realized this, It is possible to tell people to migrate to LVM without calling them names.

Re:Eh?

[gbjbaanb]

If you absolutely *can't* take those servers down

If you can't take those servers down, nature will be getting ready to do it for you. At a time when you least want it's "assistance".

Re:You're confused

[nashv]

He isn't complaining. You seem to be responding to his mentioning that "he knows how to do this on Windows" , by interpreting it as "Why is Linux so broken that it can't do a simple thing like that?" This isn't a Linux versus Windows thing. This is a Windows user, migrating to Linux and wants to know how to accomplish something. Constructive answers are more useful in such cases than getting defensive by alleging hypocrisy and double standards.

Copy On Write, Where for art thou?

[Anonymous Coward]

Why does Linux still lack this functionality?

Since the early 1990's Novell has had the ability to "Salvage" deleted files and even maintained a near limitless amount of previous versions with a Copy On Write functionality. It still exists, even on Linux in their NSS(Novell Storage Services) Volumes.

Microsoft finally got on board when their Server 2003 product implemented Volume Shadow Copies. This isn't nearly as good as Novell's implementation but, it was better than anything Microsoft had previously offered.

The original poster did mention etx3cow, which offers an awesome feature set. But, etx3cow has been "under development" for a long time without ever catching on.

Ext4 has recently been incorporated into the Linux kernel and there just isn't any excuse for its lack of a Copy On Write version history. Yet here we are, in 2010, yet again answering this question without any good answer. Linux should have a standard Copy On Write file system a long time ago. Its continued absence is shameful.

ext3cow should be merged into ext4 yesterday!

Veritas Storage Foundation

[Anonymous Coward]

http://www.symantec.com/business/storage-foundation-basic

So, wheres the 6 bullet point solutions list?

[stimpleton]

I have scrolled to the bottem and replied after a scrolling skim. No answers for this guy yet? Just vague debate. No "use either a,b,c,d,or e".

Thats telling.

I am in this guys situation at work.

Is it necessary to be a dickhead?

[Beelzebud]

Reading through this thread has brought back the memories of when I first started using Linux. There is a subset of Linux users who seem to think that acting like a giant douche bag will help people adopt the platform.

Don't get me wrong, I've found that there are some amazing people in the Linux community that are more than willing to help out someone genuinely willing to learn, but there still exists this subset of assholes that seem to think ridicule, and basically acting like a dickhead makes them superior. If you're one of those people get over yourself. Linux would be better off without you!

Re:Why do mod points stop at FIVE?

[X0563511]

Indeed. After reading the posed questions, my answer is "suck it the fuck up already and start migrating into the current century"

That's a rough way to put it, but it gets the point across.

Re:"does not yet support my older 2.4 Linux server

[omglolbah]

Win2k is still used widely all over the world in production environments.

The problem with systems that work is that you're usually not to touch them until they stop working :(

Re:You're confused

[Anonymous Coward]

Go lookup dynamic vs basic volumes and quit being a prick.

It's not that simple.