Volume Shadow Copy For Linux?

An anonymous reader writes "I was asked to manage a number of Linux servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I normally manage Windows servers and on those I would just use Microsoft's Volume Shadow Copy for this. I tried Linux LVM snapshots, but most of the servers I manage run regular partitions with ext3 file systems, so LVM snapshots will not work. I found some versioning file systems out there like ext3cow and Tux3. Those look interesting, but I need something I can use on my existing ext3 file systems. I also found the R1Soft Hot Copy command-line utility, but it does not yet support my older 2.4 Linux servers. What are you using to make snapshots on Linux?"

Comments

nilfs

[pinkishpunk]

Nilfs will have those things, but you`ll have to wait till its production ready.

Re:nilfs

[Chainsaw]

Nerds I'd Like to... You know what - nevermind.

no offense, but what a windows mentality

[Anonymous Coward]

Linux servers don't get backed up, they only get migrated to new hardware every five years. (BSD is the same, except for the second part.)

Re:no offense, but what a windows mentality

[mikemsd]

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

Re:no offense, but what a windows mentality

[Unoti]

Sweet, I'm going to install Linux on all my systems. I didn't know that Linux could prevent natural and man made disasters as well as being a stable operating system. We've been wasting all this money on backup for all these years.

There's a mix of humor and catty vitriol here all around, but here is something that addresses the serious point made in Grandparent's statement about it being a "Windows" way of thinking.

Take a look at Infrastructures.org which describes a whole way of thinking about server reliability and configuration. Where I work we essentially use this approach. The fundamental concepts around this approach concentrate more on system configuration, ability to pick a random server and drop it out the window and have another one just like it online in moments. It's less about backups, and far more about a more comprehensive disaster recovery/prevention type of thing. The types of approaches described there are probably more easily implemented using Unix/Linux, but is probably also possible with Windows boxes.

Re:no offense, but what a windows mentality

[stefanlasiewski]

infrastructures.org looks interesting, but then I see they mention things like 'NetSaint' which was renamed to be Nagios about 7 years ago, and references to "LISA '98".

Some of this information looks old. Am I right? These days, shouldn't we be thinking more about virtualization and cloud infrastructure?

That said, they do touch upon many good ideas. It seems that many mid-sized shops do follow some similar ideas.

Re:no offense, but what a windows mentality

[Unoti]

Yes, it's very old. They also talk about using cvs for version control, and mention that that world has moved on to svn, and the world has moved on a couple of times since then even. We also use Nagios rather than more ancient monitoring software. But still the central ideas are sound, even with many details changed. And practical, too.

These ideas actually apply very much to cloud infrastructure. It's really all about the cloud-- considering a machine not as just "a machine", but instead thinking of it as having a base image with certain functionality bolted on top of it. Thinking of a machine not just as a machine, but as a replaceable/exchangeable component in a larger collective system. That essentially is cloud computing. The thing a lot of people don't consider is that even a smaller cluster of machines should/could be configuration managed, maintained, and viewed this way.

Re:no offense, but what a windows mentality

[vegiVamp]

Shadow copies are not about server reliability, they're about stupid users inadvertently overwriting or deleting wrong files, and allowing them to fix their mistakes themselves, without needing to access the backup system or bothering the system administrators.

Also, versioning filesystems make a copy-on-write, so there's a backup of *every* version of a file, and not only the versions that are there when the backup (or shadowcopy) runs. It's been only this week that we've been looking fruitlessly through the backups for some vanished files. We can only assume that the files were erroneously deleted on the same day they were created, before the backups had a chance of picking them up.

Suck it up

[mewsenews]

You will have to migrate your servers with plain ext3 to LVM-based ext3. Short term pain for long term gain.

Re:Suck it up

[dhanson865]

You cant do this in windows either, not with partitions. Thats why their solution is called shadow VOLUME. Cause it need VOLUMES to work.

Our solution is called LVM Snapshots cause it needs LVM VOLUMES to work.

Now is that so hard to understand?

Well it's obvious you have never used Volume Shadow Copy because in the windows world there is no practical difference between a partition and a volume. No I'm not joking, no I'm not being a Troll.

Find a Windows server with a single drive (basic disk) and a raid Array (Dynamic Disk)
Right Click on My Computer and choose Manage
Click on disk management
right click on a unallocated portion of a "basic disk" to "create a new partition"
right click on a unallocated portion of a "dynamic disk" to "create a new volume"
Give them both drive letters
Go to the properties for each volume/partition and marvel at how you can turn Volume Shadow Copies on for a Volume and a Partition.

OMG, Microsoft doesn't care if it's a volume or a partition. Oh well, at least it works like the end user would want it to in this case.

Yes Windows uses the terminology differently than hard core linux users do but unless you understand how Windows labels things saying something like "You cant do this in windows either, not with partitions." makes you sound like you don't know what you are talking about to anyone that has seen the process it actually takes to manage disks on a windows server.

Re:Suck it up

[Anpheus]

Windows does it in precisely the way uncoordinated open source projects will never be able to do it. They told the NTFS team, a new team in charge of the volume snapshot service, and the team in charge of the logical disk management to work together, create and perform regression tests against each others code on every check-in and patch, and likely set up team liaisons whose sole purpose was to ensure interoperability.

If you told me a third party open source organization, without having full control of the developers and direction of both the ext filesystems and the LVM system, was going to write a service that performed the same function as the volume snapshot service, I would laugh at you. I would laugh and laugh. Open source, because of its nature, tends to attract developers who want to do something, and they want it to be the best at that something. At the same time, they don't want to tie themselves down to stable APIs because, well, it can be limiting and slow development. I totally understand why. So telling me that some third party is going to extend LVM with one API, EXT with another API, and then write a service to coordinate the two is mind-boggling. Those people would have to constantly commit code to match changes in either of the two rapidly changing projects, they'd have to fully understand the inner workings of both ext and LVM, and then they'd have to make it all work without corrupting anyone's data and ruining their reputation beyond repair.

On the other hand, you have projects like ZFS or BTRFS, which are just as monolithic, but more ambitious, and powered by the same developers I mentioned above. They want their solution to be the best. It takes a long time though because they essentially have to start from scratch and incorporate all the things that appear to be within arm's reach. But the people who start projects like BTRFS realize that it's a fool's errand to try and create interoperability between massive, disparately managed open source projects. GNOME and KDE only survive because they threw everything else out and decided to simply come with their own full suite of stuff. X is its own long story.

I don't want to diss open source, like I said, it creates magnificent pieces of software, and the developers really, truly tend to care about their projects. (Even if they can be a little defensive, sometimes.) But without a dictator forcing cooperation between different teams, you often see open source reinventing the wheel. Sure, LVM and EXT3 could theoretically work together to provide sane, fast, performant snapshots. But I'd like to meet the person who thinks they can pull that project off.

Re:Suck it up

[Cramer]

This is just a matter of "words". In windows a "volume" is any mounted filesystem. The underlying storage system is pretty much irrelevant.

Re:Suck it up

[Anonymous Coward]

Oh, you poor deluded thing.

Windows does it in precisely the way uncoordinated open source projects will never be able to do it. They told the NTFS team, a new team in charge of the volume snapshot service, and the team in charge of the logical disk management to work together, create and perform regression tests against each others code on every check-in and patch, and likely set up team liaisons whose sole purpose was to ensure interoperability.

I'm sure they did have interfaces and test suites and stuff set up, but I'm wary about any Microsoft claims of functionality. Usually, Microsoft stuff works for a few fuzzily-bounded cases, but do something interesting and it fails. And if it fails somewhere, then good luck finding someone who will own up to it and take responsibility to make sure it works.

What I love about open source is that its mechanics are completely open and accessible.

Valid point about the dictator thing, though. That's why only a few projects do exceptionally good work, such as the OpenBSD/OpenSSH project.

Re:Suck it up

[Anpheus]

Like I said, run a production system with more than a few dozen LVM snapshots and tell me you're getting good performance.

The abstraction away in one hand is a plus, it makes many things simpler, the problem though is that your disks look like this:

| ext3 volume | ... | ext3 copy on write snapshot volume 1 | ... | ext3 copy on write snapshot volume 2 | ...

With ZFS, your old and stale data is mixed in one volume, and a defragmentation operation would have the opportunity to move the stale data out and move the current data in to make a directory contiguous again.

The problem is, LVM's complete abstraction of the block devices with no means for inter-allocating volumes or making requests to the filesystem to un-allocate small blocks within a volume to assign to snapshots means that performant snapshots on Linux will have to wait until ZFS, BTRFS, NILFS, whatever.

"does not yet support my older 2.4 Linux servers"

[tlambert]

"does not yet support my older 2.4 Linux servers"

So upgrade your servers to a supported release instead?

-- Terry

Re:You're confused

[pbowen]

LVM snapshots only work fine if you are using LVM. I think the OP uses "regular partitions" to mean no volume manager in use.

LVM Snapshots

[theunixman]

I use them on ext3 with no problems. It's true that very early on there was a problem with them and journaled filesystems, but that has long since been solved.

There's some sort of confusion here

[vadim_t]

LVM snapshots work on a block level and don't care about the filesystem. A snapshot of any data in a logical volume should work fine, even if it's not a recognized filesystem.

A nice use for this is using a read/write snapshot to try different strategies for recovering a broken filesystem.

Re:There's some sort of confusion here

[Unoti]

Yes, and that's what he should change. Don't you think?

ZFS?

[corran__horn]

I will admit that I have not tried it on Linux, but zfs is the best of the next gen filesystems. It does cryptographically assured reads and writes (remember that transitory undetected disk malfunctions occur at a rate of ~1/TB of data), it can snapshot changes, it fricken slices bread. If it had a gender, I would probably marry it (well, I guess I can date it for a while and see how things work out). http://zfs-fuse.net/

rsnapshot is what you're looking for

[xee]

RSnapshot uses a clever blend of rsync + hard links to do what you want... you can store many incremental backups in just a little more space than a full backup. you can run rsnapshot on a backup server with lots of disk space, and all you need to expose on your target machines is SSH.

you'd create "backup" users on all the target hosts, generate a PKI key pair, and put the private key on your backup server. put the public key in the "backup" account on each target machine so the backup server can securely login without a password. then you just set up rsnapshot to log into your targets and it will use rsync-over-ssh to pull the data.

http://rsnapshot.org/

Re:rsnapshot is what you're looking for

[dfn_deux]

This is no good for a true snapshot since the rsync operation is non-atomic on a live filesystem.

Re:rsnapshot is what you're looking for

[jgrahn]

[RSnapshot] is no good for a true snapshot since the rsync operation is non-atomic on a live filesystem.

I cannot help wondering when I read stuff like that who *really need* atomic, and who just like it because it sounds cool ... If that 2.4 guy doesn't really *need* theoretical atomicity, and he can do his work with something much more simple, he should.

Re:rsnapshot is what you're looking for

[dfn_deux]

i wasn't trying to guess at what he needed, but his question was about snap shotting. One of (if not THE) key feature of a snapshot is that it is atomic. Anything that rolls through a changing filesystem one file at a time is not going to fit that bill. Also you run the risk of making "backups" that could break things that presume state consistency. If you capture the log of a daemon before the product output then your backup could have no record of the event which created the output for example.

These types of concerns are of increasing importance to professional system administrators in a time where there (to me at least) seems to be an increasing focus on meeting legally mandated audit and retention requirements.

Re:rsnapshot is what you're looking for

[X0563511]

Rsync may tally up a list of files before it operates, but said files are not "locked" at all. Files at the start of the operation can be completely 'outdated' by the time it nears the end.

Re:rsnapshot is what you're looking for

[icebraining]

One file is atomically updated by rsync, but not the entire filesystem. So you can have a DB that uses 2 files, and when it finishes copying the first, the status of the second file has already changed and therefore the two aren't compatible.

LLVM snapshots do it atomically for the whole FS.

Re:rsnapshot is what you're looking for

[hasdikarlsam]

Unison does a better job, in that it checks for changes while it's running and can be configured to retry until there are none. However, it can still be tripped up by changes that touch multiple files; it doesn't give an atomic snapshot either.

In practice, though, it's never failed in giving me working backups.

Hobbling Along

[kwalker]

Since the situation is so hobbled (Old Linux kernel, no LVM) about the only thing you will be able to do is learn to use hardlinks. The ext* filesystems support them but you will have to manage them yourself (cp -varl /source/* /destination/version). Yeah it's a huge hack, but unless you can actually fix the problem, it's about your only hope.

Re:Hobbling Along

[ckthorp]

Or use rsnapshot and be lazy...

Eh?

[ledow]

If you have backups, then moving to LVM is obviously the way to go if you desire snapshots. The others options are short-term hackery, LVM was designed from the ground up to do such things. And Ext3 has nothing to do with the price of butter.

To clarify, let me rephrase your question for the other way around

"I was asked to manage a number of *Windows* servers at work. I would like to use volume snapshots to improve my backup scripts and keep recent copies of data around for quick restore. I tried Windows Shadow Copy, but most of the servers I manage run MBR partitions with FAT file systems, so Shadow Copy will not work. I found some versioning file systems out there... Those look interesting, but I need something I can use on my existing FAT file systems. I also found --random freeware--, but it does not yet support my older Windows NT 3.5 servers. What are you using to make snapshots on Windows?"

Except, in that case, it makes more sense because the filesystem is the determining factor, not the volume management. If you have LVM, it doesn't matter what the underlying filesystem is, really. Stop faffing about - if you have a server, with backups, that you need snapshots on, take the hit and wipe the drives to a config that supports that... while you're there upgrade that damn kernel already. If nothing else, it will test that the backups you're making are actually worth the effort. It's like complaining that 95 on FAT16 doesn't support Shadow Copy. If you absolutely *can't* take those servers down, or am unable to restore your backups to another machine for testing such changes (whether because of compatibility, software licensing and/or bad backups), you have bigger problems than some random desire for a feature you don't actually *need* at the moment.

Re:Eh?

[gbjbaanb]

If you absolutely *can't* take those servers down

If you can't take those servers down, nature will be getting ready to do it for you. At a time when you least want it's "assistance".

Re:You're confused

[impaledsunset]

If they are indeed regular partitions, he can't use LVM snapshots. However, the best solution is to convert from partitions to LVM volumes. It's a little effort to do so, but switching is worth it. Second best is to wait until btrfs is more usable. As a ZFS user, I can say that filesystem-level snapshot are much nicer than LVM snapshots in lots of ways.

Another possibility is to abuse hardlinks. You can create a copy of a directory with cp -al, and then overwrite (not modify) files on the original, you'd have copy-on-write copy. If you make your backups with rsync, you can configure rsync to never write to existing files and always overwrite, then use cp -al each weak or day, to store "incremental" backups for weeks and maybe more. I personally found this solution nice, but then I installed Solaris on the backup machine and used ZFS snapshots which do the same safer, simpler and more efficiently. If the backups are stored on a separate machine, switching it to Solaris is an option.

Another thing that can be done is to keep the LVM snapshots on a separate machine, and leave the current partitions as they are. It can be done with rsync, or a drdb device can be used to sync with the server (it can be created without reformatting the partitions, but you still need to make some changes like shrinking and/or moving the data, which might destroy the data if you don't know what you're doing).

Re:ummmm

[pavon]

Only works if the partitions don't change while you are copying them. The big advantage of using LVM for this is that you can create snapshots on a live system, without resorting to remounting the partition read-only (and all the problems that will cause).

But really, those are his only options. If you insist on using plain ext3 and won't add a layer of between the FS to allow for this, then you have no choice but to freeze the partition while doing a volume-level back up.

Re:"does not yet support my older 2.4 Linux server

[0racle]

There are Enterprise Linux distributions that are both supported and still run 2.4, though not for much longer. Not everyone runs Ubuntu.

Wow, where to start...

[CAIMLAS]

Knowing where to start on this is a bit of a miffing point.

First: upgrade your shit. 2.4 kernel systems? Are you running Redhat 6? You know, from the turn of the millennia.

Second: upgrade your shit. Really,

Third: if your kernels are that old and you're using these machines for file storage/backup, chances are the hardware needs to be replaced before you even consider considering messing with them. Seriously: this stuff is ancient. Even Debian hasn't had a 2.4 kernel in 5+ years, I think.

Third: you can do what you're trying to do with rsync 'snapshots'. It works very well, failing filesystem level support. If you're sharing data over samba, this makes it easy: just put a '.snapshot' dir for these 'temporary' backups in their $HOME and hide dotfiles. Then make sure rsync ignores .snapshot. (Of course, there are other ways to do this.)

rsync snapshots (and here).

There are other sources of information out ther on rsync snapshots. There's also rsnapshot.

Chances are you'll have to upgrade before this stuff even works for you, though.

Re:Wow, where to start...

[evilviper]

2.4 kernel systems? Are you running Redhat 6? You know, from the turn of the millennia.

Redhat 6 was a 2.2.x kernel. So was Redhat 7. While the VER FIRST release of kernel 2.4.x was indeed right around 2001, the most recent revision of 2.4.x was just 4 months ago.

you can do what you're trying to do with rsync 'snapshots'.

Not really. There's no atomic rsync snapshots, so while one file in the "snapshot" will be from 12:01, another file might be from 12:15, and depending on the application, the two files might be hopelessly irreconcilable. In fact there are programs like "VSS Backup Helper" to allow your rsync snapshots (on Windows XP or later) to actually get an atomic snapshot, by using said VSS features in Windows, which is being asked about. What's more, rsync operates on the file level, which is hopelessly slow... Run rsync on a mess of small files with hundreds of thousands of folders on a 2TB+ partition, and watch it take (literally) days to complete.

Chances are you'll have to upgrade before this stuff even works for you, though.

Rsync runs perfectly fine on a 2.4.x kernel. It'll run on a 2.2.x kernel, too. Hell, I've ported rsync to ancient proprietary Unix systems (using GCC2.9x) with minimal trouble.

What's with this ageism? You think old software bit-rots if left on it's own? This isn't Windows ME we're talking about here.

Backup != snapshot != package management

[Alex Belits]

People expect a snapshot to be immediately usable and reliable, however in practice a state of device, even if synchronized with filesystem through its transaction is not a state of all data -- some data may be in buffers, prepared to be written, and rebooting into a restored filesystem may require some cleanup of such state. In particular, SQL databases are completely unsuitable for this kind of backup (this is why they have their own backup and transaction log handling procedures), and database-like applications such as mail servers, may require reindexing.

However for purposes other than those applications, file-level backup is entirely adequate, so utilities like rdiff-backup end up providing more functionality than complicated snapshot-handling procedures -- incremental backups for subtrees, readable trees in backup media, etc.

It also should be noted that backups should not be used as a replacement of package management -- on Linux anything installed through a package manager can be uninstalled through it.

Re:Backup != snapshot != package management

[bertok]

Absolutely not! Snapshots are perfectly safe for capturing your database data.

On real server operating systems snapshot support is integrated into applications, which receive a "snapshot about to occur event" so they can quiesce their writes for a short period to make the snapshot clean.

For example, on a Windows server, a VSS snapshot is a complete restorable backup of everything, including your databases, event logs, the registry, etc... It's the standard mechanism that practically all Windows backup tools use. They take a snapshot, back it up, and then release it. The point in time that the snapshot was taken turns up in the "last backed up on" date field in SQL Server!

Even third party snapshot mechanisms integrate using plugins. If you take a snapshot with, say, VMware or your SAN, then the same quiescing mechanism is triggered.

Some real server operating systems like HP-UX appear to have LVM extensions that are similar to what VSS can do, but I can't find the equivalent in Linux. From what I can see, the closest you can get is to temporarily halt writes from the ext3 filesystem, but that's not the same thing as proper application quiescing.

Re:Backup != snapshot != package management

[Craig Ringer]

"In particular, SQL databases are completely unsuitable for this kind of backup (this is why they have their own backup and transaction log handling procedures)"

While snapshots aren't ideal for SQL DBs, any real snapshot is equivalent to a point-in-time copy of the state of the file system. Restoring it and starting the database should seem to the database just like it's recovering after unexpected power failure or a process crash.

Any database that doesn't recover properly after a snapshot restore will also fail to recover properly after powerfail or a sudden hardware reset, because it's not ordering its writes properly.

Of course, proper snapshot implementations (ie not LVM) notify apps that a snapshot is about to happen so they can pause their work and enter a stable, easy to recover from state for the moment it takes to make the snapshot. So it's even easier on the database.

Now, I'll grant that for databases it's usually *better* to do incremental block-level copies, SQL-level dumps, etc using the databases own tools because doing so is usually much more _efficient_ than taking a snapshot then archiving it somewhere. But sometimes you just want the snapshot around as insurance before doing a major config change or upgrade, and for that they're just unbeatable.

While I don't much like Windows servers in general, I have a major case of VSS envy (Volume Snapshot Service, not Visual Source Safe - blech!), because it's worlds ahead of anything seen on any open *nix and has been for nearly ten years. Hell, my one and only Windows server maintains in incremental backup of its self on a remote iSCSI volume, including many point-in-time snapshots, that I can just unplug from the iSCSI storage host and boot if I need to for disaster recovery. It's impressive, and VSS is the core of what makes it possible.

Re:You're confused

[amorsen]

You currently have ext3 fs that are NOT on LVM. In the future, choose LVM.

The choice isn't that simple. LVM comes with its own complications, including a tendency to get volume offsets "wrong" so the file system data doesn't align nicely to RAID stripes. This is not good for performance.

Also, LVM has only recently acquired barrier support, and the combination of no barriers + write cache can be quite dangerous if power is lost. Even battery backed cache won't save you if you use a journalling file system (and everybody does these days) because request ordering isn't guaranteed.

I haven't touched Solaris since it had a 2 in front of its version number, but I must admit that I suffer from ZFS envy.

Re:"does not yet support my older 2.4 Linux server

[impaledsunset]

I'm troubled why people still run 2.4 server. I remember the time when I was reluctant to upgrade to 2.6, and I used prefer the older 2.4, which felt more comfortable than 2.6, regardless of how tempting the new changes sounded. But now I don't see any reason I would run this anywhere, even my router runs 2.6. Especially on newer hardware, 2.4 is really really too old.

I know there are people who probably still run Linux 2.2, but that are probably systems that are running some task well enough to require any changes, and leaving them as they are is the best. Servers are usually not like that. They need security updates, upgrades to catch up with the times, and many other changes required by the circumstances (for example, adding snapshot abilities, for which some person asked recently on Slashdot). Most production servers are not systems that you just leave running, so upgrades to the kernel are also expected and highly recommended. Not to mention that most recent distributions require 2.6.

Re:You're confused

[nschubach]

The double standard being that the Linux servers wouldn't need updated where the Windows servers would. There's an update that has to happen to support the feature. Linux is not immune to this (though it would likely do the update without a total rebuild opposed to Windows.)

Re:You're confused

[Jason Earl]

With judicious use of --link-dest rsync can do this for you without having to use cp at all.

Re:hey retard:

[Omnifarious]

In case you hadn't realized this, It is possible to tell people to migrate to LVM without calling them names.

Upgrade kernel + R1Soft

[msh104]

Just upgrade your kernel using a manual build of the 2.6 kernel.
Also install static versions of the modutils ( insmod, modprobe, etc )
Use an external ( a machine with decent software ) for this so your compile doesn't break.
I have done so in the past and it works fine. ( and plan an update for those machines, anything with 2.4 is way to old... )

After that you can just use R1Soft hot copy,
http://www.r1soft.com/tools/linux-hot-copy/

This program is free ( as in beer ) to download and works with every block device.
You can even write to a block device if you really need to.

Their commercial offerings are pretty good as good. ( and DO work with the 2.4 kernel )
We use it here at work.

I heard btrfs supports something like this as well.

Any way, good luck!

Re:You're confused

[nashv]

He isn't complaining. You seem to be responding to his mentioning that "he knows how to do this on Windows" , by interpreting it as "Why is Linux so broken that it can't do a simple thing like that?" This isn't a Linux versus Windows thing. This is a Windows user, migrating to Linux and wants to know how to accomplish something. Constructive answers are more useful in such cases than getting defensive by alleging hypocrisy and double standards.

Copy On Write, Where for art thou?

[Anonymous Coward]

Why does Linux still lack this functionality?

Since the early 1990's Novell has had the ability to "Salvage" deleted files and even maintained a near limitless amount of previous versions with a Copy On Write functionality. It still exists, even on Linux in their NSS(Novell Storage Services) Volumes.

Microsoft finally got on board when their Server 2003 product implemented Volume Shadow Copies. This isn't nearly as good as Novell's implementation but, it was better than anything Microsoft had previously offered.

The original poster did mention etx3cow, which offers an awesome feature set. But, etx3cow has been "under development" for a long time without ever catching on.

Ext4 has recently been incorporated into the Linux kernel and there just isn't any excuse for its lack of a Copy On Write version history. Yet here we are, in 2010, yet again answering this question without any good answer. Linux should have a standard Copy On Write file system a long time ago. Its continued absence is shameful.

ext3cow should be merged into ext4 yesterday!

Re:What is wrong with just plain dump?

[Carl Drougge]

I'm pretty sure the Linux version of dump doesn't do any snapshoting. The FreeBSD version can do it because the FS supports snapshots, but ext3 does not. (Maybe it will do snapshots automatically if you have a setup that will support them, but the original problem is that this is not the case.)

A method for doing exactly that

[Tolaris]

Indeed. I've done so, and documented it:

http://www.tolaris.com/2010/05/06/moving-an-existing-backuppc-partition-to-lvm/

People smarter than me documented a way to move the data, live, even when the partition is nearly full. See the comments there.

Is it necessary to be a dickhead?

[Beelzebud]

Reading through this thread has brought back the memories of when I first started using Linux. There is a subset of Linux users who seem to think that acting like a giant douche bag will help people adopt the platform.

Don't get me wrong, I've found that there are some amazing people in the Linux community that are more than willing to help out someone genuinely willing to learn, but there still exists this subset of assholes that seem to think ridicule, and basically acting like a dickhead makes them superior. If you're one of those people get over yourself. Linux would be better off without you!

Re:Is it necessary to be a dickhead?

[cynyr]

yes, it could be phrased a bit better, "to get atomic backups of data, you need LVM with the snapshot module, or be using BTRFS. If the snapshot module for LVM is unavailable in 2.4.x, you will need to upgrade" The OP is basically asking how can i use shadow volume copy to back up my FAT16 partition, running on a windows 98 computer.

Re:You're confused

[evilviper]

It can be done with rsync, or a drdb device can be used to sync with the server (it can be created without reformatting the partitions, but you still need to make some changes like shrinking and/or moving the data, which might destroy the data if you don't know what you're doing).

DRBD does not require resizing partitions. The meta-data can be kept on a small partition on an entirely different disk if you so choose. At 128MB each, with minimal data being written, you could well plug-in a small USB Flash drive, and put the DRBD metadata for numerous partitions on it, without touching any of the hard drives.

Then, on the secondary DRBD system, you just "connect" the relevant partitions when a snapshot is desired, all changes get synced up to date, and then disconnect whenever you like. It's better than a snapshot, since it's on a physically separate system. If you had a large enough hard drive array (2TB SATA drives are cheap), you could even have a single server being the "DRBD secondary" server for dozens of other servers, receiving "snapshots" periodically, and perhaps, automatically...

Re:hey retard:

[19thNervousBreakdown]

LVM snapshots suck because you can't store the snapshot data on the filesystem you're snapshotting. Sure, there's tons of ways to come up with extra space to store the snapshot data in, but they're all gigantic pains in the ass.

All it needs is the ability to exclude particular blocks from the snapshot, which should be a ridiculously easy option to implement for anyone who's worked on the snapshot code, and then people who aren't experts in kernel hacking can take care of the rest of the layers to make it a user-friendly operation, but nobody will do that because of hurf durfy crap that doesn't account for how people actually use computers.

LVM snapshots for iterative backups?

[dstech]

Depending on how long you're keeping them around, LVM Snapshots are likely to be a bad choice anyway. Their intended use-case is to have a very short lifespan, because they're intended to be used like so:

1. Create snapshot

2. Mount snapshot & copy data to backup server

3. Unmount & destroy snapshot

The point behind them is to create an unchanging version of a live partition so that you can copy the data out without worrying about whether it is being updated while you copy. Since the snapshots keep a diff of all changes to the original volume, they continue to grow in size as you make changes to the original volume. When the snapshot runs out of space, it simply dies (completely... can't mount it or anything, just have to destroy it).

There are some other possibly valid use-cases (e.g., if you have simple throw-away virtual test machines, you can build a gold image, and snapshot it and then mount & use the snapshot, which allows for a quick restore to the gold state), but keeping iterative backup copies on the local volume for quick restoration isn't really the best idea.

Re:Why do mod points stop at FIVE?

[X0563511]

Indeed. After reading the posed questions, my answer is "suck it the fuck up already and start migrating into the current century"

That's a rough way to put it, but it gets the point across.

Re:"does not yet support my older 2.4 Linux server

[carton]

+1.

u r doing it rong.

If you need to keep around such old software, it needs to be running inside Xen/VirtualBox and/or become NFS-booted so that it's insulated from the hardware. That way, you're not forced to keep around old hardware to run your old software. If you insulate with Xen/VBox at the block level you can use LVM2 on the host system to do snapshots but are still constrained by the legacy filesystem. If you NFS-boot, you can use future filesystem-level snapshottable Linux filesystems to do snapshots, or you could buy a NetApp and use proprietary software to do the snapshots, or Solaris (if it sticks around), or any of a variety of things. You can argue about which level of storage insulation is best in the long run: the filesystem level has certain advantages at least with ZFS and probably with whatever future Linux snapshotting filesystem comes along becuase of variable block size: small files get blocks in 512-byte increments, and any file larger than 128kB gets a multiple-of-128kB sized block. Since blocks are the unit of compression and deduplication, you want the largest size block possible, but not too large or you suffer from the read-modify-write RAID5-like tax. A larger block will compress better and takes only one entry in the deduplication hash table. If you insulate at the block layer then all blocks will have to be the same size and a relatively small size, which makes compression and dedup work not nearly as well because they can't give big blocks to big files. however for old software you may want to change as little as possible, and especially with sketchy linux 2.4 NFS clients maybe it's not safe to run certain apps over an NFS root, or maybe your ancient distribution doesn't support NFS booting well.

One way or another, though, you need to find a way to keep the apps running while minimizing the blocks of ancient code on which you're still dependent, and this should be your overriding concern. You need to structure your plans to encapsulate the code subject to bit-rot rather than flailing around on /. for some freshmeat app-of-the-day that claims to solve * with FUSE and some stupid Perl script. This is the difference between a serious professional douchebag who surveys the industry and can smell the difference between high and low quality, and a pathetic flailing do-my-homework-for-me everything-inclding-windows-has-stengths-and-weaknesses-right-tool-for-the-job doomed blinking medicated idiot douchebag. Do not be that douchebag.

Re:hey retard:

[stikves]

Easy on the language...

But LVM is *the* solution on Linux. It's not very difficult to set up, and is very stable. As an additional bonus it supports all file system types (as long as they can exist on LVM volumes).

Re:"does not yet support my older 2.4 Linux server

[omglolbah]

Win2k is still used widely all over the world in production environments.

The problem with systems that work is that you're usually not to touch them until they stop working :(

Re:"does not yet support my older 2.4 Linux server

[Snowhare]

Even if you are using SUSE or RHEL, support for 2.4 is about to vanish completely. The last version of RHEL to use a 2.4 kernel was RHEL3 - which goes EOL October this year. I actually have one very old system to convert because of this. There is no version of SUSE using a 2.4 kernel and still in general or extended support. Even for 'self-support' (it's a bit of an oxymoron - it just means you can use the forums and the knowledge base but nothing is going to get fixed by Novell. How this is different from 'Googling it' isn't obvious to me), the last SUSE support for 2.4 will EOL in November 2012.

Re:You're confused

[Mr. Slippery]

Check out rsnapshot, which uses rsync and hard links.

Re:Virtualise!

[Cramer]

See Also: VMware consolidated backup

The guest OS has to have vmware tools running for the backup to work correctly. i.e. the VM has to be prepared to be backed up. And if there are things like a database running inside that VM, they have to be prepared for backup too. Simply copying the files out from under a running VM is a recipe for complete disaster.

Re:You're confused

[Craig Ringer]

LVM snapshots also just aren't all that good.

They require you to pre-allocate space, so you have to guess how much copy-on-write difference will accumulate between the original and snapshot over the lifetime of the snapshot.

If the snapshot runs out of space, it *should* cleanly disable its self. Pity about the file system mounted from it that has no idea its backing block device just vanished. It gets messy, fast, when an LVM snapshot runs out of storage.

LVM snapshots are really inefficient, because they track all block changes, not just user-level file data and metadata. This massively bloats the snapshot, reducing how long it lives until it runs out of backing store and disables its self.

LVM snapshots don't share backing store. If you have three of them, snapshot t+3 has to store all the data snapshot t+2 and snapshot t+1 do, and so on. The differences between the real fs (t) and snapshot t+1 land up being stored three times in three separately allocated backing stores. You waste a HUGE amount of space this way, and it's hard to reliably predict how much you need so your snapshots often vanish out from under you are you're trying to use them.

LVM is useful, but for someone used to the Volume Snapshot Service (VSS) on Windows servers, to ZFS, or to any of the "enterprise-y" file systems often seen deployed with big SANs, it's just going to make them cry.

Re:"does not yet support my older 2.4 Linux server

[ArsonSmith]

Exactly, about 2 1/2 years ago I worked for a large company that everyone here has heard of that at the time was running ~4000 servers on a modified Redhat 6.2 image. There was a large code base that got lost sometime in the start-up phase of the mid 90s that was much easier to never touch the OS then to re-write the code.

rsync --link-dest

[ranulf]

In that case, using rsync locally would be a good choice.

If you use --link-dest=DIR, rsync will hard link to any files that are identical, so you can have snapshots that are accessible as an entire tree but that consume little more space that a snapshot delta.

Re:"does not yet support my older 2.4 Linux server

[omglolbah]

Some of our systems still use token ring coax networks and OpenVMS servers.

The problem is that the fuckers just work... and keep on chugging along happily in the basement of the plant...