Ext3cow Versioning File System Released For 2.6

Zachary Peterson writes "Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system as it appeared at any point in time through a natural, time-shifting interface. This is can be very useful for revision control, intrusion detection, preventing data loss, and meeting the requirements of data retention legislation. See the link for kernel patches and details."

Comments

So which is it?

[EveryNickIsTaken]

Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system... Well, is it the file system, or the file system manager?

Overhead?

[HateBreeder]

Couldn't find real-world information about space and performance overhead.

Does it store many copies of each file? or only the differences between the old and the new version?

CVS/Subversion replacement ?

[BuR4N]

This might be far fetched but how far off is it to use these filesystems as a revision control system replacement ?

Never tinkered with any of these filesystems, but wouldnt it be very comfortable for at least us developers to have a filesystem that worked something like Subversion. Just hook up something on the network and use it as the central code repository.

Re:CVS/Subversion replacement ?

[scottv67]

We should probably ask some VMS users about that. They had a versioned filesystem 20 years ago.

It's actually closer to 30 years ago. I can't believe VMS is celebrating it's thirtieth birthday this year.

http://h71000.www7.hp.com/openvms/25th/index.html

Having multiple versions of a file is *extremely* handy. That feature saved me bacon many-a-time. For those of you who have never been fortunate enough to login to a VMS system, the file versioning looks like this to the user: scott_file.txt;5 scott_file.txt;4 scott_file.txt;3 scott_file.txt;2 and so on The file version incremented each time you modified the file. You could set the number of file versions that the OS would keep for you. I don't remember the maximum number of versions of a file that you could keep but I remember seeing version numbers that were five digits wide. The version number wrapped after a while. Thanks, -Scott

Re:CVS/Subversion replacement ?

[Anonymous Coward]

Never tinkered with any of these filesystems, but wouldnt it be very comfortable for at least us developers to have a filesystem that worked something like Subversion. Just hook up something on the network and use it as the central code repository. You mean like what happens when you enable svn autoversioning and mount your repository as a filesystem?
http://svnbook.red-bean.com/nightly/en/svn.webdav. html

So

[El Lobo]

Is this like MS Shadow Copies or like Apple's Time Machine? Not trolling but just somebody enlight me, what is new here?

Re:So

[psbrogna]

I don't think it's supposed to be new (it's one of the things I miss most about VMS). It's outstanding functionality to have both for end users and sysgeeks/devs; built right into the file system level (ie. LOW). I prefer this approach to the hacks that other O/S's have implemented at a higher level. It's always harded to do something like this down deep at the roots rather than add it on as superficial gloss later. Granted, the end users don't usually notice or appreciate the diff but over time it keeps complex sys's like O/S's from becoming a teetering tower of shims and bolted on widgets.

Re:Hooray for repeating history!

[Anonymous Coward]

So because it was a good idea 20 years ago, it somehow isn't good that it's been implemented now? Sure, in an ideal world we'd all have been using versioned filesystems since the advent of VMS, but we havn't.

Actually a tell a lie; the ISO9660 spec. copies the VMS design and also allows files to have a version number, using the exact same scheme I.e. the version # is appended to the file following a semi-colon. So "FOO.BAR;1" is a valid ISO9660 filename.

VMS file versions someone?

[ntufar]

It reminds me of VMS file versions.

In VMS if you had a file named article.txt, each time you modified and saved it in editor, a new version was created named article.txt;1 article.txt;2 article.txt;3 and so forth. So after a long session of edit and saves you could end up with a hundred copies of file in your directory. A lot of clutter in the directory but easy access to older versions of the files.

With Ext2cow you basically get the same functionality in a bit different way. By default you see only article.txt file. If you need to access a previous version of the file you need to specify a cryptic code like this: article.txt@10233745. A bit cumbersome but, hey, how often you access older version of your file anyways. Looks better than VMS' approach.

This filesystem seems like a perfect solution for me as I am writing my Ph.D thesis. Currently I take backup every day and name it thesis20070420.tar.bz2, thesis200070421.tar.bz2, thesis20070422.tar.bz2 and so forth in case I need to go back and see how it looked some time ago.

However, in my home directory I have a lot of large audio and video files that I would never want to be versioned. I wander if Ext3cow keeps extra copies of the files if I move them around, change file named but do not modify the content. Probably I would have to make a new partition and put my text files I am working on there under Ext3cow and leave my media files on ext3.

Re:VMS file versions someone?

[physicsnick]

Hmm, when I read your post I thought I'd come here and suggest Subversion. Seems everyone else has done the same.

You really should use it. It's much easier to set up than you'd think, especially if you're on a Debian/Ubuntu box. If you use the file:/// syntax, you don't even need any kind of daemon or http server running; the client can do everything on its own. Say your thesis is currently sitting in ~/thesis, it's this easy to set up:

sudo apt-get install subversion
svnadmin create ~/thesisrepo
svn import ~/thesis file:///home/${USER}/thesisrepo -m "Initial import"
mv thesis thesisbackup
svn co file:///home/${USER}/thesisrepo thesis

That's it, you're done. ~/thesis is now a working copy of your repository, the repository itself (which will hold all versions of your files) is contained in ~/thesisrepo, and your original folder is backed up as ~/thesisbackup.

To work on your thesis, go into ~/thesis and start writing as you've always done. When you want to save a snapshot of the current state of your thesis (i.e. commit your changes), open a bash terminal, go into ~/thesis and type svn ci -m "some message". That's it. Much easier than running a backup; you can just stick it in a daily (even hourly) cron job. To back up all versions of the thesis on removable media, tar up the ~/thesisrepo folder and put it somewhere safe.

There's a bit more to know about it; namely you need to tell subversion when you add, remove, move or rename files. A good source for that is the Subversion Book, specifically Chapter 2.

Re:True undelete

[Anonymous Coward]

Could always just hardlink everything in certain directory trees to a backup dir that is swept periodically. Essentially just "tagging" not actually erasing.
We used that a lot on our servers to avoid accidents.
Does not protect against overwriting contents of course, and doesn't do revisions - but hey, you said undelete - not all the features of this tool.

Oh, and that has advantage of not taking up any extra disc space (apart from the tiny bit for the hardlinks and their tree).

Smells like dirvish

[Zekat]

This sounds like http://www.dirvish.org/, which is nearly as nice as the automatic file snapshots done by the "Network Appliance" fileserver boxes I've used at the last 2 out of 3 workplaces.

Security, backups

[Midnight Thunder]

This solution certainly helps if you accidentally delete something or need to go back to an older version. SVN is one solution, but it is a bit more explicit, while solutions like this and Apple's Time Machine help avoid needing to remember to update your repository. It should be noted that this doesn't replace backups, since this does not protect against hard-drive corruption. I do have a few of questions though:
    - what are the security considerations here?
    - can you delete the existence of file, as to ensure that it is not easily found again?
    - what are the effects on hard-disc storage space, ie are there any estimates to how much extra storage is needed for this?

Performance?

[Anonymous Coward]

So how does the mechanism affect performance? Aren't the files going to be very fragmented after a while? How long does it take to make those snapshots?

Re:True undelete

[jonadab]

Undelete isn't what makes this really cool, IMO. I don't generally delete stuff I still want, so that isn't really a big issue.

What I want, that a versioning filesystem can deliver, is the ability to revert a file back to an earlier version, after I've saved changes that turn out to be undesirable. This is a mistake I *do* make from time to time, often enough that I have been really hoping for a versioning filesystem in modern operating systems. This, to me, is a killer feature. I'm currently using FreeBSD, but this feature would by itself be enough to bring me back to using a Linux distribution, once it gets to the point of being included. Without it, once you save your changes and exit the application you can't go back. The past is lost. With a versioning filesystem, that's no longer true. I consider this to be *THE* feature for filesystems, far more important than things like journaling, much less performance tweaks. I have been wanting it ever since I saw the automatic versioning on OpenVMS, and I've been waiting, waiting, hoping, wondering why we don't have it in modern operating systems. I *want* this.

Interesting - I have a couple of questions

[ratboy666]

No flaming -- I don't have the time to research this, so I'll just post the questions!

1 - What happens to large databases? I am assuming a delta storage method, but that might slow down the database (specifically, I use mysql).

2 - Large files? Specifically, deletion (I store lots of videos)

3 - Usenet spools? (Lots of small files, deleted regularly).

I suspect that I would have to segregate my files...