Ext3cow Versioning File System Released For 2.6

Zachary Peterson writes "Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system as it appeared at any point in time through a natural, time-shifting interface. This is can be very useful for revision control, intrusion detection, preventing data loss, and meeting the requirements of data retention legislation. See the link for kernel patches and details."

Comments

So which is it?

[EveryNickIsTaken]

Ext3cow, an open-source versioning file system based on ext3, has been released for the 2.6 Linux kernel. Ext3cow allows users to view their file system... Well, is it the file system, or the file system manager?

Re:So which is it?

[Bob54321]

From the example screenshot it appears it is a file system. You take a snapshot of your system at some point in time and it stores this data even when files change. Of course, with any file system it is important to have functionality that allows you to view the files as well...

Re:So which is it?

[hpavc]

You don't take a snapshot, thats the big deal with it.

Re:So which is it?

[Anonymous Coward]

here is another screenshot.

Re:So which is it?

[beset]

Is it not just a layer on top of the filesystem, like VSS / Shadow copy?

Implementing this on a network with a lot of shared drives is a godsend. You go from constantly pulling "critical" deleted files off tape to almost never having to go near one.

Re:So which is it?

[orangesquid]

Actually, it looks like you do take a snapshot from the screenshot, and with how some programs are designed, this might be sometimes better if you're going to use it for logs or programs that auto-save every N minutes. I can't get the page to load, and the coral cache isn't NS resolving (?), so I'm not completely sure, but the wikipedia page on Versioning file systems implies that ext3cow is snapshot-based.

I want to put ext3cow together with SELinux and call it Secure Cow---no, better yet, Sacred Cow. How about it?

Under VMS (Files-11 is a versioning FS), there are many ways of opening a file, and I believe appending records to a record-oriented file doesn't actually create a new version (correct me if I'm wrong), which is a good idea for logs; the number of retained versions is also adjustable.

http://n0x.org/copyfs/ and http://wayback.sourceforge.net/ are purportedly two versioning FS's for linux.

Re:So which is it?

[N3WBI3]

What I cant find on the site is how, in terms of space and speed, this is better than version files on a regular old ext3 filesystem.

Re:So which is it?

[JensenDied]

great, a screen shot like that is worse that most conspiracy nuts pictures.

Re:So which is it?

[Anonymous Coward]

What do "it" and "the" refer to? And what the heck is a "file system manager"? And why does such an ignorant retarded illiterate question get a score of 3?

There was a time when /. wasn't populate by morons, but it didn't last very long.

Re:So which is it?

[Man Eating Duck]

FWIW, if you remove the "_thumb" from the url you end up with a legible version.

What a name

[Anonymous Coward]

So is it EXT or is it just a FAT cow?

Re:What a name

[morgan_greywolf]

Well, they were originally going to call it "Rosie O'Donell Versioning File System" but the name was too long and the acronym ROVFS just conjured images of that awful rap by "MC Rove" at the awards dinner.

Re:What a name

[suggsjc]

I know the parent was making a joke, but the name might actually have a bigger impact than expected.

If it really wanted to take some market share, why not ext3+?

Re:What a name

[Anonymous Coward]

You're no one to talk, weighing over 400 lbs as you do. At least Rosie O'Donnell has a life, rather than sitting at a terminal all day long stuffing your face with RC cola and snickers bars.

Overhead?

[HateBreeder]

Couldn't find real-world information about space and performance overhead.

Does it store many copies of each file? or only the differences between the old and the new version?

Re:Overhead?

[xumio]

it is done by storing different blocks.

Re:Overhead?

[JoeD]

Check the "Publications" link. The first one is an article in "ACM Transactions on Storage".

It's a bit dry, but there is an explanation of how it stores the versions, plus some performance benchmarks.

Re:Overhead?

[DaveCar]

Couldn't read TFA (slashdotted), but I would *imagine* that 'cow' is copy on write and that it just uses new blocks for the changes - so only the differences, but not minimal differences.

Re:Overhead?

[Anonymous Coward]

Generally speaking (the site is slashdotted, so I can't read the specifics on this implementation)- when you write out files to the drive they spread out all over the place and each chunk has an i-node or information node that tells a little about what file it is from, and points to the next and last inodes, etc.. what happens when you snapshot is, it starts creating an alternate chain if things change. So say you have a file with 3 inodes and you create a snapshot- and then change the middle i-node, you're now taking 4 inodes worth of space. They're not completely separate copies. Now if you change one of the remaining two, you're only taking 5 inodes worth of space. If you end up changing all of them, only then do you essentially have a duplicate.

Re:Overhead?

[anilg]

COW has been present for a long time in ZFS since Solaris 10. The overhead there is negligible and its quite stable. Last I heard, it was being ported to FUSE on linux. Upcoming in the next releases of FreeBSD and OSX. Wiki has a pretty good article.

Re:Overhead?

[init100]

Generally speaking - when you write out files to the drive they spread out all over the place and each chunk has an i-node or information node that tells a little about what file it is from, and points to the next and last inodes,

Umm, no. At least for ext3 and similar filesystems, each file or directory corresponds to exactly one inode. The inode contains information about its owner, group, filetype (plain file, directory, symbolic link, FIFO, device file, etc), as well as permission information and extended attributes (such as for ACLs, SELinux security contexts, etc). It also contains pointers to blocklists, but each block does not have a separate inode.

Re:Overhead?

[Xiph1980]

How would it work if you have a certain file of n nodes, and you insert data at the start of the file. Does it shift the data adding a node at the start? Or does it read it as all new blocks adding n+1 new nodes...
The first would cost a bit more performance, to check what actually did change, and what not, but the latter would cost a lot of diskspace.

Re:Overhead?

[arodland]

Unix has no "insert in the beginning" or "insert in the middle" operations on files. Nor "remove from the beginning" or "remove from the middle". You can write to the end, you can overwrite any data that's in the middle, or you can truncate. That's about it. Anything else requires you to read-modify-write, either through memory or through a temporary file.

Re:Overhead?

[51mon]

COW is available in Linux for all file systems that support the appropriate quiescing system call (i.e. as of April - ext3, ext4, gfs2, jfs, reiserfs, xfs).

So if you just want block level copy on write you get this via Linux LVM already, with the snapshot facility.

It looks to me that the difference here is the metadata storage, and interface. i.e. Some understanding of the filesystems structure is introduced into the process, which will improve performance, and reduce overheads, and no doubt provide other potential benefits to in the future.

Although on the other hand it is increased complexity and mixing of layers, file systems were hard enough without these complexities.

Since the example filename given is a valid filename, the interface may be a tad suspect as well. You pays your money (or not), and still choose ext3 or reiserFSv3 on Linux, if you still want your data tomorrow without excesses fsck times or memory usage.

CVS/Subversion replacement ?

[BuR4N]

This might be far fetched but how far off is it to use these filesystems as a revision control system replacement ?

Never tinkered with any of these filesystems, but wouldnt it be very comfortable for at least us developers to have a filesystem that worked something like Subversion. Just hook up something on the network and use it as the central code repository.

Re:CVS/Subversion replacement ?

[markov_chain]

There are systems like Clearcase that do that, but the maintenance is a pain. (Not to mention not FOSS).

Re:CVS/Subversion replacement ?

[Bacon Bits]

It's a bad idea to use this kind of thing for version control, IMX. The documentation through TFA is very... sparse.

Q: What happens to old snapshots when the disk begins to fill up?
Q: How do I manage snapshots?
Q: Are snapshots atomic?
Q: What happens when a snapshot fails? What can cause a snapshot to fail?

Windows Server 2003's Shadow Copies works in much the same way, AFAICT, and MS goes out of their way to caution against using Shadow Copies as a replacement for backup or version control. I expect this kind of thing will be used for the same uses as Linux: it [probably] allows you to access locked files, and lets you recover accidentally deleted or modified files.

Re:CVS/Subversion replacement ?

[mlock]

Did you look at FSVS http://freshmeat.net/projects/fsvs? Might do what you need - snapshots your system using a subversion repository. And allows for multiple users - like every subversion repository.

Re:CVS/Subversion replacement ?

[hey!]

Version control is to this thing as keeping your vehicle under control while you drive is to having an airbag.

The point of version control is embodied in the name -- it gives you control. Not only does it give you the power to time travel to specific dates, it gives you the ability to find specific versions, to branch and merge, to mediate cooperation between developers.

This sort of thing would be useful in certain version control scenarios, e.g. the guy who checked out the software and has been modifying it without commits for weeks on end.

Re:CVS/Subversion replacement ?

[herrlich_98]

Clearcase does have a higher admin overhead than CVS. Clearcase also does not work particularly well over a WAN and I suspect ext3cow or something similar would have the same issue. The use model for a SCM tool based on ext3cow would be similar to NFS which you usually do not use on a WAN.

Re:CVS/Subversion replacement ?

[cduffy]

Subversion's backend is a transactional filesystem (though it sits on top of a BDB interface or a separate FS), and many of the tools it provides work by describing a set of changes as filesystem operations (go down this directory, now go down that directory, now open this file, now seek to this position, now write this text...)

That said, revision control is about much, much more than just storing snapshots that can be retrieved later. Think about branching and merging -- particularly intelligent merge algorithms where you want metadata about exactly where each patch that was merged came from such that if you merge two branches you aren't trying to re-merge patches common between the two. Also, the storage constraints for a revision control system are tighter -- you want to compress your deltas, since you're going to be storing tens of thousands of revisions.

There are also requirements such as being able to backup and restore your full history; modify it (in corner cases -- let's say a developer checked in some test code with a critical password hardcoded); and do all sorts of other things that a COW filesystem won't typically have written support for.

Taking a naive view of revision control that leaves out branching and merging is a sure way to make revctl geeks mad at you (or ignite a snapshot-versus-changeset flamewar -- the snapshot view, which is what a filesystem approach [like what SVN does] lends itself to, is considerably less powerful than the changeset approach [like what BitKeeper or Arch do] in terms of the merge algorithms that can be welded on top without a force-fit).

Re:CVS/Subversion replacement ?

[644bd346996]

The first thing I thought when I saw the headline was this: Don't we already have GIT?

Take a look at this: http://kerneltrap.org/node/4982 Note particularly the bit where Linus says

In many ways you can just see git as a filesystem - it's content-addressable, and it has a notion of versioning, but I really really designed it coming at the problem from the viewpoint of a _filesystem_ person (hey, kernels is what I do), and I actually have absolutely _zero_ interest in creating a traditional SCM system.

Re:CVS/Subversion replacement ?

[ciggieposeur]

This might be far fetched but how far off is it to use these filesystems as a revision control system replacement ?

We should probably ask some VMS users about that. They had a versioned filesystem 20 years ago.

Re:CVS/Subversion replacement ?

[maraist]

Clear-case requires that your work-space be on a mounted volume called a view. So there is significantly more overhead to clearcase than subversion/cvs/rcs. It's been a while since I've worked with it, and it was on different hardware than I'd normally use, so I can't recall how efficient checkins were. Theoretically they would be efficient, because you've spread the cost of it over the course of all file-operations.

Re:CVS/Subversion replacement ?

[maraist]

wouldn't it be very comfortable for at least us developers to have a filesystem that worked something like Subversion

Something like.. Depends.. They solve very different problems. Consider the 'work-space'.. Where is that in the file-system version-control. Clear-case, the only other file-system VSC I'm aware of has an explicit 'view' directory structure which mirrors the read-only master data. All mutations of data are through admin commands (which don't act like a file-system at all) or manipulations with your particular view directory-tree. The goal of this project AFAIK is to have a single-command 'snapshot', and otherwise the file-system would emulate a normal file-system. clear-case requires an entire book of CLI commands which you have to learn to make their file-system operate in a useful way.

The reason you need a view in version-control is that you have multiple people editing the same file at the same time.. Aside from MS's file-locking in the 80s and 90s, it's never a good idea to share a view with two people (or obviously even two work-spaces).

When you want an atomic snapshot, it's going to be of all directories. But, by nature, different people will be in different states when this snapshot occurs. You can't cleanly separate out different project snapshots (or even tasks within the project) unless you use completely different file-systems (e.g. partitions).

Theoretically you could devise a file-system that works on sub-directory trees, but I can't imagine how that works with deleted files.. A non-directory/file-name specific method would work off inodes, which can live both inside and outside of a directory. Granted this is an implementation detail, but a non-trivial one. At least, not for UNIX environments.

So, in a primitive sense.. Having an NFS or SAMBA mountable drive, which people collaborate in, then have a periodic cron-task run snapshot can be pretty useful.. Especially if the data is mostly read-only (such as a website publication). Individuals can check-back and see what has changed in their HTML or image files pretty easily (by merely referencing an obscure file-name extension). Though GUI tools might be frustrated since you don't normally manually type in a file-name, you pick from a list, which gets populated with the appropriate directory-listing command, which necessarily would only ever look at the current snapshot.

Currently you can use subversion as this sort of shared-file-system through webdav clients (again, by using an external or periodic checkin command). So effectively nothing is gained in this respect.

Where I really see advantage is for the items that they list in their mission-statement.. Auditability/security, and backup support. If you install an RPM or run 'make install' as root, you can do a system-wide check of what has actually changed (though I haven't found their performance metrics on this). Depending on the available tools, you could effectively undo an RPM install (if nothing else, then by "cp $file@$timestamp $file" all over the place).

I see this as an excellent file-system for /etc, /usr, /home. Probably not so good for /var (unless rollback support is VERY efficient - imagine rolling back the database to an hour ago after a massive mistake). I absolutely love this for any registry-style configuration trees (basically most things in /home). Probably would suck for /tmp (as the whole point of that directory is usually for ultra-performance, plus it would quickly fill up the disk).

Re:CVS/Subversion replacement ?

[scottv67]

We should probably ask some VMS users about that. They had a versioned filesystem 20 years ago.

It's actually closer to 30 years ago. I can't believe VMS is celebrating it's thirtieth birthday this year.

http://h71000.www7.hp.com/openvms/25th/index.html

Having multiple versions of a file is *extremely* handy. That feature saved me bacon many-a-time. For those of you who have never been fortunate enough to login to a VMS system, the file versioning looks like this to the user: scott_file.txt;5 scott_file.txt;4 scott_file.txt;3 scott_file.txt;2 and so on The file version incremented each time you modified the file. You could set the number of file versions that the OS would keep for you. I don't remember the maximum number of versions of a file that you could keep but I remember seeing version numbers that were five digits wide. The version number wrapped after a while. Thanks, -Scott

Re:CVS/Subversion replacement ?

[maraist]

What happens to old snapshots when the disk begins to fill up?
Ugh.. you asked the wrong question.. I've had to reread the question a couple times. Your 'old' snapshots should be unmodified, just like when ext3 fills up, you can always read your old data with zero problems.. The issue on ext is when you are out of unallocated disk blocks. Generally means you can't append to a file or create a new non-zero-byte file. But you were still able to mutate existing allocated blocks. The new problem is that after every snapshot, ANY updates of ANY existing blocks require the allocation of a new block. So in ext3, only some applications would fail, writes to stdout or stderr would cause file-IO errors (that are generally piped to log-files). If the application doesn't exit, then they can still continue to operate. But, things like databases generally continued to work, because they usually operate in a virtual file-system that uses pre-allocated disk-blocks. They also use pre-allocated journals. So generally they can guarantee that no single point of failure can corrupt the data (except for hard-drive block errors). But with COW, this assumption is no longer true.. And it's possible that a DB journal implementation can get into an invalid state if it gets a COW-full error.

Thus /var (and probably /tmp) should probably not use COW (or at least you should use crazy monitoring of disk-space like you never needed it before).

Are snapshots atomic?
In Linux, a file-system is a kernel-level driver AFAIK, so I assume it has the power to perform an isolated kernel-level lock, just like a CD-ROM or floppy-disk would. So the real question here is, how long does this central atomic operation take? Or what does it scale based on.. # inodes (benefits partitions with small number of potentially large files), constant-time (by-design defers all delay to individual COW operations), change-set size (benefits rarely changing data like manually edited word-documents and configuration data). Can this DOS the system? (calling snapshot too often effectively locking every file-system operation).

What happens when a snapshot fails? What can cause a snapshot to fail?
Yeah, I second this.. But to call it ext3 AND be included in the kernel, I can only imagine that just like journaling on ext3, it's a pretty isolated, and only a minimalistically invasive feature.

Re:CVS/Subversion replacement ?

[Anonymous Coward]

Never tinkered with any of these filesystems, but wouldnt it be very comfortable for at least us developers to have a filesystem that worked something like Subversion. Just hook up something on the network and use it as the central code repository. You mean like what happens when you enable svn autoversioning and mount your repository as a filesystem?
http://svnbook.red-bean.com/nightly/en/svn.webdav. html

Re:CVS/Subversion replacement ?

[Em Ellel]

Clearcase does have a higher admin overhead than CVS. Clearcase also does not work particularly well over a WAN and I suspect ext3cow or something similar would have the same issue. The use model for a SCM tool based on ext3cow would be similar to NFS which you usually do not use on a WAN. Slightly off-topic, but I would not discount Clearcase on WAN. There are many reasons to hate Clearcase, but lack of distributed/wide area support is not one of them, thats actually one place I've seen it shine. Back in the day(about 10 years back) the company I worked for had part of the dev team in middle of nowhere China. The CC admin set up a satellite CC server in China and since the internet connection was unreliable (could not even reliably FTP a 10k file), he set up server-to-server sync via E-Mail. Warped enough but it worked like a charm, though it took a day or so, but the version control was being synced.

-Em

Re:CVS/Subversion replacement ?

[jgrahn]

Slightly off-topic, but I would not discount Clearcase on WAN. There are many reasons to hate Clearcase, but lack of distributed/wide area support is not one of them, thats actually one place I've seen it shine. Back in the day(about 10 years back) the company I worked for had part of the dev team in middle of nowhere China. The CC admin set up a satellite CC server in China and since the internet connection was unreliable (could not even reliably FTP a 10k file), he set up server-to-server sync via E-Mail. Warped enough but it worked like a charm, though it took a day or so, but the version control was being synced.

I am 95% sure that this setup means "branch A is mastered at site A, and branch B at site B". Meaning people at site A would just see a read-only copy of branch B.

It's distributed alright, but with pretty severe limitations.

Re:CVS/Subversion replacement ?

[Zwaxy]

GNU Emacs offers a similar capability. If I edit file.txt, it renames the previous version to file.txt.~337~ then writes a new file.txt. I have it set to keep just the newest 3 old versions, automatically deleting any older versions.

To achieve this, I have the following in ~/.emacs:

(setq delete-old-versions t kept-new-versions 3 version-control t)

Re:CVS/Subversion replacement ?

[Rufty]

Cool! So what features of a modern OS is emacs still lacking? (Other than a decent text editor?)

Re:CVS/Subversion replacement ?

[Anonymous Coward]

Good one on completely failing to miss the point, which is that one wants a way to make ancient snapshots go away when one runs out of space -- not a lecture on what happens when there are no unallocated blocks are left, as if no one else knew. If wasn't "the wrong question", but an incredibly wrongheaded and stupid answer.

Re:CVS/Subversion replacement ?

[Zwaxy]

Do any modern OSes come with a decent text editor by default?

In Windows, all I get is notepad, right?

I recently installed Ubuntu 7.04, which came with gedit as the default editor. That's not much more capable than notepad.

Re:CVS/Subversion replacement ?

[jZnat]

All Unixes come with vi... ;)

Re:CVS/Subversion replacement ?

[Anonymous Coward]

Being a long time VMS user, the versioning is very handy, but I wouldn't begin to consider it a version control system. With no coordination of versions between files you have no idea what versions of what files make up a particular build, other than maintaining that information for yourself. At least with the Ext3COW, you could get the state of all files at a known time. You'd still need to maintain which times were valid builds, but it wouldn't be as bad as numbered versions a la VMS.

Re:CVS/Subversion replacement ?

[Anonymous Coward]

All Unixes come with vi... ;) Hey, I said "editor" not a roman number.

Re:CVS/Subversion replacement ?

[mcallan]

Although ext3cow has revision history, it probably has no change propagation.
Revision control systems (like CVS or Subversion) must have both.
Change propagation is needed to keep everyone in sync.
So you'd have to add a tool for that.

That's the beauty of a file system, though.
You can accomplish anything just by adding new tools on top.
Instead of marrying tool functions (like revision control) to the file system,
you keep them separate. The big advantage being
that you can choose them independently, and mix and match
(there being many file systems to choose from,
  and many revision control systems).
In this sense, maybe ext3cow is a step in the wrong direction?

"To be sure, ext3cow is not the first versioning file system...
  VMS, TOPS, CedarFS, FFS, Elephant, LFS, CVFS, AndrewFS, WAFL,
  SnapFS, Venti, Spiralog, Plan-9, Wayback, VersionFS..." (FAQ).

To most of us it sounds like a new idea, but it's not.
It just never caught on. I wonder if the reason is, at least partly,
that revision control belongs in a separate tool?

Excellent work but...

[Toffins]

ext3cow looks like excellent work, but being an externally maintained add-on to the kernel, one problem is that it will not be not synchronously available with new kernel releases. The latest available version is 2.6.20.3-ext3cow.patch which is behind the latest kernel. It would be better if it could be accepted and maintained inside the kernel.

Re:Excellent work but...

[oliverthered]

troll
It would be better if the kernel has a stable ABI/API then the patch would work for any 2.6 kernel. /troll

Re:Excellent work but...

[ajs318]

The Linux kernel will never, ever have a stable ABI. Compatibility across versions is guaranteed only at the Source Code level, not the binary level. This is 100% intentional, and the only people it really hurts are those who would deny us access to the Source Code. And they deserve it.

Re:Excellent work but...

[Anonymous Coward]

How about us who don't want to recompile everything whenever a new kernel release comes out? It is a freaking pain in the butt.

Re:Excellent work but...

[oliverthered]

Your wrong, it also hurts those people who write drivers that aren't accepted into the kernel. And it also hurts end users or haven't you noticed the lack of Linux drivers for a lot of hardware.

Re:Excellent work but...

[Toffins]

Compatibility across versions is guaranteed only at the Source Code level

(Disclaimer: Linux is excellent) But is compatibility even guaranteed at source code level?
Here are some specific examples where source level API changes have occurred:

1. Consider that up to linux-2.6.6 all SATA disks were treated as IDE PATA disks accessible via /dev/hd*, but in linux-2.6.7 they started to be treated as SATA disks only accessible via /dev/sd*. This changeover caused existing SATA disk systems to become unbootable after upgrading to linux-2.6.7 because the boot device at /dev/hd* was no longer accessible. Never documented in kernel/Documentation/*

2. And between linux-2.6.15 and linux-2.6.20 the way the usb subsystem handled usb devices was changed so that usermode usb drivers like the usermode speedtouch driver was broken due to kernel returning EINVAL from each USBDEVFS_SUBMITURB command which is required after a USBDEVFS_CONTROL command issued by the modem_run ADSL line monitoring process. This generates thousands of error messages per second via syslogd. No news of this particular aspect of the usb changes was ever documented in kernel/Documentation/*.

Re:Excellent work but...

[ajs318]

How about us who don't want to recompile everything whenever a new kernel release comes out? It is a freaking pain in the butt.

No it isn't. That's a filthy lie made up by people who want to sell you pre-compiled binaries and stop you mucking about with the Source Code, and nobody who can spell 'make clean && make install' believes it. (Or you could use Gentoo, which automates the recompilation; or a distribution using pre-compiled .rpm or .deb binary packages, which will have been recompiled for you by the distro's own team.)

Anyway, not everything will change at one time. You only need to recompile such applications and libraries as actually break.

Re:Excellent work but...

[cyclop]

If someone writes kernel drivers correctly, those drivers will end in the kernel mainline. Linux supports out of the box more hardware than every other OS, no matter how obsolete and obscure. If you don't have your drivers accepted, AFAIK it's a problem with your code not being of enough good quality, nothing else.

Re:Excellent work but...

[smenor]

Perhaps what the GP meant wasn't that recompiling everything isn't difficult so much as slow, tedious, and annoying?

Re:Excellent work but...

[sofar]

It appears that this is the long-term goal for ext3cow. If you look at the formatting of the patch it's obvious that they have worked towards fully integrating their code into the kernel tree from the start (instead of building a separate set of code that compiles outside of it just like -e.g.- the fglrx installer does).

I would assume that their next step is to submit it to Andrew Morton and stage it for later merging into the mainline tree. From how the code looks I can see that that might go quite fast.

Re:Excellent work but...

[diamondsw]

Hi, this is the real world calling. We've been leaving messages for several years as Linux has failed to work on the desktop. We wanted to let you know that we've found the problem, and it's not going to be cheap to fix. Essentially, users want to be able to download and install software or install it from a CD and just have a binary work. "Package management" and dependency hell confuses them and reminds them of DLL's on Win95.

You're going to have to decide if you want every last thing to be GPL and zealously "free", or want it to be accepted by the masses. You will NOT get both.

Re:Excellent work but...

[Anonymous Coward]

Try a userspace application. I still have one from the dawn of time that still works.

Hint: the syscall interface itself is absolutely backwards compatible.

Re:Excellent work but...

[Ant P.]

Hi, this is the real world calling. The last software distributed on CDs that wanted kernel access in the way you're condoning, instead of behaving itself and using system libraries, was the Sony Rootkit.

In summary, fuck off.

Re:Excellent work but...

[644bd346996]

I really don't think recompiling a driver is going to be all that slow compared to the cost of recompiling the kernel as a whole. As for the tedious and annoying, that only applies to drivers that aren't free enough to be included in the kernel tree.

Re:Excellent work but...

[Anonymous Coward]

The Linux kernel will never, ever have a stable ABI ... This is ... intentional, and the ... people it ... hurts ... deserve it. Achtung! Heil! Jawohl!

Do you have any idea how much this sounds like the ranting of a self righteous, pretentious ass?

Re:Excellent work but...

[smenor]

My mistake - you're definitely right.

Somehow I was just picturing taking a few hours rebuilding world - which I haven't done in many years, but it left a very bad taste in my mouth (and - gratuitous Apple plug - that's one of the things I really don't miss since switching to OS X).

Re:Excellent work but...

[snoyberg]

Hi, this is the real world calling. We've been leaving messages for several years as Linux has failed to work on the desktop. We wanted to let you know that we've found the problem, and it's not going to be cheap to fix. Essentially, users want to be able to download and install software or install it from a CD and just have a binary work. "Package management" and dependency hell confuses them and reminds them of DLL's on Win95.

You're going to have to decide if you want every last thing to be GPL and zealously "free", or want it to be accepted by the masses. You will NOT get both.

Please explain to me why opening up Synaptic, typing in the name of the program you want, choosing "install" and clicking "apply" is harder than downloading "myprogram.exe", double-clicking, clicking about three more next buttons and choosing finish. By the way, that's assuming the EXE doesn't demand you agree to licenses or choose where to install stuff.

I really do think the linux approach is more user-friendly.

Re:Excellent work but...

[ajs318]

No, that's what you think you want, because that's what you're used to from Windows and you haven't yet felt the benefits of having binaries compiled by someone other than the author of the package. What you really want is a massive repository with every piece of software ever written in it, and a decent package management system (i.e., anything but RPM). Try Debian or Gentoo. Note that the latter compiles everything from source by default, yet isn't really any the worse for it (compiling most things takes longer than downloading).

Note that even the mighty Debian isn't perfect, because it still has separate "developers'" packages -- they're just called foo-dev as opposed to foo-devel. One day, I'll hack apt to make it automagically fetch a corresponding -dev package if there is one -- unless somebody beats me to it.

Re:Excellent work but...

[makomk]

If someone writes kernel drivers correctly, those drivers will end in the kernel mainline. Linux supports out of the box more hardware than every other OS, no matter how obsolete and obscure. If you don't have your drivers accepted, AFAIK it's a problem with your code not being of enough good quality, nothing else.

Not necessarily. For example, the driver for the USB TV tuner I'm currently using isn't in the kernel and probably never will be. Unfortunately, it's a hybrid analog/DVB-T device with a silicon tuner, and the in-kernel infrastructure is designed around the assumption that devices are either analog or digital and that tuners are simple. Either one can be worked around relatively easily, but handling devices that are both requires either messy code or infrastructure improvements. The author doesn't want to do the former and doesn't have enough pull to achieve the latter, so I'm stuck using an out-of-kernel drivers tree indefinitely.

Re:Excellent work but...

[asninn]

"source code level" refers to the syscall interface between the kernel and userland.

Re:Excellent work but...

[Doug Neal]

A huge number of problems in Windows can be attributed to its lack of package management. Every installer is pretty much allowed to do whatever it wants, put files where it wants, change registry keys, whatever.. and when was the last time you saw a Windows program with an uninstaller that worked? I mean really worked? They all leave crap lying around afterwards that they "couldn't" remove for some vague/unspecified reason. Sometimes you don't even get an uninstaller at all. There's no version tracking, and no management of dependencies. Everything just has to ship with all the libraries it needs and hope it doesn't break anything else, which it doesn't always manage. You end up with a total mess, even if you're careful. Your average Windows PC needs reinstalling once a year at least to stay usable, on almost all of the occasions that someone's asked me to check out a few problems on their computer, I've ended up reinstalling just because it would be quicker than to clear up the mess.

The "dependency hell" that you speak of has been a non-issue for years, even Red Hat makes a passable stab at it these days.. there's plenty of issues stopping Linux becoming a mainstream desktop OS but package management isn't one of them. Users don't want to have to run installers from CDs or whatever as you described, it's just what they're used to doing at the moment. If you showed a complete computer novice Synaptic or Click N Run, and then showed them the equivalent in Windows, which do you think they'd prefer?

Re:Excellent work but...

[marcello_dl]

Your distro's package management compared to DLLs? Let us know whi

On my debian and ubuntu boxes i type "aptitude install program" and i have a securely-signed program installed after the minimum possible time and download size (thanks to dependencies), and it gets automatically checked for updates. For each package there is the source if i want to recompile it. If i have no time on gentoo, IIRC, i can install it as precompiled binary.

In the sadly real windows world i have to separately download big static blobs or even separate libraries (See gtk for gimp) from the dev site (or a rootkit from the latest gimp3forwindows.dyndns.org). No automatic updates, if the app provides one it will have a popup competing with all the others and a slightly different interface. Also the firewall will complain, and the antivirus will complain. and windows will complain if the vendor didn't pay them for a certification. And, dunno about vista but xp installers and uninstallers can be 20x slower than aptitude.

Re:Excellent work but...

[Nutria]

Perhaps what the GP meant wasn't that recompiling everything isn't difficult so much as slow, tedious, and annoying?

Sure, if you use Gentoo.

Otherwise, you remember that this isn't 2002, and that CPUs are fast enough and the kernel does a good enough job at scheduling the CPU that a kernel build only affects you if you're trying to watch a movie using the nv or vesa driver. Otherwise, you kick off the build and go to sleep, eat dinner, watch some TV, etc, etc, etc.

kick ass

[Anonymous Coward]

this is brilliant if it works reliably with minimal overhead.

lets hope it gets picked up by the major distros

I could really use this - can I have a nautilus add on for it?

True undelete

[ex-geek]

Undelete, not half-assed, desktop based trash can implementations, is something I've always been missing on Linux. And yes, I generally know what I'm doing, but i'm also human and do make mistakes.

Re:True undelete

[Anonymous Coward]

Could always just hardlink everything in certain directory trees to a backup dir that is swept periodically. Essentially just "tagging" not actually erasing.
We used that a lot on our servers to avoid accidents.
Does not protect against overwriting contents of course, and doesn't do revisions - but hey, you said undelete - not all the features of this tool.

Oh, and that has advantage of not taking up any extra disc space (apart from the tiny bit for the hardlinks and their tree).

Re:True undelete

[19thNervousBreakdown]

I've always wondered about this. Aren't files always eventually deleted with an unlink() call? What reason is there that unlink() can't be modified to instead move the link to a .Trash/ which is then scrounged when more space is needed? You could either auto-delete the oldest files, or if you wanted to not affect FS fragmentation delete a file whenever you needed to clobber one of its sectors. Sure, performance will drop when you get a drive full of deleted files that have to be cleared every time you write, but it wouldn't be that bad, and it would only really be useful on /etc /usr/share /home and maybe /var. It doesn't even seem like you'd need to modify the underlying FS, except maybe for the sector clobbering part, but I don't know the kernel internals well enough to say for sure.

Any kernel/FS hackers know this one? There's probably a real good reason that I'm missing, if not I've been thinking for a while that it'd be a good place to start my kernel hacking.

Re:True undelete

[ex-geek]

guess what, if you had enough energy to type www.google.com the first 8 links are all great projects and ways to do exactly what you said for the keywords "linux undelete"

but then, typing those letter into a web browser is simply way to much effort.

These options went out of the window with the introduction of journaling in ext3. But even with ext2, they barely worked, especially for large files. They didn't work for me anyway.

you must be either management or incredibly lazy.

I guess you are the 18-year-old intern who believes to know everything then?

Re:True undelete

[xenocide2]

There's a couple reasons for it not being in the kernel. First, it misleads users who expect some degree of data security. The good news is that sort of person likely follows kernel patches to the FS and would likely be aware of the problem, possibly even writing a script that replaces rm with a real-rm.

The second argument is that it's better handled in user space, so the OS doesn't have to make that sort of policy. There's no reason you can't just alias rm to some .Trash, or configure your Desktop Environment to do so (GNOME does, for example). There's all sorts of things you have to decide that might not suit everyone. For example, if I delete a file on a USB drive, does it go in a .Trash storage in the USB drive, or do we copy it over to a main .Trash folder? Many people don't realize they have to empty the trash to reclaim space on their thumbdrive in GNOME.

The final argument I can come up with is security problems. We can't have one global .Trash bin in a multiuser system. And quotas. And permissions.

Reading historic archives of the LKML suggests it's at least come up once. I guess Torvald's opinion is that anything that CAN go in the userspace SHOULD. Can't explain the webserver in kernel though. Perhaps that opinion has changed some time in the last 10 years?

Re:True undelete

[rahimobius]

This sound very much like the functionality of the Recycle Bin in windows. Sometimes I wish I had a recycle bin as well. (I have a nag for typeing rm when I mean mv). But I agree with xenocide2 that this is something for user space. It certainly is not trivial to implement if you want a secure multi-user environment. But for single-user developer-systems a aliased/hacked rm and a deamon could be of great help when you accidentally rm two files instead of replacing one with the other ;)

Re:True undelete

[jonadab]

Undelete isn't what makes this really cool, IMO. I don't generally delete stuff I still want, so that isn't really a big issue.

What I want, that a versioning filesystem can deliver, is the ability to revert a file back to an earlier version, after I've saved changes that turn out to be undesirable. This is a mistake I *do* make from time to time, often enough that I have been really hoping for a versioning filesystem in modern operating systems. This, to me, is a killer feature. I'm currently using FreeBSD, but this feature would by itself be enough to bring me back to using a Linux distribution, once it gets to the point of being included. Without it, once you save your changes and exit the application you can't go back. The past is lost. With a versioning filesystem, that's no longer true. I consider this to be *THE* feature for filesystems, far more important than things like journaling, much less performance tweaks. I have been wanting it ever since I saw the automatic versioning on OpenVMS, and I've been waiting, waiting, hoping, wondering why we don't have it in modern operating systems. I *want* this.

Re:True undelete

[TheNetAvenger]

and I've been waiting, waiting, hoping, wondering why we don't have it in modern operating systems. I *want* this

Look up Windows 2003 Server, WindowsXP, Vista...

Re:True undelete

[gmack]

Undelete in windows is also desktop based. Ever notice that uninstallers don't delete to the "Recycle bin"? You can also try opening a cmd window and deleting something with del and notice that it does not appear in the "recycle bin"

Re:True undelete

[Gordonjcp]

I have been wanting it ever since I saw the automatic versioning on OpenVMS, and I've been waiting, waiting, hoping, wondering why we don't have it in modern operating systems.

Someone I know has the email signature "DIGITAL had it *then*. Don't you wish you could buy it *now*?"

Re:True undelete

[Cyberax]

Kernel-level webserver has real performance benefits. And it is not enabled by default.

Re:True undelete

[aaarrrgggh]

I still miss the "salvage" from Netware-- the ability to restore any revision to a file as disk space permits. Just hacking rm doesn't fix someone overwriting a file.

As for security, you could disable salvage for sensitive volumes or directories, or have firm policy based wipes of deleted files on a scheduled basis. Often times, Salvage was most useful when a problem was discovered 20 minutes after it occurred.

It sounds like ZFS will do a better job of allowing snapshots to support something like this, but I'm disappointed it isn't in the kernel now.

Re:True undelete

[Ant P.]

Can't explain the webserver in kernel though.

The Tux server has never been a part of the official tree. What's there to explain?

Re:True undelete

[FoogyFoo]

I actually did a class project on this in grad school. So did these guys. They did it for ext2, I did it for ext3.

So yes, it is possible. It can be done a couple of different ways:

• move any deleted stuff to /trash, possibly replicating the directory structure underneath
• move any deleted files to ./.trash

The first has the advantage of confining all deleted items to one location, but has the disadvantage of possibly crossing filesystem boundaries, causing a world of headache.
The second is simpler, keeps deleted items close to their original location, and doesn't need to worry about filesystem boundaries.

I went with the first approach; it was a pain to implement, but yes, it worked. I think the guys I linked to did the second approach.

Filesystem drivers can be a dangerous place to do kernel hacking. Especially if it's the filesystem you boot from (then you need to build your own initial ramdisk - I used yaird). So go for it if you want to, but be prepared to endure some truly funky things in your filesystem (reboots fix most of the problems I saw. except when it kernel panicked on boot).

Re:True undelete

[fireboy1919]

You mean, for example, remapping the unlink calls to libc to actually move things to ~/Trash?

Surely you wouldn't want that on all accounts. Only for users. It'd be chaos if every single script on your computer that generated temp files had them moved rather than deleted.
How about putting it into the .bashrc (or .zshrc, or whatever) file to be loaded using the preload trick?

That way, all users that have that .bashrc file can have it on, and everything else won't.

There is a library for this.

something I've always been missing on Linux
It works like a charm. I've been using it for years. It's available as a package in most distros, and has been for quite a long time.

I should also note that it can be implemented systemwide - so that *everything* is used that way. But that's a good way to kill your system. Your computer sometimes needs to delete things.

Re:True undelete

[steveha]

I disagree with you. I want true undelete on my desktop.

First, it misleads users who expect some degree of data security.

Yes, true undelete is the enemy of true delete, and when you have a document with sensitive info in it, you want true delete. That is not a good reason to refuse to implement true undelete; it just means we need a way to control the file versioning on a file-by-file basis.

The second argument is that it's better handled in user space, so the OS doesn't have to make that sort of policy. There's no reason you can't just alias rm to some .Trash

Not enough. Suppose the user overwrites one file with another, by mistake. One common way this can happen:

$ cp old_version new_version # oops, wanted cp new_version old_version

Or the classic mistake of editing a file, and writing it out in the wrong place (clobbering a backup version you actually intended to keep, or whatever).

The final argument I can come up with is security problems. We can't have one global .Trash bin in a multiuser system. And quotas. And permissions.

True undelete should ideally be controlled by some flag on the file. If you create a file, the default flag should be inherited from the directory in which you create the file. We even have the flag already; if you are a Linux user, run "man chattr" and look up the "undeletable" flag (-u).

I guess Torvald's opinion is that anything that CAN go in the userspace SHOULD.

The tools for viewing your pool of deleted files, requesting an undelete, etc. should all be user-space tools. But you really can't do true undelete without file system support.

steveha

Re:True undelete

[Otterley]

ZFS cannot become a part of Linux unless/until Sun releases it under a GPL-friendly license.

Re:True undelete

[roystgnr]

There's a couple reasons for it not being in the kernel. First, it misleads users who expect some degree of data security.

What security? When you use "rm", that doesn't overwrite your file on disk, it just unlinks it. Anyone with root access can read your recently deleted files from /dev/[sh]d? if you didn't overwrite them before deleting, and making data theft less convenient for someone who has already rooted your system isn't worth making data recovery less convenient for you. And from the other direction, anyone without root access wouldn't be able to read your .Trash files any more than they can read your /tmp files; you say "We can't have one global .Trash bin in a multiuser system", but that's a solved problem.

For example, if I delete a file on a USB drive, does it go in a .Trash storage in the USB drive, or do we copy it over to a main .Trash folder?

On the USB drive. When one answer to your question has the downside of "takes a microsecond to relocate a hard link" and the other has the downside of "takes an arbitrarily large time to move a file", it's probably safe to make the former answer global policy.

Many people don't realize they have to empty the trash to reclaim space on their thumbdrive in GNOME.

That's because Gnome's trash implementation is in userspace, and so the obvious behavior of "automatically delete the oldest things in the .Trash directory whenever you need new space" isn't an option for them. It would be an option for the kernel. It's a fine philosophy to move policy into userspace whenever possible, but some policies just aren't possible in userspace. If you dug into glibc you could probably implement "automatically delete *my* oldest trash whenever I need new space", but you'd probably need some funky ACLs in .Trash to allow people to delete each other's deleted files but not to read them (or even to read their filenames).

Re:True undelete

[Deagol]

Isn't zfs slated to appear in FreeBSD 7.0? Is the snapshot ability of zfs close enough?

I hear you, though. I'm a die-hard FreeBSD user, who migrated from Linux a few years back. The ext3cow announcement had me questioning my own loyalties, too. :)

After reading one of the papers, it looks like you need to actually take a snapshot, though. I was hoping for an more automatic version system, as with the VMS versioning stuff.

Re:True undelete

[Fweeky]

Back in the days of the Amiga, I kept a hefty portion of my data on SFS. When you deleted a file, SFS moved it to a magic .recycler directory (or something like that), and only actually cleared the file when the blocks were reused. If you wanted the file back, you just copied it out of the recycler.

You could do something like this without going anywhere near the kernel -- just LD_PRELOAD or otherwise blat the function and replace it with your own. However, personally I think it's a shit solution -- half the data loss I've experienced hasn't had anything to do with unlink -- write() is just as capable of ruining your day, and without COW you're basically screwed without backups (which are never fine-grained and regular enough).

Re:True undelete

[ckaminski]

XP doesn't have this feature unless you're using Windows Server 2003 with Volume Shadow Copy enabled.

The answer for Windows is TortoiseSVN.

Re:True undelete

[jonadab]

> After reading one of the papers, it looks like you need to actually take a snapshot, though. I was hoping
> for an more automatic version system, as with the VMS versioning stuff.

Oh, bummer. I, too, was wanting it to be a bit more automatic, like what's in VMS. I wouldn't mind if I had to expressly turn it on once for any given directory, but once I turn it on I'd like it to automatically version those files whenever they change in perpetuity.

Re:True undelete

[jonadab]

> Look up Windows 2003 Server, WindowsXP, Vista...

Wait, NTFS has filesystem versioning? Really?

[Looks it up.]

Huh. Volume Shadow Copy Service does sound rather similar, on the face of it (except that some of the information about it on the web seems to indicate that it only versions the whole filesystem (rather than individual files) which would be a bit of a pain, but possibly not a deal killer, depending how well it's implemented).

Microsoft sure has not done a very thorough job of *advertising* such an important and useful feature, though. This is the first I've heard of it.

That's almost enough to make me want a Windows-based CIFS fileserver on my home network. I could use rdesktop to administer it, and apart from that I'd hardly have to deal with the Windows UI at all... Hmmm. The hardware requirements and the license fee are the real killers on that for me right now, given my budget constraints. Still, it's an interesting possibility to ponder.

The other possibility would be to get a used Alpha system on eBay, a hobbyist license from Encompass or whatever they're calling it these days, and set up Samba for VMS. That would probably be a bit of a learning experience, getting it set up, but the automatic versioning implementation and interface in VMS are really slick. But this option would probably cost rather more, as used AXP systems go for a bit more money than used PCs. Plus, I have to admit, I would be less confident in my ability to set it up.

On the gripping hand, the Windows system would need more regular maintenance. VMS is old as the hills and stable as bedrock -- once you manage to get it set up (which, admittedly, takes more work than with more modern systems), you never have to touch anything. (If you train a monkey to change the backup tapes, a VMS system can go for years without even *having* a system administrator. It's a whole different universe from the Windows world of needing to log in as administrator a couple times a week just to check on things.)

Eh, it's all moot. I can't afford a fileserver right now, so which technology I'd use is immaterial. With ext3cow drawing attention to this issue finally, maybe by the time I get my budget whacked into shape there'll be versioned filesystems included in all the major BSD and Linux distros, and I'll be Bob's nephew.

Hooray for repeating history!

[Anonymous Coward]

VMS had a versioning filesystem 20+ years ago. You'd create a file and any time you'd edit it a number would be appended as the version:

eg foo.txt;1 foo.txt;2

Re:Hooray for repeating history!

[Anonymous Coward]

So because it was a good idea 20 years ago, it somehow isn't good that it's been implemented now? Sure, in an ideal world we'd all have been using versioned filesystems since the advent of VMS, but we havn't.

Actually a tell a lie; the ISO9660 spec. copies the VMS design and also allows files to have a version number, using the exact same scheme I.e. the version # is appended to the file following a semi-colon. So "FOO.BAR;1" is a valid ISO9660 filename.

Re:Hooray for repeating history!

[ajs318]

This is one of the things I missed when I moved from VMS on a VAX 11/750 {or it might have been a 780?} to MS-DOS on a '286. The commands were kind of similar between the two OSes, though DOS didn't have EVE -- which for me was the killer app. Version numbers, EVE, case-insensitivity and commands that were not "telegraphese" {there not being such a word as "txtspk" in those days when mobile phones were analogue, half-duplex [you squoze a switch in the handset when you wanted to speak, and let go to listen to what the other person was saying] devices operating on VHF and where you had to know roughly where the receiver was, as each transmitter mast had its own STD code} were among the reasons why I preferred VMS to Unix.

Great days. Of course, Unix won in the end ..... it does that because after using it for awhile, the things you hated it for at first stop bothering you, and eventually start making sense. Maybe it's got cat powers!

Re:Hooray for repeating history!

[TractorBarry]

And Fujitsus (ex ICL) VME (Virtual Machine Environment) also has generation numbers. And they're a brilliant idea.

e.g. Access a file by name eg. "OPEN_FILE(FOO)" and that will open the highest generation (latest version) of the file. Want to access an old version ? Simply specifiy something like "OPEN_FILE(FOO(23))". Obviously there are tools for tidying up old generations etc.

And when you edit a file it is always saved to a new, higher, generation so you can always go back to previous version after you;ve made a pigs ear of your edit :)

Operating systems that only allow one copy of a file to exist at any one time are simply archaic.

Re:Hooray for repeating history!

[Anonymous Coward]

Tops10 and Tops20 used version numbers before VMS existed (RSX - which VMS is derived from didn't). On these systems the file name is stored in 3 parts (on unix this is a single field) that consisted of the filename, file extension and the version number. On Tops20 a "." was used to separate all three items and on VMS a "." to separate the filename from the extension and a ";" to separate the extension from the version (later releases of VMS let you use the "." in addition to ";"). In the file system the version number was "word" (2 bytes) field.

In all these OSes you could specify files with time/date modifiers and or relative to the current or most recent version. E.G. if you wanted the second most recent version you could do a "DIR *.*;-1" and you would see the version 1 back.

The ODS file system wasn't the most efficient, but the user interface was certainly nicer and more
consistent then what they're proposing. Of course, this is being retrofitted (kludged) on existing systems and, as such, isn't the worst I've seen.

So

[El Lobo]

Is this like MS Shadow Copies or like Apple's Time Machine? Not trolling but just somebody enlight me, what is new here?

Re:So

[psbrogna]

I don't think it's supposed to be new (it's one of the things I miss most about VMS). It's outstanding functionality to have both for end users and sysgeeks/devs; built right into the file system level (ie. LOW). I prefer this approach to the hacks that other O/S's have implemented at a higher level. It's always harded to do something like this down deep at the roots rather than add it on as superficial gloss later. Granted, the end users don't usually notice or appreciate the diff but over time it keeps complex sys's like O/S's from becoming a teetering tower of shims and bolted on widgets.

Re:So

[TodMinuit]

It's more like Plan 9's Fossil, only without the extremely cool Venti.

Re:So

[delire]

Not trolling but just somebody enlight me, what is new here?

It is for Linux. That is what is new. The two examples you give are for other operating systems. Raising your eyes to the top of the page will reveal this article is in the section "Linux". It's a bit tricky I know.

Psst: it's not a race.

Re:So

[init100]

It is for Linux. That is what is new.

Actually, snapshots with copy-on-write functionality is not new in Linux, but it hasn't been available in the filesystem itself. The Logical Volume Manager is able to create and use COW snapshots, and has been for some time.

Re:So

[spitzak]

There are already versions of this on Linux. One I use at work does it instead by making a new directory with the older versions of the files in it. To see yesterdays version of ./foo.txt you look at ./.snapshots/yesterday/foo.txt. This seems a lot nicer as you can more easily see when files are created and deleted as well as modified, though it is possible that technical limitations prevented this from using this naming scheme. I not sure what the system is, as it is a large file server, it is running Linux, though the filesystem is possibly proprietary.

I think the big deal is that this is a modification of the ext3 filesystem, rather than an all-new filesystem.

Re:So

[delire]

Actually, snapshots with copy-on-write functionality is not new in Linux, but it hasn't been available in the filesystem itself.

Precisely.

Re:So

[samkass]

Apple's Time Machine isn't just a *file* backup system. It's a *record* recovery system. Neither MS Shadow Copies nor this provides an API for software to search records back through time and pull a single record back to the present (ie. a single address book entry or photo). It's frustrating having people equate them so closely when it misses half the point of Time Machine.

Re:So

[osu-neko]

I think the big deal is this: I suspect that the system you're using at work works by keeping older versions of each modified file, so if I modify 1 byte in a 1MB file, the filesystem uses 1MB for the current version and 1MB for the version inside .snapshots/yesterday/, whereas if this was implemented on the filesystem level, it would just use one (most likely 4K) block.

Re:So

[ckaminski]

Care to elaborate on said system at your work?

Re:So

[spitzak]

I think it may be Network Appliances.

Well, congratulations.

[jimicus]

Well done to all who worked on this patch. Guess this means you've almost caught up with OpenVMS now, then? [throws another log of karma on the fire].

All joking aside, I never really liked VMS much. It was extremely good at being very verbose whilst being extremely bad at clear English.

Re:Well, congratulations.

[MichaelSmith]

Well done to all who worked on this patch. Guess this means you've almost caught up with OpenVMS now, then?

In the sense that you had multiple versions of every file? Well yeah but it is on a per file basis rather than a per volume basis so you can't ask it to give you the entire volume (or even a directory) as it was at a particular time.

And I remember being caught by the 32000 version number limit, with a batch job which maintained a status file and purged the file after every run. The version number still incremented and when it hit the ceiling the job started crashing.

Re:Well, congratulations.

[jonadab]

Actually, I rather liked VMS. Besides the filesystem versioning, which was extremely cool, one of the other things I liked about it was the help system, which was browseable in a way that made it possible to find commands when you didn't know what they were called. The Unix man page system just can't deliver that. These days, of course, you just do a web search and find the information you need, so it's less important, but still, the design of the VMS help system was interesting and, IMO, good.

I didn't like everything about VMS, of course. You'll note that I'm not currently using it, or anything. Still, there were definitely some good things about it, and the filesystem versioning was one of the best. I am more than a little pleased to see other systems finally catch on and get this feature.

VMS file versions someone?

[ntufar]

It reminds me of VMS file versions.

In VMS if you had a file named article.txt, each time you modified and saved it in editor, a new version was created named article.txt;1 article.txt;2 article.txt;3 and so forth. So after a long session of edit and saves you could end up with a hundred copies of file in your directory. A lot of clutter in the directory but easy access to older versions of the files.

With Ext2cow you basically get the same functionality in a bit different way. By default you see only article.txt file. If you need to access a previous version of the file you need to specify a cryptic code like this: article.txt@10233745. A bit cumbersome but, hey, how often you access older version of your file anyways. Looks better than VMS' approach.

This filesystem seems like a perfect solution for me as I am writing my Ph.D thesis. Currently I take backup every day and name it thesis20070420.tar.bz2, thesis200070421.tar.bz2, thesis20070422.tar.bz2 and so forth in case I need to go back and see how it looked some time ago.

However, in my home directory I have a lot of large audio and video files that I would never want to be versioned. I wander if Ext3cow keeps extra copies of the files if I move them around, change file named but do not modify the content. Probably I would have to make a new partition and put my text files I am working on there under Ext3cow and leave my media files on ext3.

Re:VMS file versions someone?

[JohnFluxx]

Why don't you use svn?

Re:VMS file versions someone?

[Anonymous Coward]

This filesystem seems like a perfect solution for me as I am writing my Ph.D thesis. Currently I take backup every day and name it thesis20070420.tar.bz2, thesis200070421.tar.bz2, thesis20070422.tar.bz2 and so forth in case I need to go back and see how it looked some time ago.

Sounds like a job for version control (e.g. CVS or Subversion)

Re:VMS file versions someone?

[ntufar]

Why don't you use svn?

Frankly, using SVN would be just too much effort for me: I may forget to commit the changes after a day of work; the files are binary .odt files; I need to teach my wife to use it.

I would rather have a Ext3cow to sit there and so everything for me in background. But, you gave me a good idea. I never thought of using SVN for anything besides source files...

Re:VMS file versions someone?

[GauteL]

"If you need to access a previous version of the file you need to specify a cryptic code like this: article.txt@10233745. A bit cumbersome but, hey, how often you access older version of your file anyways. Looks better than VMS' approach."

This is exactly what a graphical file manager should abstract away through concepts such as time machine.

This announcement is just Linux file systems starting to catch up with features from file systems such as ZFS. Very good news.

Re:VMS file versions someone?

[arivanov]

Not quite.

This is more like NetApp and other high-end NAS and SAN systems where a facility like this is used for backup. The backup system looks at a snapshot taken at X:00 and backs it up at leisure while the users continue to read/write to the filesystem on top of it. Once the backup is complete you obsolete the checkpoint on which the backup was operating. As a result you have a true backup of the filesystem at point X, not something that spread from X to X+N hours.

This is a killer feature as far as any enterprise is concerned and one of the main reasons people at one point start looking at SANs and NAS-es.

Re:VMS file versions someone?

[Yaztromo]

Frankly, using SVN would be just too much effort for me: I may forget to commit the changes after a day of work; the files are binary .odt files; I need to teach my wife to use it.

1. Why not just extract your ODF file before committing? Other than graphic figures it's all text data inside a ZIP wrapper.
2. Why is your wife working on your thesis?
3. Why would you be any more likely to forget to run "svn commit" than you would be to tar your files up every day? And if you're likely to forget either, why not just do it as a cron job to run daily?

Yaz.

Re:VMS file versions someone?

[cortana]

Interesting... but tracking the revisions of a file by name has some limitations. What happens if I rename a file (also to another directory)? What happens if I rename a directory itself? Is the file metadata (owner, access permissions, modification times, extended attributes (including selinux labels, ACLs and user extended attributes)) versioned?

I guess some of this info is on the project's home page, which is down at the moment...

Re:VMS file versions someone?

[karmatic]

So mount SVN over webdav, and turn on auto-versioning. Whenever you make a change, it gets committed as a new revision.

Re:VMS file versions someone?

[porl]

wait, your wife is writing your thesis now?? :)

Re:VMS file versions someone?

[osgeek]

Or better yet, SVK.

Re:VMS file versions someone?

[jonadab]

Funny you should bring that up. Up to now I still keep nearly all of my data (both text and media) on FAT32 filesystems. Among other things, this makes it easier to switch operating systems. I'm currently using FreeBSD, but if something goes wrong with my OS, I can pop in a Knoppix CD and access my data no problem, or take the drive upstairs and stick it in the Windows 98 computer that my family uses and access my data, or... whatever.

Automatic versioning, however, is enough of a killer feature (one I've been wanting for a long time, ever since I saw it in VMS) that if it's decently well implemented I would be willing to give up the extra portability and lock myself in to a single operating system. I will certainly be looking forward to trying it out, once the distributions start picking it up and including it.

Re:VMS file versions someone?

[balor]

If you're doing your Thesis in LaTeX it makes a _lot_ of sense to use Subversion or Bazaar, instead of backing up like that. Then you can script a svn dump of the whole repository daily.

Just my 0.2p from someone going through the same thing.

Re:VMS file versions someone?

[TheNetAvenger]

This is exactly what a graphical file manager should abstract away through concepts such as time machine [apple.com].


I know this is SlashDot, but why reference a non-shipping product as the GUI standard example for this feature when it has been being used in Windows 2003 and WindowsXP for over 4 years?

Vista even goes a few steps beyond previous Windows versions and Time Machine.

PS from the last Beta I played with, Time Machine's UI has a ways to go to catch up to the simplicity of right click -> previous versions in Windows.

Re:VMS file versions someone?

[rbanffy]

If I got it correctly, you would only have a new copy of the directory when you rename or move the file. The file will only be copied if it is changed.

And it is only necessary if you are doing it based on files. If you do it based on blocks, then only the blocks that were changed get copied.

It seems quite cool. Too bad all servers even remotely related to it appear to have been slashdotted.

Re:VMS file versions someone?

[caseih]

ext3 and ext3cow are inode file systems. So if you rename the file or move it anywhere on the disk, the inodes allocated to the file stay the same. With ext3cow, the inodes that make up the versions would stay the same too.

Re:VMS file versions someone?

[radarsat1]

Not to repeat too much what other people have said, but I _highly_ recommend using subversion to keep track of your school work. I am currently writing my Master's thesis, and I'm using subversion to track my .lyx and .tex files. Not only does it allow me to go back to old versions, but it helps me to keep everything synchronized when I switch from working on my laptop to my desktop or to the computer in my lab. Version control is not only for code, it is for _source files_. That includes anything that is not auto-generated, including text documents.

Re:VMS file versions someone?

[physicsnick]

Hmm, when I read your post I thought I'd come here and suggest Subversion. Seems everyone else has done the same.

You really should use it. It's much easier to set up than you'd think, especially if you're on a Debian/Ubuntu box. If you use the file:/// syntax, you don't even need any kind of daemon or http server running; the client can do everything on its own. Say your thesis is currently sitting in ~/thesis, it's this easy to set up:

sudo apt-get install subversion
svnadmin create ~/thesisrepo
svn import ~/thesis file:///home/${USER}/thesisrepo -m "Initial import"
mv thesis thesisbackup
svn co file:///home/${USER}/thesisrepo thesis

That's it, you're done. ~/thesis is now a working copy of your repository, the repository itself (which will hold all versions of your files) is contained in ~/thesisrepo, and your original folder is backed up as ~/thesisbackup.

To work on your thesis, go into ~/thesis and start writing as you've always done. When you want to save a snapshot of the current state of your thesis (i.e. commit your changes), open a bash terminal, go into ~/thesis and type svn ci -m "some message". That's it. Much easier than running a backup; you can just stick it in a daily (even hourly) cron job. To back up all versions of the thesis on removable media, tar up the ~/thesisrepo folder and put it somewhere safe.

There's a bit more to know about it; namely you need to tell subversion when you add, remove, move or rename files. A good source for that is the Subversion Book, specifically Chapter 2.

Re:VMS file versions someone?

[Christian Engstrom]

In VMS if you had a file named article.txt, each time you modified and saved it in editor, a new version was created named article.txt;1 article.txt;2 article.txt;3 and so forth. So after a long session of edit and saves you could end up with a hundred copies of file in your directory. A lot of clutter in the directory but easy access to older versions of the files.

And to get rid of the clutter, you just used the PURGE command. I really loved this aspect of VMS. :)

At glindra.org, I have published a set of open source command line utilities for Windows and Linux, that implement the VMS commands dir, copy, rename, delete, and purge.

All the programs support VMS-style file version numbers (in the same way that Emacs does). Other aspects, such as option names that can be truncated as long as they are unique, and suboptions within parentheses, are also inspired by VMS.

The utilities are not an attempt to emulate VMS exactly, however, but an independent design that is based on principles that I happened to like when I came across them under VMS.

Please feel free to have a look at them if you are interested.

Re:VMS file versions someone?

[Anonymous Coward]

Yeah, my thesis is backed up through SVN, and this is really nice because I can sync between multiple computers, such as my laptop at home, my office computer, and other random places where I'm running portions of my thesis, such as pyhton codes on various computers around my lab. It's nice to update them all to the current version. Can't do that through a single-system versioning system, such as Ext3cow.

Re:VMS file versions someone?

[Nevyn]

This announcement is just Linux file systems starting to catch up with features from file systems such as ZFS. Very good news.

AIUI ZFS just has entire FS snapshotting, although it's read+write ala. LVM2 (came with RHEL4). So no, this would be different (although ZFS proponents would probably argue that it's much easier to create an FS in ZFS, so it's not quite as big a leap).

Re:VMS file versions someone?

[Anonymous Coward]

In VMS if you had a file named article.txt, each time you modified and saved it in editor, a new version was created named article.txt;1 article.txt;2 article.txt;3 and so forth. So after a long session of edit and saves you could end up with a hundred copies of file in your directory. A lot of clutter in the directory but easy access to older versions of the files.

I also thought about VMS (age shows up)... I had it configured then to keep only a certain number of older versions and it was quite useful. For a Linux implementation you could prepend '.' to backup copies to make them hidden from a usual "ls" command: .article.txt.1 When you need to see backup copies "ls -a" will show them too.

It looks like a simple and useful solution. I wonder why something like that was not done in Linux long time ago. Or was it?

Re:VMS file versions someone?

[sofla]

It reminds me of VMS file versions.

Me too. But I'm pretty sure the VMS versioning was just a convenience feature, and that the versions were independent files - not implemented in the filesystem and not implemented using copy-on-write. I vaguely recall being able to rename a ;2 file to something like "name_SAVE" so that I didn't lose that particular version when it purged. But it has been a long time since I've touched VMS.

Re:VMS file versions someone?

[file-exists-p]

You would put your phd thesis in the hands of a filesystem which has not been extensively used by thousands of users for years ? I would not.

I would use CVS and make a backup of the cvsroot everyday on a CD. But I am a bit paranoid.

Re:VMS file versions someone?

[DamnStupidElf]

However, in my home directory I have a lot of large audio and video files that I would never want to be versioned. I wander if Ext3cow keeps extra copies of the files if I move them around, change file named but do not modify the content. Probably I would have to make a new partition and put my text files I am working on there under Ext3cow and leave my media files on ext3.

Since ext3cow uses copy on write (or more realistically, metadata magic to emulate copy on write without the actual copy), unless you are routinely modifying your media files they will only take one file version and no extra space. On the other hand, when you accidentally do rm *mp* without thinking, you can just go back to the old version of the files and restore them. Moving files is just a matter of changing which directory contains a pointer to the file inode (think in terms of creating a new hardlink to the file in the directory you're moving the file to, and then deleting the hardlink in the original directory: no data movement required).

I haven't gotten to the part in the ACM article describing tools to automatically manage the old snapshots, but some sort of exponential decay is probably the most useful: Keep all versions less than 1 day old, then only keep one version between 2 days and one week old, only keep one version between 1 week and 1 month old, etc. That way you have the original version, and then a yearly, monthly, weekly, and daily version of any given file, with just the differential data between each version actually stored to disk. For manipulating lots of files, or very large files, it would be useful to have per directory policies as well (e.g. /tmp probably does not need any versioning, and the bin and lib directories might only need one old version at most). When I get home, I am definitely compiling 2.6.20.3 with this patch on my laptop.

Re:VMS file versions someone?

[Deagol]

I have a cron job that runs svn against /etc and /usr/local/etc once an hour. That way I can recover if I totally hose a working config file. Works pretty well.

Re:VMS file versions someone?

[Anonymous Coward]

Very cool! Thank you! I use subversion (w/ server) for source code, but I had never realized how trivially easy it is to set up a repository with the file:/// syntax.

The C in CVS.

[SharpFang]

Concurrent...

Sure you can "go back in time", but two users working on the same file at the same time would be a pain. Networking would require additional layers - even plain SAMBA/NFS, but still. Plus a bunch of userspace utilities as UI to access it easily.

It's not bad as a backend for such a system, just like MySQL is good as a backend for a website, but by itself it's pretty much worthless.

Interesting sponsor for ext3cow

[Anonymous Coward]

The ext3cow project sponsor SecurityEvaluators is a rather interesting company in terms of some of their funding arrangements (sorry, cannot publish details here).
(2006) FBI Head Wants Strong Data Retention Rules
(2005) EU Approves Data Retention

Re:Interesting sponsor for ext3cow

[Panaflex]

Are they funded by the BoogeyMan? Perhaps Santa Clause? I think it's a bit of a "diss" to use a couple of articles to paint Z's project... it's clearly stated on the Developers page:

"This work was, in part, supported by the National Science Foundation under Grant #0238305 and by the Department of Energy."

Of course ISE has numerous clients and sponsers as well. Just check their webpages.

Re:Interesting sponsor for ext3cow

[Anonymous Coward]

it's clearly stated on the Developers page: "This work was, in part, supported by the National Science Foundation ..."

Irrelevant. I clearly said I was referring to the sponsor of the project, not the project itself or its developers.

Of course ISE has numerous clients and sponsers as well. Just check their webpages.

Yes, they do, and, no, there is no website that I am aware of with details of some interesting aspects of their funding arrangements, and no "dissing" is intended but as I said before, I will not be giving any further pointers here - you'll just have to do your own research off-line (Googling will not help).

Smells like dirvish

[Zekat]

This sounds like http://www.dirvish.org/, which is nearly as nice as the automatic file snapshots done by the "Network Appliance" fileserver boxes I've used at the last 2 out of 3 workplaces.

Re:Can No One Else INNOVATE?

[heffrey]

What evidence do you have that this is reverse engineering?

Or do you mean that they are re-implementing Time Machine?

Security, backups

[Midnight Thunder]

This solution certainly helps if you accidentally delete something or need to go back to an older version. SVN is one solution, but it is a bit more explicit, while solutions like this and Apple's Time Machine help avoid needing to remember to update your repository. It should be noted that this doesn't replace backups, since this does not protect against hard-drive corruption. I do have a few of questions though:
    - what are the security considerations here?
    - can you delete the existence of file, as to ensure that it is not easily found again?
    - what are the effects on hard-disc storage space, ie are there any estimates to how much extra storage is needed for this?

Re:Security, backups

[maraist]

what are the security considerations here?
No worse than block-allocations of the existing ext3.. You just have more than just one suite of data-blocks associated with one inode.

The '@' character might allow for some as yet not-conceived social engineering hacks.

Who can run the snapshot operation (most likely root), might be of concern.. I would assume that a system cron-task would be the target initiator.

can you delete the existence of file, as to ensure that it is not easily found again?
Considering that one of the original mandates of this project was to provide an auditable system, I'd say no. But it stands to reason that there is some fsck level operation that can wipe out non-current copys beyond a certain date. Even if it doesn't already exist, it's certainly possible if you've got unmounted block-level access.

what are the effects on hard-disc storage space, ie are there any estimates to how much extra storage is needed for this?

Hmm. I probably shouldn't answer this without reading the white-papers.. But why let that stop me.

Since it's block-level COW, it at least needs to get a "bit fatter" (har har).

Details not withstanding... There's the obvious duplication of all data-blocks that have changed.. BUT for every file + datablock change you have at least 1 allocation block that changes and possibly a copy of the inode info (which doesn't allocate a new disk-block, but does consume 1/kth of a block, depending on how many inode descriptions fit into a disk block). If changes have locality, then multiple data-block changes may only require a single allocation-block + inode change. But chances are that multiple changes in the same file will hit different allocation blocks. So you could see in the worst case 1/k + 2n disk block allocations, and at best 1/k + 1 + n allocations. Well, really, at best, we have 1/k + n allocations, because inodes store the first several data-block allocation references in the inode. Thus, if your changes within the first several kilobytes (or if your file is that small), your change overhead storage is minimal.

Two cases where this overhead is minimal are log files and small files. Small files, as already stated only require the inode to be copyed-off. Log-files are generally MASSIVE, but only ever touch the last allocation block (between snapshots).

Naturally random-access files like memory-mapped bookkeeping files (like those used in postgres and apache) would suck, the file would be guaranteed to be fully mutated on every snapshot. Thankfully they're pretty small (only a couple K).

source-code files and other documents generally are fully overwritten, BUT are VERY rarely updated (relative to their likely consumption of disk-space).

Email files are pretty static BUT the directory that holds them would get pretty chaotic. Unless you use mbox storage, then I guess it's more like log-files and would work great. :) Does anybody still use this style of mail server anymore?

Low-activity databases could work (such as trivial mysql-based password tables, and other configuration-oriented database deployments). But any reasonable amount of write activity would overload your disk space in short order.

Pretty much anything in /var or /tmp should probably not be put on this type of file-system.

Performance?

[Anonymous Coward]

So how does the mechanism affect performance? Aren't the files going to be very fragmented after a while? How long does it take to make those snapshots?

Re:Performance?

[Loconut1389]

almost no time, it just freezes the inode table and starts creating new inodes for changes instead of modifying old ones.

Re:Can No One Else INNOVATE?

[siride]

So I guess since it's old news, they just shouldn't bother right? Nobody is claiming this as an innovation. It's just a good feature that's getting added finally. After all, as people have said, VMS had this 20 years ago. Even MS and Apple didn't add something like it until the past two years. I should add that this project started back in 2005, so it's been worked on about during the same time Apple's stuff has.

Re:Can No One Else INNOVATE?

[siride]

Actually, the project started in 2003. So they were ahead of the game. Apple didn't demonstrate Time Machine until 2006.

Re:Can No One Else INNOVATE?

[SaturnNiGHTS]

this isn't a "recent project"...it was started in july of 2003...hardly stolen from microsoft or apple if older incarnations existed before they were developed. this is merely a version release. http://en.wikipedia.org/wiki/Ext3cow

Ze First Step (ZFS)

[udippel]

Done it, been there.
Guess, this is the first step to approach ZFS, which for some stupid licence reason doesn't seem to have an easy path into the Linux kernel.
ZFS does a few, actually a lot, more. But why not write a different solution, for a plurality of choice.
May the best win !

Re:Ze First Step (ZFS)

[OverlordQ]

IIRC the main reason ZFS wont make it into the kernel is that a non-trivial amount of the filesystem kernel code would need to be re-written.

Re:Ze First Step (ZFS)

[anilg]

That would be ZFS, built from the scratch to avoid the problems faced by traditional filesystems. It is, in fact, a volume-manager+file system, with RAID built into it. Not to mention the very easy to use commands.

Am I missing something here??

[MarsBar]

Looks to me (having read the paper) like you need to manually snapshot a file every time you might want to (later) revert back to it.

Now I don't know about anyone else but that's not what I want from a system like this: I want a system that keeps transaction logs, essentially, so that I can literally ask for any file as it was at any time.

Re:Am I missing something here??

[wild_berry]

My suggested solution would be to write a daemon that monitors atime or file writes and snapshots each time. Or configure a mount option or kernel tunable that enables or disables automatic snapshots -- you don't want to fill your /var/log with multiple copies of files that only get appended...

some background

[pikine]

I'm answering questions that people posted so far altogether.

Is it a file system or a file manager?

It is a file system. You access old snapshot by appending '@timestamp' to your file name. You have to first instruct ext3cow to take a snapshot first before you can retrieve old copies, otherwise it simply behaves like ext3. It appears that snapshot is always performed on a directory and applies to all inodes (files and subdirectories) under it.

My complaint is its use of '@' to access snapshot. Why not use '?' and make it look like a url query? Better yet, use a special prefix '.snapshot/' like NetApp file servers.

Does it store many copies of each file? or only the differences between the old and the new version?

How far off is it to use these filesystems as a revision control system replacement?

ext3cow takes it's name from "copy on write," and it does this on the block level. When you modify a file, it appears to the file system that you're modifying a block of e.g. 4096 bytes. COW preserves the old block while constructing a new file using the blocks you modified plus the blocks you didn't modify.

You can think about it as block-level version control. However, when you save a file, most programs simply write a whole new file (I'm only aware of mailbox programs that try to append or modify in-place). Block-level copy on write is unlikely to buy you anything in practical use.

Does it provide undelete?

Only when you remember to make a snapshot of your whole directory. An hourly cron-job would do, maybe. There is always the possibility you delete a file before a snapshot is made.

Re:some background

[TooMuchToDo]

Only when you remember to make a snapshot of your whole directory. An hourly cron-job would do, maybe. There is always the possibility you delete a file before a snapshot is made.

You could always replace rm with a script to perform a snapshot before the rm actually takes place. Problem solved.

Re:some background

[why-lurk]

Most of the posts here are focusing on the snapshotting feature of this copy-on-write filesystem. However, if you read the publications from the website, it's pretty clear that the main problems they are trying to solve are data security and secure deletions.

The innovations that they describe in their papers are that they are using authenticated encryption (that is, not starightforward symmetric block cyphers) via a cryptographic device driver, and that they can *securely* delete a file, in one or all snapshots, with very good performance by overwriting just a small portion of the file (the "stub"). This is accomplished, fairly cleverly, by using an encryption scheme dubbed All or Nothing (AON), which will make recover of the plaintext intractible if even a small portion of the ciphertext is missing.

Pretty cool really. Now, I would agree that their current interface of on-demand snapshots, and their semantics for accessing prior versions need a bit of work to achieve a decent level of practical usability. And I definitely would not expect wonderous performance, compared to a unencrypted snapshot system, like LVM.

    --kirby

Re:Can No One Else INNOVATE?

[beezly]

Go away MacTroll...

Veritas VxFS has had this for years. Snapshotting has been implemented in the Linux LVM layer for ages. This is just another way to do it.

I don't know anything about the technical implementation of Vista Shadow Copies or Apple's Time Machine, but if it's anything like ZFS then I'll be impressed. I believe there are rumours about the next release of OS X using ZFS (which was developed by Sun), but I'll believe it when I see it.

Re:Can No One Else INNOVATE?

[Anonymous Coward]

If you've ever studied b-tree filesystems, or filesystems in general, I'd call forking the inode chain rather obvious. I independantly thought up the same idea when I studied these only to find out of course that that is precisely how it is implemented.

Re:Can No One Else INNOVATE?

[CCFreak2K]

According to Wikipedia, Apple's Time Machine isn't even released yet. Maybe the person reverse-engineered Time Machine in the future and used the code to come back to the past...?

Also from Wikipedia, Windows XP Professional includes a similar feature, although it doesn't do as much as the facility included in Windows Server 2003.

Are you paid to make shit up or what? Can I get a job there?

The FS in SVN?

[Zaharazod]

What about the possibility of using a filesystem with built-in history storage as the backend for a Subversion repository? Client access would not change at all; assuming the underlying versioned FS were at all scalable though, I would imagine that increased performance and decreased complexity over things like BDB and FSFS might be well worth it.

Re:Can No One Else INNOVATE?

[Anonymous Coward]

Prior art, dude. Apple didn't invent filesystem snapshots. And btw. even Windows XP had system restore in 2001, which is a rudimentary version of that. Btw. does ZFS sound familiar?

Such a simple idea as taking a snapshot shouldn't even be patentable, it is too general. And it's implementations can differ significantly.

Re:Can No One Else INNOVATE?

[siride]

Because it wasn't REVEALED until 2006, so even if Apple was working on it in 2002 (not likely, since Open Source projects generally have longer cycles than proprietary ones due to manpower issues), the ext3cow people would not have been aware of it. Why do you think people are stealing this from Apple? It's a good idea that follows logically from ideas found in revision control software such as Subversion and its predecessors. And as others have pointed out, VMS had this 20 years ago. The idea certainly has been in existence for longer than Apple has. The wikipedia article indicates that the TENEX operating system in the 60s first had versioning filesystems. In any case, Apple hardly invented it, Apple was hardly the first to use it, and Linux implementations have been released before Apple even demoed Time Machine. So, basically, you are 100% wrong.

Re:Can No One Else INNOVATE?

[macs4all]

I believe that they may use ZFS for OS X Server first, but it will be another filesystem supported in addition to HFS+.

As for me being a troll: When does debate end and trolling begin?

I was simply pointing out that this "smelled" much like Time Machine, albeit a clumsy, wholly unintuitive version of the underlying technology.

I know that Apple didn't invent the idea of file versioning. What they invented (as is their trademark), was the way to make that technology USEFUL and ACCESSIBLE.

Can't tell, its slashdotted

[tinkertim]

Well, is it the file system, or the file system manager?

I can't tell, the site is experiencing the /. effect.

Mirror of the patch (I grabbed it when I saw this in the firehose) can be grabbed here until my server gets sluggish too.

in /usr/src type : patch -p1 linux-2.6.20.3-ext3cow.patch

The site said its not been tested with other kernel versions, but if you feel brave just s/linux-2\.6\.20\.3/your-version/g. Haven't tried it, but should work.

It wen't dark just around the time I was getting the docs and utilities.

Did anyone happen to grab the utilities? Got a link?

Re:Can No One Else INNOVATE?

[cortana]

"Theft" how?

And of what IP?

Make a specific allegation or stop trolling, please.

No Data

[wild_berry]

I can't see anything linked from the ext3cow.com site, save for the near-silent mailing lists. I'm tagging this 'slashdotted'. There's not even a huge amount on the Wayback Machine: http://web.archive.org/web/*/http://ext3cow.com

I guess that this is a fork of the ext3 code with Copy On Write functionality and userland tools to make snapshots and time-travel the snapshots. Wikipedia's article on Ext3cow names Zachary Peterson, the submitter of the article, and links to an ACM Transactions on Storage paper at http://hssl.cs.jhu.edu/papers/peterson-tos05.pdf.

Re:Can No One Else INNOVATE?

[macs4all]

You're right. Snapshots shouldn't be patentable. Apple's Time Machine GUI SHOULD, however. It was the "non-obvious" icing on an old, moldy cake.

But don't ever breathe the term "System Restore" and "Time Machine" in the same post again. Comparing those two concepts as if they were equals is like comparing the Space Shuttle to a Model-T Ford: Yes, they are both "mechanized transportation", but...

Re:Can No One Else INNOVATE?

[Anonymous Coward]

Look, I know you're a troll, and that you don't really use a Mac. But just for the education of others, you can dig around at fsf.org and find archives of talks that Richard Stallman gave about the GNU project way back when it started in the 80s, where he describes his idea for a versioning file system in Unix.

So versioning was always supposed to be part of the GNU system; as Linux is the kernel for GNU, this is just a new implementation of an idea from years and years ago. Next time you want to troll as a Mac user, try to learn some history.

Re:Can No One Else INNOVATE?

[siride]

I don't think ext3cow has a nice Apple-like GUI (which personally make me want to vomit in terms of functionality and looks). It's a command line interface. So they clearly aren't even TRYING to steal anything from Apple.

Full-System restore

[squizzar]

Doesn't this provide some kind of system restore as well? Assuming your entire system is on this FS, then any changes made, no matter how complex could be rolled back? Attempted to install some driver and broke everything? Just revert to the state before you made the changes... Of course, that means it's probably patented by Microsoft...

Linux is catching up to BSD...

[mi]

BSD operating systems had filesystem snapshots functionality for several years now... Linux is catching up — in a usual Linux way with patches, which one has to collect from all over...

Or am I misreading the write-up and this new ext3cow thingy is much more than that?

Re:Linux is catching up to BSD...

[Jokkey]

Linux has had filesystem snapshots (via LVM) for quite a while too. Ext3cow, as I understand it, differs in that it lets users access previous versions of individual files from within the current filesystem, rather than creating a snapshot of an entire filesystem or disk. As far as I know, it takes space out of the existing ext3 filesystem to do this, rather than using previously unallocated space within the disk volume group.

Re:Linux is catching up to BSD...

[emj]

The BSD feature is the same as ext3cow, and it's been there for a while. The LWN snapshots were very cumbersome some years ago, and they are block snapshots not Filesystem snapshots.

Re:Linux is catching up to BSD...

[Mysticalfruit]

Actually Linux has supported snapshots through the LVM layer for several years as well. This isn't a filesystem snapshot, it's a per file snapshot system.

Re:Linux is catching up to BSD...

[darrylo]

ZFS has recently been added to FreeBSD. ZFS is also rumored to be added to OS/X.

So, yes, Linux does have some catching up to do. ;-)

Re:Linux is catching up to BSD...

[Nevyn]

Not only are you "mis-reading" the summary, but you've blatantly failed to even look at your own so-called link to snapshots. Hint click on the snapshot link on Kirk's page and it'll tell you about linux LVM snapshotting from 7+ years ago.

Compatibility and copy on write...

[argent]

My first thought was the same as yours, why not use the ".snapshot" prefix from netapp, so that scriopts and tools written for Netapp servers will continue to work.

Second, I have hundreds of mail folders saved in files with names like "user@example.com". Oops.

Block-level copy on write is unlikely to buy you anything in practical use.

For binary files (eg, databases) it will. And it's pretty cheap to implement... for a whole-file write operation where the file is first truncated the cost is the same as if they didn't bother to COW, and it keeps lots of complete copies of log files from being created.

Re:Compatibility and copy on write...

[greed]

If the prefix was '@@', then I'd suspect them of being ClearCase fans.

I always suspected ClearCase as being, originally, written by some guys who missed the VMS filesystem on UNIX; all the other cruft being added later.

But boy could it panic a Red Hat box when we tried it....

Re:Can No One Else INNOVATE?

[init100]

Do you actually mean that it is "IP theft" to take functionality from the Linux Logical Volume Manager and implement it per file in the file system instead? Hardly.

Re:Can No One Else INNOVATE?

[macs4all]

Speaking of being paid to make shit up: Of COURSE Time Machine, which relies on OS X 10.5 (Leopard) isn't RELEASED yet; but it was DEMOED, fully-formed, at Apple's WWDC last summer.

Hence my "Reverse Engineering" comment in my original post.

So what was your point, again?

Re:Can No One Else INNOVATE?

[init100]

As I replied to another post: It's IP theft, direct or indirect; but IP theft just the same.

As I replied to another of your posts: It isn't.

Re:Can No One Else INNOVATE?

[macs4all]

As I said, when does debate end and trolling begin?

I've been using Macs since they were called Lisas.

Next. Idiot.

And my original post acknowledged that file versioning isn't a new idea. Learn to read.

Re:Can No One Else INNOVATE?

[ajs318]

"Theft", you say. Theft is unlawfully taking something that belongs to another person, with intent permanently to deprive them of it {so it's generally a defence to theft that you believed the former owner intended to destroy the article, since you can argue that you intended only temporarily to deprive them of it [for however long it would have taken them to destroy it]; though if the article derives value from the manner of its destruction [for example, a cream cake that they intended to destroy by eating it] then this defence may not work}.

Please explain what exactly it is that Apple no longer have, that they used to have before your alleged "theft" occurred?

Also, versioning file systems existed back in the days of the PDP-11.

Re:Can No One Else INNOVATE?

[frogstar_robot]

There is a huge difference between reverse engineering and reimplementing. To reverse engineer a thing, it has to be possible to study it in detail. Seeing a cool demo and making something that works like it isn't reverse engineering, that is re-implementing. Also, neither reverse engineering or reimplentation isn't automatically stealing either. Apple would be in a pretty piss poor spot if they themselves could not re-implement. It surely isn't as though only Apple has the right to make accessible technology and that anybody else who does so is some sort of thief.

All that can be as it may be. IMHO I don't think re-implementation is what is going on here. File versioning is a very very old idea and this filesystem is just another take on it.

Fanboy

[Slashdot Parent]

This is not even close to the same thing that is a BSD filesystem snapshot, but don't let interrupt your furious fanboy wankfest.

BSD snapshots are a lot like LVM snapshots (that have been available in Linux since 1998), except that under Linux, you are not limited to 20 snapshots.

What ext3cow does, which you would realize if you would have opened your ears before your mouth, give you true point in time recovery. In other words, without ever manually "taking a snapshot", like you'd have to under BSD, you can simply revert your filesystem to where it was at any arbitrary point in time. ("Oh crap, I need to revert to where we were at 8:52:12pm last Thursday!")

BSD, to my knowledge, does not support anything this advanced.

Re:Fanboy

[mi]

you can simply revert your filesystem to where it was at any arbitrary point in time.

No, you can't. According to this example you need to issue an explicit "snapshot" command — I checked my facts before posting, as well as I could, anyway. There is no word yet on the maximum number of snapshots — they may well be limited to 20 as well.

What a major oopsie, I might add... I mean, you could've come up humbly with something "As far as I know, ext3cow is better, because it requires no explicit snapshot-taking". Then it would not have been such an embarassment to be wrong.

But you chose to use terms like "wankfest", "mouth before ears", etc... Fanboy indeed...

Re:Fanboy

[Anonymous Coward]

Look you petulant cock, I read the thread through and you do appear to be a fanboy - complete with inaccurate statements and personal attacks.

I run on BSD, I hack on BSD - you do us BSDers no favors by being a complete wanker.

Learn some manners, next time post something intelligent - perhaps a nice summary of how easy soft update snapshots are to use, maybe a benchmark or two.

Re:Fanboy

[Anonymous Coward]

You're still an idiotic ranting fanboy-wannabee whose brain farts are a sign of adolescent stupidity, and instead of admitting it, you jump on the "ahah, I'm so right, because you said snapshots are automatic and they aren't. Oh I'm so good my neurons hurt, uh uh uh"

BSD snapshots are independent of the underlying filesystem and work by basically scanning and creating a file that marks the state of the filesystem at the time - this being the snapshot - that you can later mount and look at.

Ext3Cow works by checking what has changed and allocating new blocks for that data, essentially marking which blocks correspond to which version in the filesystem itself. Similar name, concept, completely different implementation and objective (you can bet time of snapshotting and retrieval is way different for both cases, as well as snapshotting limits, deletion, blah blah).

Go away boy, go play with your dolls or something.

Cows?

[Real1tyCzech]

ext3cow???

...and people think Microsoft's naming conventions are silly. How can anyone expect people to take Linux seriously when they don't even take themselves seriously?

Okay, you can call me an MS fanboy and bury this post now.

Re:Cows?

[644bd346996]

You seem to think that the name of a filesystem matters. It does not. In particular, desktop users should never have to know the name of their filesystem. If "ext3cow" keeps anybody from switching to Linux, it will be because they needed to learn the name of the filesystem, not because the name is pathetic.

Re:Cows?

[Anonymous Coward]

Copy on write. Nothing to do with bovines.

Ubuntu?

[wile_e_wonka]

I heard Ubuntu was planning to upgrade to Ext4 for Feisty, and then it fell through, and instead they were planning on Ext4 to be available as a patch approximately the same time Feisty was released. Is Ext3cow the change that Ubuntu was planning to impliment? (I realize Ext4 is different from Ext3cow, but I'm wondering if Ubuntu's getting this as an automatic update)

Re:Can No One Else INNOVATE?

[jonadab]

Actually, filesystem versioning is older than Apple as a company, much less OS X. ITS had it in the sixties, and VMS has had it since the late seventies. Nonetheless, it's an undeniably useful feature, and I'm glad it's finally making its way into the major OSes.

Re:Can No One Else INNOVATE?

[shani]

As for me being a troll: When does debate end and trolling begin?

Good question.

I was simply pointing out that this "smelled" much like Time Machine, albeit a clumsy, wholly unintuitive version of the underlying technology.

Here, for instance, the trolling begins at the word "clumsy".

NILFS?

[stu42j]

Anybody use the similarly featured NILFS?

NILFS is a log-structured file system developed for the Linux, and it is downloadable on this site as open-source software.

http://www.nilfs.org/en/index.html

Re:Can No One Else INNOVATE?

[rbanffy]

You say it's unintuitive just because it has no GUI.

And it shouldn't have one - it's a file system, not a userland application. The userland applications will come and may even look like Time Machine (I was once impressed, but it got less and less impressive over time, as I learned more about ZFS and LVM snapshots). I hope not - It's cool but not that much functional.

OSX is a nice piece of software and sure solves a lot of problems for its users, but claiming this is in any shape or form inspired on Time Machine is a veritable troll.

BTW, as was pointed out before, OS-level file versioning exists in a for or other since the VMS days. Most probably it was coded on VT05 terminals.

VMS file versioning was lame

[mkcmkc]

VMS was my first real OS, and I don't miss it at all. Its versioning was fairly useless--one of the first commands everyone learned was PURGE, to get rid of all of the clutter. In order to be useful, other versions have to be out of view during normal operation...

Re:VMS file versioning was lame

[Anonymous Coward]

Was that hard to redefine DIR?
You cannot have used VMS much...

On version naming

[porpnorber]

I've written something like this myself (just a prototype, so no good performance, but rather slicker feature-wise), though I doubt it will see the light of day. Still, I can answer your questions about namespaces. Anything that messes with the filesystem namespace in any way can, of course, cause problems. The 'real' solution is new system calls, new shells that know about them--a top to bottom extension of POSIX filesystems.

Not so practical in practise.

Why not use '?'? Perhaps you are not yourself a Unix/Linux user--that one's a shell wildcard, one of the oldest and most entrenched, and would cause all kinds of quoting problems. Actually, '@' is quite unusual in being a still-available character. Why not use '.snapshot/'? Unix philosophy, and it turns out to be true: the less typing a user has to do, the more useful the feature in practise. And I say that with conviction, as someone who has had a prototype running on their desk, and has had the pleasure of typing 'cd ..@tuesday_afternoon' and having it work. Plus, of course, as you yourself point out, someone is already using the ''.snapshot' syntax--and dollars to doughnuts, not just NetApp, but joe users who take 'snapshots' with cp -R, too.

Someone else asked what about files named 'joe@rhubarb.com'--it's not a good or beautiful answer, but it turns out to be practical enough just to pretend the '@' was escaped if the time part fails its syntax check; the problem isn't 'solved' but all the software I use daily then seems to work normally. I don't know if the cow takes this route, though. Again, the only industrial grade solutions are a controllable namespace (a wart in the making) or a mechanism whereby applications can delcare their awareness or otherwise of this feature at the syscall level (a tough sell).

Re:On version naming

[pikine]

The 'real' solution is new system calls, new shells that know about them--a top to bottom extension of POSIX filesystems.

Tools augmented with snapshot support won't save you any typing. You would have to specify additional command line, which is likely going to be longer than namespace hacks. If you're concerned about number of characters to type, you should prefer namespace hack.

Why not use '?'? Perhaps you are not yourself a Unix/Linux user--that one's a shell wildcard

In Unix, you can escape both '*' and '?' using backslash and still have filenames containing these shell pattern characters; this is unlike Win32 that forbids these characters to appear in file names at all. So I don't see this a problem.

Unix philosophy, and it turns out to be true: the less typing a user has to do, the more useful the feature in practise.

The Unix philosophy is inspired by Huffman coding---frequently used features should be shorter to type. For this reason, I don't think a long, tedious snapshot prefix/suffix is a problem because you shouldn't need to hunt for snapshot too often. If you do, you should let someone else manage your files.

My department uses NetApp (also provides a versioning file system) which uses the '.snapshot/' prefix, and I've been pretty happy with that convention. I don't find myself digging .snapshot very often, possibly once every few months, and it's easy enough to navigate. Under '.snapshot' you find the various hourly, daily and weekly snapshots, and it's pretty self-explanatory.

Re:On version naming

[porpnorber]

I do agree that sugaring the namespace remains useful even when there is improved API support. But an extended API opens the way for much richer application behaviour (of course, if poorly designed, this could be more tedious, but poor design is not a i>requirement), and for some different, useful tradeoffs, such as time-aware globbing and file browsers.

As to using '?' and escaping it with a backslash, sure, great solution! And when you quote that, you merely have to escape the backslash with a backslash, or two, or four. Of course, if there's a literal '?' in the filename, that needs to be escaped, and we can't really use backslash, because that's already escaping the '?' at the shell level. Hmm, '@' would be good, but we're trying to keep that free in case an email address is a filename. So maybe we can use '??'? And then we can avoid ambiguity by escaping both questionmarks with a backslash or two. Or four. Or eight. No, really, trust me, using a non-special character is a hugely better choice. (Think of it as a language design matter, because that's what it really is.)

Finally, as to the NetApp experience--I'm not saying '.snapshot' won't work. I am saying that I have my own testbed, and I'm my own guineapig. 'diff -c foo.c foo.c@11:33' is something I want to say often enough to be worth giving it a character, and I do want it to be terse. It's nice! It's really nice!

It's possible that it makes a difference that my prototype offers continuous protection rather than snapshots, and that the time syntax is relaxed. If nothing else, my own arrangement doesn't have a limited set of 'snapshots' with which to prepopulate the magic directory. But having had it, I wouldn't voluntarily trade '@' for '.snapshot/', no I would not.

Re:Can No One Else INNOVATE?

[heffrey]

Your original post said:

It isn't bad enough that MacroSuck(tm) has to copy Apple at each and every turn, now LINUX Devs have to do it, too?

I mean, REALLY? Given that Unix has had the concept of file "versioning" since I don't know when (but a long-azz time!), and Linux has had what, like fifteen years to come up with something like this, I find the timing of this "revelation" highly suspicious.

This is a reverse-engineering of Apple's Time Machine, through and through.

I never thought I'd see the Penguin stoop to MacroSuck(tm)'s "R&D" tactics. Bleh!

Mod me "troll" if you must; but you KNOW I'm right...

I guess I need to learn to read too since I don't see the bit where you "acknowledged that file versioning isn't a new idea"......

it's NOT a versioning filesystem

[sloth jr]

It's simply a filesystem with snapshots. Big deal. It'll only do cool stuff when you tell it to make a snapshot, not every time a file changes.

Re:Can No One Else INNOVATE?

[Anonymous Coward]

Usually when someone comes on here and makes what appear to be pro-Apple statements but with an obvious lack of basis in fact, they're trolling to make Apple look bad and further the "Apple cult" mythos. I'm sure you've seen this sort of thing before.

Maybe you just made a mistake, but if you calm down and read over what's been said, I think you'll realize that you haven't exactly been level-headed about this. Save it for when some Linux window manager or Microsoft re-implements Expose badly (as happens every week it seems). The release of yet another copy-on-write filesystem is a bad time to paint Apple as the One True Innovator. It's still extremely unlikely that this has anything to do with Time Machine, which isn't a filesystem innovation in the first place.

Interesting - I have a couple of questions

[ratboy666]

No flaming -- I don't have the time to research this, so I'll just post the questions!

1 - What happens to large databases? I am assuming a delta storage method, but that might slow down the database (specifically, I use mysql).

2 - Large files? Specifically, deletion (I store lots of videos)

3 - Usenet spools? (Lots of small files, deleted regularly).

I suspect that I would have to segregate my files...

time machine

[douthat]

I don't know anything about shadow copy, but Apple's Time Machine is all userland. There is a process that looks for file system events and logs the files that have been changed. Every x time units (e.g. 1 hour) a heavily hardlinked copy of your most recent backup is copied to a new tree and the newly modified files are copied over there. Every y time units (e.g. 1 day), all but the day's newest backup are deleted. If you run out of space, old trees are also deleted.

Re:Moo?

[Anonymous Coward]

Holy shit, for a moment that felt as if somebody had posted one of my passwords or something.

I mean... uh... ARRRRRRRRR JIM LAD

How Many beers in a six-pack?

[ChrisA90278]

Why ask silly questions like "Does it store many copies of each file?" It's "COW" Copy on Write. What's next "When was the war of 1812?" "How many beers in a six-pack?" The South Pacific is the Southern part of which ocean?"

The answer of course is that ext2cow copies the part that is changes or "written".

Re:How Many beers in a six-pack?

[scottv67]

Why ask silly questions like "Does it store many copies of each file?" It's "COW" Copy on Write. What's next "When was the war of 1812?" "How many beers in a six-pack?" The South Pacific is the Southern part of which ocean?"

You forgot "Who's buried in Grant's tomb?" :^)

Re:How Many beers in a six-pack?

[osu-neko]

How long was the Hundred-Years War? ;)

Re:How Many beers in a six-pack?

[acvh]

"You forgot "Who's buried in Grant's tomb?" :^)"

Nobody. It's above ground.

Re:How Many beers in a six-pack?

[HateBreeder]

Ain't you the genius?

Copy On Write - does not indicate in any way what is copied - It just says WHEN it's copied.

Perhaps it's the entire file that is being copied? it did change after all or Perhaps it copies just the modified block/cluster?, or Better yet, It could be just the few bytes that were changed!
They all, interestingly enough, fit exactly into the description of Copy On Write.

Maybe if it was called "CCBOW" as in "Copy Changed Block On Write", you'd have a case...
but it seems you're just a moron trying to troll around.

Re:How Many beers in a six-pack?

[OmegaDave]

Copy on Write has a very specific meaning in the file system world. So the poser wasn't trolling, you're just ignorant.

Can I create layers like in unionfs?

[a1mint]

I currently use unionfs in a vserver based virtualization solution, which works pretty well. When I add a VM, I create a unionfs mount, layer an empty writable directy on top of a read-only shared directly. Can ext3cow replace unionfs for me?

No way

[Burz]

The process isn't nearly as nice in practice as you make it out to be.

Features like ext3cow are kernel patches, not separate driver modules. Re-compiling a kernel can sometimes take *hours*, and who the hell is going to master the patch, config, make AND bootloader commands and switches to run the whole process every time their distro issues a security update for the kernel?

Its bad enough we have to keep track of and re-compile additional modules when kernel updates are issued. But re-patching and re-compiling the whole kernel is just beyond the pale even for most techies.

Re:No way

[marcello_dl]

Your point forgets about some distro's facility for building the kernel from source. Make-kpkg under debian based distros is one excellent example.

It might take hours if you build a lot of modules on less than 1ghz machines, but it runs unattended. A newbie reading the comment might understand it takes hours of tweaking which is not true. The first time one should try to understand what is a config file and modules, so to save time (a make oldconfig after using a config known to work on your machine saves a lot of trouble)

And bootloader setup after installation of a new kernel is often a matter of checking symbolic links.

This is very cool

[Deagol]

I was hoping for something more transparent, like the VMS version mechanism. No user intervention needed.

I envision the day when hard drives are so large that every version of every file can be stored indefinitely. Imagine being able to, as a senior CS student, fetch some code that you wrote freshman year but deleted. Very useful indeed!

Re:This is very cool

[Manannon]

I remember my freshman year code - I'm quite happy it was deleted :o)

Re:Can No One Else INNOVATE?

[sofla]

Given that Unix has had the concept of file "versioning" since I don't know when (but a long-azz time!)

*scratches head* Unix? Versioning? Never seen it myself. Not to say it isn't there, but over the years I've used several *ix flavors and fs versioning isn't something I've come across. I suppose next you'll tell us Unix has file locking (afaik it doesn't, unless you count advisory locks. I don't).

This is a reverse-engineering of Apple's Time Machine, through and through.

I hate to be one to point this out, but, er... Time Machine is a BACKUP tool. Don't believe me? Go to http://www.apple.com/macosx/leopard/timemachine.ht ml and read the copy yourself, being sure to pay special attention to use of phrases like "the drive you're backing up to". How on earth you could possibly confuse a backup tool with a versioned file system is beyond me.

Re:Can No One Else INNOVATE?

[descubes]

Go away MacTroll...

Meanwhile, from the source of the web site: <meta name="Generator" content="iWeb 1.1.2" />. I thougth this had quite a familiar look, it looks quite a lot like the web site I made with iWeb myself... Except I did not remove the "made on a Mac" logo to please the zealots.

sendfile

[kasperd]

Does this file system also provide a new implementation of the sendfile system call? Since it already does CoW, it should be possible to make sendfile also do CoW as long as both source and destination are on the same file system. If cp would then make use of the sendfile system call, then even cp would give you CoW, that would be really cool.

Like GoBack?

[Anonymous Coward]

Sounds a bit like GoBack. I'm not much of a Linux geek, but I am trying to switch over. One of the few programs I can't find replacement for is GoBack. Is there a Linux replacement for GoBack? Would this file system do the trick?

Synaptic doesn't distinguish

[Burz]

...between user-facing apps and all the other miscellera in a Linux system (libraries, daemons, other backends, etc.). A regular user operating a packaging front-end like Synaptic is a recipe for quick disaster (or frustration, whichever comes first).

A front-end like Xandros Networks, Ubuntu's, or Freespire's is kindof OK, as long as the user doesn't mind being chained to that distro's central repository.

As soon as users need software not supplied by the OS vendor (Microsoft, Apple, Debian...) then Windows and OS X become orders of magnitude easier to use than popular Linux distros. The same packaging and dependency logistics means that targetting Linux users with a program that can be installed simply and reliably is also much harder.

I want to KISS my Mac every time a kernel update is downloaded, because I DON'T have to recompile all the drivers I added to the system.

Linux is NOT going anywhere in the PC market in this shape. It will find niches (like governments and banks) as a thin-client solution that will inspire very few people to run it at home.

Undelete is the job of the OS

[DragonHawk]

First, it misleads users who expect some degree of data security.

So does data journaling or file versioning or automatic bad sector relocation (okay, that last one isn't part of the kernel, but it still effects things). Point being, it isn't 1985 anymore. You don't counter data remanence by disk wipes anymore.

The second argument is that it's better handled in user space...

I disagree with that assessment. That's the way Windoze does things, and it sucks. The problem is that any old program can and usually does just delete files. You have to have the whole world agree to rewrite their software to your undelete API, and that's just not going to happen in reality.

I suppose you could implement a shim in the userspace library implementation of the unlink() system call, but there are also efficiency reasons to implement undelete in the filesystem itself. As others have said, I pine for functionality like as found in Novell NetWare, which handled this efficiently and easily. SALVAGE saved me a lot of time restoring user files from tape, and it was doing so in 1993.

...so the OS doesn't have to make that sort of policy

"Mechanism, not policy." I'm not asking for the kernel to mandate undelete, just provide the mechanism for me to turn it on, should I so wish.

The final argument I can come up with is security problems. We can't have one global .Trash bin in a multiuser system. And quotas. And permissions.

All the more reason to implement it in the kernel, in the filesystem, and not just as bogus .Trash directory somewhere.

NetWare simply didn't count deleted files in quotas. They were purged on a LRU basis automatically, as needed, so this wasn't a problem.

Permissions were the same as anything else. If you had permission to deleted the file, you had permission to undelete it. In *nix land, this would be write permission on the containing directory.