ZFS On Linux - It's Alive!

lymeca writes "LinuxWorld reports that Sun Microsystem's ZFS filesystem has been converted from its incarnation in OpenSolaris to a module capable of running in the Linux user-space filsystem project, FUSE. Because of the license incompatibilities with the Linux kernel, it has not yet been integrated for distribution within the kernel itself. This project, called ZFS on FUSE, aims to enable GNU/Linux users to use ZFS as a process in userspace, bypassing the legal barrier inherent in having the filesystem coded into the Linux kernel itself. Booting from a ZFS partition has been confirmed to work. The performance currently clocks in at about half as fast as XFS, but with all the success the NTFS-3g project has had creating a high performance FUSE implementation of the NTFS filesystem, there's hope that performance tweaking could yield a practical elimination of barriers for GNU/Linux users to make use of all that ZFS has to offer."

Comments

Why not in the kernel?

[seanadams.com]

The in-kernel vs userland distinction has always struck me as quite arbitrary. So in one case you're linked at compile time and in another case you compile them separately and go through system calls. Why should that make one of them a derivative work and the other not? In either case the file system can be taken out and you still have a perfectly functional kernel that can run other file systems. Same goes for graphics drivers.

The GPL doesn't attempt to codify all the intricate details that it would take to define such a distinction in the license. It's only described as an accepted rule of thumb in the FAQ. So what's the deal? It seems like this rule is really holding back some commercial support for Linux - is the current situation what we really want, and at any rate how did we get here? Would we be better off if such a separable, non-essential feature could be linked in somehow instead of needing to be put behind extra layers of abstraction?

Re:Why not in the kernel?

[vialation]

It has been decided that the deciding factor with the GPL is whether or not something links to it. Instead of a userspace program which simply makes a system call by tripping an interrupt or using the SYSENTER opcode (which would not require kernel source code to do), a kernel module (which, without FUSE, a filesystem must be) actually gets linked into kernel space, and has its symbols added to the kernel. Any calls to other parts of the kernel directly call the kernel. That is the distinction.

Re:Why not in the kernel?

[TeknoHog]

The in-kernel vs userland distinction has always struck me as quite arbitrary. So in one case you're linked at compile time and in another case you compile them separately and go through system calls. Why should that make one of them a derivative work and the other not?

I agree the kernel vs. userland issue is arbitrary. However, think about all the closed-source software running on Linux, or opensource with other licenses but GPL v2. These are legally possible only because we make the distinction.

Re:Why not in the kernel?

[LWATCDR]

I am not a kernel hacker but I believe you are confusing loadable kernel modules with user space code.
I am pretty sure that kernel modules run at the kernel level and can access all the same structures that a driver compiled into the kernel can.
FUSE file systems run at the same privilege level as a user program does. In theory it is a slower but more robust system. If a FUSE file system crashes you can just restart the filesystem and remount the drive. If a kernel level file system crashes it can cause a kernel panic and bring down the entire system.
So the distinction between user-land and kernel drivers is anything but arbitrary.

Re:Why not in the kernel?

[Paul Jakma]

It has been decided that the deciding factor with the GPL is whether or not something links to it.

This is absolutely not true. "It has been decided": When? By whom? This claim is not only unfounded, it is also quite false (just ask a lawyer..).

The parent absolutely should NOT be modded as informative.

Re:Why not in the kernel?

[BrainInAJar]

How is it Sun's fault that the GPL is incompatible with anything other than itself?

The Right Thing(tm) is to keep the license as it is. It ensures the Solaris code has to be shared (like the GPL), but doesn't pollute source code around it ( GPL - viral clause = CDDL. Same license as firefox, or apache)

Linux wanting to pillage from the project isn't a good enough reason to make it impossible for people to write non-GPL drivers for Solaris

Re:Why not in the kernel?

[notamisfit]

The version of the GPL included with the Linux kernel states at the top:

• NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work".

Not sure how far back this clarification really goes, but I think it predates the GPLv2-only one, making it at least six years old.

Re:Why not in the kernel?

[jimicus]

Alright then, how about "The de-facto understanding is..."?

The existence of the LGPL goes some way to suggesting that this indeed is the generally accepted understanding. It's not been codified in any legal judgement that I'm aware of, but that's only becaue there have been relatively few legal judgements worldwide on this particular aspect of the GPL. Most breaches seem to be far more flagrant.

Re:Why not in the kernel?

[BrainInAJar]

"Because the CDDL was deliberately made incompatible with the GPL, that's why. Also ask Sun about the needless patents they seem to want to hold on ZFS."

The CDDL is more or less the exact same license Firefox or Apache are under. Sun has a lot of ISV's and IHV's that want to be able to write code ( storage drivers, for example ) that can link against the Solaris kernel without having a team of lawyers analyze to see if it's okay.

It wasn't chosen to be incompatible with the GPL, it was chosen to provide some of the same protections ( share my code... ) without being incompatible with other licenses.

As for the patents, ask IBM how many patents they've got expressed in Linux

"People could still write non-GPL drivers for Solaris regardless. I take that to mean you don't understand the meaning of a GPL compatible license."

You can't link to GPL'ed code with an incompatible license. Hypothetically if EMC or Symantec wanted to write a closed-source driver for this hypothetical GPL Solaris, they'd have to pull an nVidia ( which is a lot of effort for purely non-technical reasons ), or stop supporting the platform.

Would that really be in Sun's best interests, if I(S|H)V's stopped supporting them?

Re:Why not in the kernel?

[BrainInAJar]

or they could avoid the hassle of managing a dual licensed project & the inevitable GPL-only fork ( what happens if someone wants to import a driver from linux? that's an instant fork ) by keeping the perfectly-fine license they have right now

relicensing Solaris as GPL is entirely unnecessary and doesn't help Sun or Solaris at all... the only people it helps is Linux, and that shouldn't be the primary concern of OpenSolaris.

If OpenSolaris happens to help Linux, great, but it shouldn't hurt itself & go out of it's way to do it

Re:Why not in the kernel?

[ChrisA90278]

"It has been decided": When? By whom?"

Linus decided. He wrote specific terms into the modified version of the GPL he uses with Linux. He makes it very clear. Linux does NOT use the "standard GLP" it makes a few changes for example he removed the "and later versions" part.

Re:Why not in the kernel?

[Schraegstrichpunkt]

Linus decided. He wrote specific terms into the modified version of the GPL he uses with Linux.

Retroactively, AND claiming to cover code for which he is not the copyright holder. The notice at the top of the COPYING file in the kernel source code more-or-less amounts to his opinion---and if you do a little research (read: Groklaw), you'll find that Linus clearly isn't a lawyer and doesn't know what he's talking about when it comes to legal matters.

"Linus said so" is just not a valid argument about a legal matter, unless you're trying to defend yourself against a copyright infringement claim that he's making about code he holds the copyrights for.

Re:Why not in the kernel?

[afidel]

Constant changes in and out of kernel space are never good for overall system performance.

Re:Why not in the kernel?

[lymeca]

The Right Thing(tm) is to keep the license as it is. It ensures the Solaris code has to be shared (like the GPL), but doesn't pollute source code around it

No. First off, do not turn this into a continuation of the debate of which is more Free: copyleft licenses such as the GPL, or BSD-style "our code is SO free you can make it NON-free!". This specific situation is much simpler: Linux has been GPLv2 licensed for about 15 years now, and the GPL itself is much older than that. Sun created the CDDL in the 2000's (not sure which year exactly) and has thereby acheived deliberate license incompatability with Linux and the GPL. There are other non-copyleft licenses which are GPL-compatible that Sun could have chosen, but they didn't: not that there's anything inherently wrong with this of course, it's their decision. Also, the GPL is NOT "incompatible with anything other than itself" like you state, check this list to find all the licenses that disprove this wildly inaccurate statement.

It is not inherently Right or Wrong to use the CDDL for ZFS or the rest of Solaris. Sun holds the copyright on that code, so Sun can choose to license those projects however they see fit. It is not your place to say what license is correct for Sun to use on their code. Perhaps to your dismay, Sun has been seriously considering GPLv3 for Solaris, much like they did with Java and the GPLv2. And for all your talk about Linux "pillaging" code from Sun, you conveniently forget that this was a main driving reason for Sun to license Java under the GPLv2: license compatibility with GNU/Linux for easy inclusion of Java technology in distributions to attract users and developers.

( GPL - viral clause = CDDL. Same license as firefox, or apache)

While the CDDL was originally based on the Mozilla Public License, it is not the same thing. Perhaps more importantly though, Firefox is tri-licensed under the MPL, LGPL, and yes, the GPL. I can get a copy of Firefox licensed under the GPL, so it's especially important not to mislead others into thinking that such a prominent Free Software project exclusively uses a non-copyleft license, the CDDL, which isn't even one of the three licenses Firefox uses in the first place! Oh, and Apache has its own license altogether, albeit a non-copyleft one, the Apache license.

Re:Why not in the kernel?

[the_womble]

Linux does NOT use the "standard GLP" it makes a few changes for example he removed the "and later versions" part.

Wrong. Here is the COPYING file for the Linux kernel.

He has added a note clarifying that userland programs making system calls are not derivative works. That is clarifying his interpretation of the license. At the most, it adds an exemption, which is common practice.

The "and later versions" clause is not part of the GPL, it lies outside it. In effect, it makes the work automatically multiple licensed as and when new versions of the GPL comes out: so if you distribute something under GPL v2 with that clause, when v3 comes you it will become dual licensed under 2 and 3, when v 4 comes out it will be triple licensed etc. This allows people to redistribute under v2 or any later version, with or without the "and later versions" clause.

Re:Why not in the kernel?

[mabinogi]

There is no license that gives you the right to re-license someone else's copyrighted work.
The GPL doesn't require that everything linked to it be GPL'd, it requires that any changes or derivative works be licensed under a license (any license) that adds no more restrictions than the GPL itself does.

Grub

[jshriverWVU]

Booting from a ZFS partition has been confirmed to work

Grub has supported ZFS booting for a while (forget which branch though).

Re:Can't you make a binary blob kernel module?

[jshriverWVU]

Can't you just make a binary blob kernel module? That is basically what they are doing. In the case of Nvidia they write the binary blob driver and have an OSS driver to interface between the kernel and the blob. In this case ZFS is using FUSE instead of creating it's own interface code into the kernel.

Re:It's time for Sun

[AKAImBatman]

They also have co-opted gnome

By "co-opted" I presume you mean, "Made major contributions to"?

Re:It's time for Sun

[PygmySurfer]

Let's find a way to settle these license issues. ZFS looks to be great innovation, but Sun appears to be playing license games with the express purpose of keeping Linux at bay.

Sorry, it's Linux that's playing the license games, not Sun. One only needs to look at ZFS support in FreeBSD to see that (Speaking of, where's the 'ZFS On FreeBSD!' story?).

The GPL "everything under our license" philosophy is the sole cause of these so-called "license issues". If Linux wants to use Sun's code, why should Sun have to release it under Linux' license?

Re:Never mind ZFS

[Krondor]

What I desperately need is a reliable caching filesystem with decent performance.

The $COMPANY network is loaded with Linux workstations and servers, all with their own lotsabyte drives -- and the only things those drives are used for is a tiny system image. Meanwhile the network is getting hammered.

Are you asking for a network based filesystem like AFS? Did I misunderstand your issue?

Cheesy Intro to ZFS Video

[kaleco]

If you can suffer the bizarre presentation style Sun have used for this video, it's quite informative about the benefits of ZFS.

Why the big deal about booting from it?

[pair-a-noyd]

Really? Stick an old 20gb drive in as your boot drive and boot from whatever you have to to get up and going, load ZFS modules, mount all drives and enjoy. What's so terrible about booting from a different drive / file system? Most mobos now let you hang boatloads of drives of all types on them.
I can't think of any reason why it would be so terrible to boot up from an old 20gb with ext2/ext3 or anything else, then run the rest of your system under whatever. I'm doing that now anyway, I boot from ext2 then everything else is ext3. Doesn't make my performance suffer any that I can tell.

Besides, I suspect that most people that would run ZFS are the type of people that leave their machines up for months at a time. In that case, why the panic attacks over booting issues?

I hope they can find some way to resolve the license issues, I'm excited about ZFS (in concept and theory) and I would love to give it a go. Finally a system that's up with the times.

Parts of ZFS already GPLv2'd

[andrewd18]

According to Darren Moffat at SUN, parts of ZFS are already licensed under the GPL v2. Maybe there is still hope for a native solution. Not, of course, that I mind using FUSE.

http://blogs.sun.com/darren/entry/zfs_under_gplv2_ already_exists

Now about that headline, yes I really did say that ZFS code is already available under the GPLv2. I will be completely honest though and make it clear that it isn't all of the ZFS source. It is, sufficient amount to be able to boot an OpenSolaris based system from GRUB, that means that support for mirroring and the checksum and compression support is there but radiz isn't nor are the userland commands. It is possible that this might be enough to get someone started. Still don't believe me check out the updated GRUB source on opensolaris.org, specifically all the files with zfs in their name - every single one of them under the GPLv2 or later.

Re:Parts of ZFS already GPLv2'd

[thesandbender]

Lets be more specific... ZFS (as a whole) is not GPLv2'd.

From the TFP : Now about that headline, yes I really did say that ZFS code is already available under the GPLv2. I will be completely honest though and make it clear that it isn't all of the ZFS source.

Well that's fantastic... which parts do we get? The ones that make ZFS revolutionary or the ones that make it a rehashed XFS, JFS, Rieser, etc? I don't see how this is any different than any of the bait-n-switch scams that people post to /., digg or other sites. Yes... you can use "part" of the FS but if you want the whole thing you'll have to use Solaris or FUSE (or BSD as others have pointed out).

FUSE defeats the entire purpose. ZFS is meant to run and support a large/huge file store. What admin in their right mind would do that through userspace unless it's solely for backup?

The point is, ZFS is not functionally viable for Linux on the environments for which it was intended.

The headline...

[Savage-Rabbit]

ZFS On Linux - It's Alive! Shouldn't it be more like:

ZFS On Linux - It's Alive!! IT IS ALIVE!!! MWUHAHAHAHAHAHAHA!!!!!

The manic laughter is especially important!

Re:It's time for Sun

[jimwelch]

Everyone that has a license is "playing the game". That is required by most copyright laws. The only truly free software is in public domain, the downside to PD is code confiscation that is possible. BSD, GPL, M$ all use a license with restrictions. A restriction limits one or more freedoms. You have to choose which freedom to give up.

Re:It's time for Sun

[Anarke_Incarnate]

Linux is a kernel (OS If you don't want to be pedantic) and LINUS (Linus) is a programmer and benevolent dictator.

Confuse them again at your own peril.

Legal question

[TopSpin]

Why couldn't ZFS be distributed separately in kernel module form and installed by the user? Ubuntu and others integrate mscorefonts, NVidia drivers and others this way. Is the OpenSolaris license so heinous that it's worse than those examples?

I doubt it.

Re:Legal question

[burndive]

Actually, there is nothing wrong with Sun's license. The problem is the GPL, which does not allow CDDL code to to link to it; so the Linux foundation would be the one sending C&D letters to that project, since it would violate the kernel's GPL.

Only if you distribute binaries.

There is nothing that stops me from developing and distributing a version of Sun's ZFS such that it works with a Linux kernel. I can do anything I want with GPL code while I have it, including link it with my CCDL patch (and publish said CCDL code under the CCDL license), as long as when I distribute GPL source or binaries, I abide by the terms of that license.

kernel patch?

[FunkyELF]

Why couldn't this be implemented in the kernel and have the patches to that kernel be hosted in a country which doesn't care too much about licensing?

Re:kernel patch?

[burndive]

Why couldn't this be implemented in the kernel and have the patches to that kernel be hosted in a country which doesn't care too much about licensing?

Why hide? Nothing in the CCDL or GPL licenses prevents you from publishing a CCDL patch to a GPL kernel. Publishing the patched kernel source or binary might run you into some problems, but I don't see anything preventing you from publishing the patch itself.

Please correct me if I'm wrong.

Re:It's time for Sun

[PygmySurfer]

ZFS is on OpenSolaris and Sun has claimed to be considering GPL for OpenSolaris. Are they, or aren't they? On top of that, the FSF has muddied the waters through their activity on the GPLv3, further complicating the entire issue.

I don't care if Sun says they're considering GPLing OpenSolaris, ZFS, or anything else for that matter. The poster I replied to accused Sun of keeping ZFS from Linux by not GPLing it - when it's the goddamn GPL that Linux uses that is preventing the inclusion of ZFS!

I don't think you can blame the whole situation on Linux's use of the GPL, which is not coincidentally the reason why many people contributed to Linux. Given that Linux is today considerably ahead of all BSDs in most ways, I think adoption of the GPL is likely the only reason Linux is here today.

I'm not sure how Linux can be ahead of the BSDs, as Linux is just a kernel, while the BSDs are entire operating systems. But let's say you were referring to Linux distributions being "considerably ahead" - I've never seen this. I've always found the BSD's to be elegant systems to work on, and Linux systems to be a mess (I unfortunately have to admin hundreds of Linux boxes at work). Linux supposedly has better driver support, yet I've always found FreeBSD supports my hardware just fine (and for many things, like wireless drivers, I've found the BSDs to have better supprt than Linux). Linux may perform a bit better in some instances, but IMHO the negligible performance gains aren't worth the aggravation.

Finally, if you don't care about software freedom, and only your freedom, why don't you go run BSD, and stop complaining about Linux?

I use FreeBSD on my personal server, and I believe BSD code to be more free than GPL code, but that's irrelevant. Frankly, I'm sick of the Linux community telling everyone else what to do with THEIR code. Besides, you can hardly call my post a complaint - if anyone was complaining, it was the original post I replied to.

That's great!

[greginnj]

... but does it run on Plan 9 ?



/me ducks ....

"The FSF guideline is"

[Per Abrahamsen]

The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not. It is a guideline, not a hard rule though, and I suspect they would consider user-space ZFS for a derived work using a technical trick to avoid being linked into the kernel. I.e. infringing. However, since the FSF doesn't own the kernel, their opinion on the subject doesn't matter.

Re:"The FSF guideline is"

[pthisis]

The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not. It is a guideline, not a hard rule though, and I suspect they would consider user-space ZFS for a derived work using a technical trick to avoid being linked into the kernel. I.e. infringing. However, since the FSF doesn't own the kernel, their opinion on the subject doesn't matter.

OTOH, some respected kernel developers (e.g. Alan Cox) certainly have explicitly said they believe binary kernel modules must be GPL and that things like nvidia's drivers are probably infringing. Not exactly the same thing but it's enough that I'd still call this a grey area until it's tested.

Re:"The FSF guideline is"

[Paul Jakma]

The FSF has used the syscall interface as a guideline to determine whether something is a derived work or not.

I don't believe that's true. I think you're thinking of the explicit exemption Linus put in the COPYING file of the Linux kernel to say that the syscall interface was a GPL interface (there are Linux contributors who disagree to an extent with him on that).

Re:But, doesn't Linus...

[Sancho]

He certainly owns the rights to parts of the Kernel, and thus his opinion on the matter has some importance. However he has incorporated patches into the source tree without requiring that the copyright be turned over to him, and thus, just about everyone who has contributed to the kernel collectively 'holds the copyright' to it.

The Linux kernel, for better or for worse, is going to be stuck at GPLv2.

Re:Ah, well then...

[Sancho]

Yes, there are many times when copyright prevents the creation of new works. For example, I can't modify Harry Potter 6 so that Dumbledore doesn't die at the end and re-sell it. At least, not until the copyright expires. Once the copyright expires, you get great works like Wicked and Son of a Witch, both of which are based upon The Wizard Of Oz. Of course, no more copyrights will ever expire, thanks to greedy copyright-holding corporations and a corrupt government.

The idea, though, is that the work which you wish to modify might not have existed at all had copyright not existed. The person who created the work might not have desired to if s/he wasn't going to be able to sell it. Or they might not have been able to devote their lives to their craft, and thus ended up not having enough time to create as much. I highly doubt that the entire library of Stephen King would exist today if he wasn't a professional writer, paid for his craft.

It's obviously impossible to know what might have been, but I think that the reasoning behind copyright (in general) is sound. The problems in the current implementation are that copyright is effectively endless (meaning that the creation of new works based on the original is forever forbidden--forever being the key word) and that fair use rights are going out the window.

Noooooo!!!!

[r00t]

Linux is becoming a microkernel. Linus might even get a passing grade.