Ubuntu Plans To Make ZFS File-System Support Standard On Linux

An anonymous reader writes: Canonical's Mark Shuttleworth revealed today that they're planning to make ZFS standard on Ubuntu. They are planning to include ZFS file-system as "standard in due course," but no details were revealed beyond that. However, ZFS On Linux contributor Richard Yao has said they do plan on including it in their kernel for 16.04 LTS and the GPL vs. CDDL license worries aren't actually a problem. Many Linux users have been wanting ZFS on Linux, but aside from the out of tree module there hasn't been any luck in including it in the mainline kernel or with tier-one Linux distributions due to license differences.

Comments

Re:ZFS is nice...

[Anonymous Coward]

Why would you need nvidia drivers on a file server? Use Ubuntu Server, it's made for, well, being a server.

Re:ZFS is nice...

[Frnknstn]

I run ZFS on any / every machine I can, server or not. That is one filesystem where the features outweigh all possible concerns.

What he didn't say

[n1ywb]

is anything like "ZFS will be the default". He just said that it would be in the distro.

BTRFS is getting there

[DarkOx]

I don't why so many in the Linux community are so hooked on ZFS. BTRFS has a feature set that is rapidly getting there, its becoming more a more mature in terms of code that is already in the upstream.

Why not just put your energy there?

Re:BTRFS is getting there

[Phil Urich]

Hell, it's already in many cases a superior experience on Linux, starting with that you can shrink a BTFS volume but you still can't shrink a ZFS volume. I suppose in the enterprise-centric world that ZFS is aimed at that's pretty much never an issue, but I've even run into it personally multiple times myself working for a small business and have been glad that I was running BTRFS instead. Frankly, for many use-cases it seems like running ZFS on Linux is more hassle for the sake of then more hassle later on.

Re:BTRFS is getting there

[Guspaz]

Because BTRFS is and has always been redundant? ZFS is far more mature, and stories abound of BTRFS failing on people. BTRFS is still unstable, particularly their RAID5/6 support. Developers should be putting their efforts into ZFS instead of BTRFS.

Re:BTRFS is getting there

[TrekkieGod]

I don't why so many in the Linux community are so hooked on ZFS. BTRFS has a feature set that is rapidly getting there, its becoming more a more mature in terms of code that is already in the upstream.

Why not just put your energy there?

As someone who uses both zfs (for file server storage) and btrfs (for the OS), my reason for using zfs is raidz. If btrfs implemented something similar, I'd drop zfs.

Re:BTRFS is getting there

[rl117]

It's really quite simple. ZFS is a great filesystem. It's reliable, performant, featureful, and very well documented. Btrfs has a subset of the ZFS featureset, but fails on all the other counts. It has terrible documentation and it's one of the least reliable and least performant filesystems I've ever used. Having used both extensively over several years, and hammered both over long periods, I've suffered from repeated Btrfs dataloss and performance problems. ZFS on the other hand has worked well from day one, and I've yet to experience any problems. Neither are as fast as ext4 on single discs, but you're getting resilience and reliability, not raw speed, and it scales well as you add more discs; exactly what I want for storing my data. And having a filesystem which works on several operating systems has a lot of value. I took the discs comprising a ZFS zpool mirror from my Linux system and slotted them into a FreeBSD NAS. One command to import the pool (zpool import) and it was all going. Later on I added l2arc and zil (cache and log) SSDs to make it faster, both one command to add and also entirely trouble-free.

Over the years there have been lots of publicity about the Btrfs featureset and development. But as you said in your comment that it's "rapidly getting there". That's been the story since day one. And it's not got there. Not even close. Until its major bugs and unfortunate design flaws (getting unbalanced to unusability, silly link limits) are fixed, it will never get there. I had high hopes for Btrfs, and I was rewarded with severe dataloss or complete unusability each and every time I tried it over the years since it was started. Eventually I switched to ZFS out of a need for something that actually worked and could be relied upon. Maybe it will eventually become suitable for serious production use, but I lost hope of that a good while back.

Re:BTRFS is getting there

[ArmoredDragon]

I've been using ZFS on Linux for about 3.5 years now, it's been pretty stable. I can't say I've heard of a case of it failing for somebody other than user error.

Re:BTRFS is getting there

[Guspaz]

zfsonlinux hit both unstable and stable releases on Linux earlier than btrfs: if your only definition of stable is how long it's been around on Linux, then btrfs is still less mature.

Being in-tree says nothing about the stability of a module, but ZFS doesn't need to be under the GPL to be in Linus' tree: the GPL does not forbid code aggregation. That said, neither Linus nor the ZoL team want ZoL in Linus' tree.

Re:BTRFS is getting there

[cas2000]

Because there's really no comparison between btrfs and ZFS. ZFS is years ahead in both stability and features. Only someone who's never used both would say that they are in any way close.

The only really useful thing that btrfs does that ZFS does not is rebalancing - that's a great feature and i'd love to see it in ZFS (but it will probably never get there).

ZFS has lots of features that btrfs doesn't have and likely never will.

Re:BTRFS is getting there

[rl117]

I mean the performance gains as you add more discs.

And regarding adding discs to an array, you certainly can. Just add addtional raid sets to the pool. That is, rather than adding discs to the existing array, you scale it up by adding additional arrays to the same pool. See the documentation.

Re:BTRFS is getting there

[rl117]

It can certainly work when everything is working correctly. Have you tested its behaviour when things don't work correctly, for example by pulling the cable on one of the discs as it's running? Does it carry on running, does it transparently recover when you plug it back in? When I had a cable become unseated and the connection glitched, Btrfs happily toasted the data on the drive, and its mirror, and panicked the kernel whenever the discs were plugged in; I had to zero them out on another system before I could even try to reformat them. One of the major historical weak points has been that the failure codepaths were poorly tested, and this can come to bite you quite badly.

Re:BTRFS is getting there

[ZorinLynx]

Likewise; we use it all over the place in our department. We have a bunch of 96TB/80TB usable ZFS file servers based on 24 4TB SATA drives. The performance is amazing for the price and they are rock solid under all kinds of heavy load, except for one tiny bug we hit recently that has been fixed already.

Re:BTRFS is getting there

[ZorinLynx]

I recently "fixed" one of our ZFS fileservers at work which was performing very poorly by *removing* a failing drive. The drive was taking a few seconds to read blocks, obviously dying, so it was slowing down the entire system. As soon as I pulled it ZFS finally declared it dead and the filesystem was running at full performance again.

I felt so confident being able to just walk up and yank the troublesome drive; that's how much trust I've built in ZFS. It's incredibly stable and fault tolerant.

Re:BTRFS is getting there

[Bengie]

Nearly all of the original Sun devs that created ZFS in the first place, still work on OpenZFS full time and are paid to do so. OpenZFS is very actively developed. They have 2 or more presentations per year about all of the changes they're constantly making and some of the upcoming big changes. Currently they are focusing on standardizing ZFS between FreeBSD, Luminos, and Linux. It's a large refactoring effort to have all ZFS's code bases to live in the same tree. One OpenZFS code tree for all OSes. Everyone will be in sync.

While you can't shrink ZFS pools because they cannot do that atomically, and they refuse to do anything that allows the end user to shoot themselves in the foot, like leaving the FS in an inconsistent state, you can create a new pool that is smaller and import your larger pool into the smaller one, as long as it fits. Can't do it in-place, but you can do it. It just sucks to do that with a 1PiB+ pool. But who shrinks those?

Re:BTRFS is getting there

[rahvin112]

The features you list as "specific" to zfs exist in btrfs. btrfs can have dedicated parity drives or you can spread the data and parity across multiple drives in any order or pattern you would like.

The write hole in btrfs is AFIAK also present in zfs and listed as a risk of a power failure during write on a raid pool with COW filesystems. This risk is that loss of power during write can result in multiple different parity blocks for the same data and that in such an instance the filesystem cannot identify the correct data or parity (depending on the order you write them) and there are only a few solutions to this that involve resorting to a known good (older) copy and result in lost data (from the write).

IIRC this is a listed risk in the FAQ for ZFS. Just as the same write hole risk exists in btrfs. Also IIRC ZFS takes the path of writing parity before data such that it will lose new data rather than risk a corruption of existing parity blocks. Whereas, again IIRC btrfs COW's the new data then COW's the parity block which risks inconsistent parity but at less risk of data loss (as parity can be recomputed).

Two different solutions to the same problem that is intrinsic to COW filesystems with parity data. Neither is particularly better IMO as both run the risk of data loss in an extreme event. Though such events are rare.

Re:ZFS is nice...

[sexconker]

A typical home Linux server - AKA an old PC - won't have IPMI. Actual servers typically will have IPMI, but they cost $BIG_BUCKS$. And even then, IPMI is extremely limited.

On the Dell servers I bought a few months ago I can't do anything useful with it beyond power on/off or text-only console redirection over serial (over LAN) before the OS loads (I can get into BIOS and the RAID controller ROM, not much else).
Unless of course I pony up more cash for their iDRAC Standard/Pro/Enterprise/etc. shit. THEN I can get graphical console redirection, some storage space to flash firmware from, and even USB/optical drive redirection.

Re:ZFS is nice...

[Bengie]

It makes other FSs look like FAT32.

CDDL and GPL don't mix

[Bruce Perens]

Regardless of what Ubuntu has convinced themselves of, in this context the ZFS filesystem driver would be an unlicensed derivative work. If they don't want it to be so, it needs to be in user-mode instead of loaded into the kernel address space and using unexported APIs of the kernel.

A lot of people try to deceive themselves (and you) that they can do silly things, like putting an API between software under two licenses, and that such an API becomes a "computer condom" that protects you from the GPL. This rationale was never true and was overturned by the court in the appeal of Oracle v. Google.

Re:ZFS is nice...

[TheRaven64]

No. RAID isn't better handled at other layers. If you don't know about the filesystem semantics then you need NVRAM or journalling at the block level to avoid the RAID-5 write hole. RAID-Z doesn't have this problem. If you're recovering a failed block-level RAID, then you need to copy all of the data, including unused space. With ZFS RAID (all levels), you only copy the used data. There are numerous other advantages to rearranging the layers, including being a lot more flexible in the provisioning.

It's also a mistake to think of ZFS as a layer. ZFS has three layers: the lowest handles physical disks and presents a linear address space, the middle presents a transactional object store, and the top presents something that looks like a filesystem (or a block device, which is useful for things like VM disk images).