Ubuntu 16.04 LTS To Have Official Support For ZFS File System (dustinkirkland.com)

LichtSpektren writes: Ubuntu developer Dustin Kirkland has posted on his blog that Canonical plans to officially support the ZFS file system for the next Ubuntu LTS release, 16.04 "Xenial Xerus." The file system, which originates in Solaris UNIX, is renowned for its feature set (Kirkland touts "snapshots, copy-on-write cloning, continuous integrity checking against data corruption, automatic repair, efficient data compression") and its stability. "You'll find zfs.ko automatically built and installed on your Ubuntu systems. No more DKMS-built modules!" N.B. ext4 will still be the default file system due to the unresolved licensing conflict between Linux's GPLv2 and ZFS's CDDL.

Comments

For home users, basically meaningless.

[Anonymous Coward]

All file systems are approximately the same for most day to day users. I would be interested in knowing which is fastest at read/writes.

Re:For home users, basically meaningless.

[Anonymous Coward]

All file systems are approximately the same for most day to day users. I would be interested in knowing which is fastest at read/writes.

And that's meaningless without specifying the hardware you're doing the comparison on, your access pattern(s), file system layout, data distribution within the file system, and other factors.

Re:For home users, basically meaningless.

[Anonymous Coward]

I think you don't know what ZFS really is. It's a very different deal than etx4, ufs etc... It is the file system that made HW raid controllers obsolete. Even with a single disk setup you get a lot of features that you don't have on most of the other FS. It is a big deal just because of cheap snapshots, and data integrity checks.
And no, BTRFS is not even close... yet.

Re:For home users, basically meaningless.

[UnknownSoldier]

> I would be interested in knowing which is fastest at read/writes.

Ignoring the fact that this is a HIGHLY ambiguous question, i.e. you don't specify _which_ RAID setting, here are some benchmarks:

= 2010 =
http://www.zfsbuild.com/2010/0...

= 2013 =
ZFS On Linux 3.8 Kernel, ZOL 0.6.1
https://openbenchmarking.org/r...

= 2015 =
A PERFORMANCE COMPARISON OF ZFS AND BTRFS ON LINUX
* https://www.diva-portal.org/sm...

Re:For home users, basically meaningless.

[Curunir_wolf]

I think you don't know what ZFS really is. It's a very different deal than etx4, ufs etc... It is the file system that made HW raid controllers obsolete.

It also made just about any computer with less than 8 GB of RAM obsolete. It's also not very friendly with applications that need large chunks of RAM, like a database or large Java VM application - the ARC cache causes a lot of fragmentation and is often slow to release it when other applications need more.

Re:For home users, basically meaningless.

[Aaden42]

On 64-bit hosts, the ARC cache is a non-issue. Java needs contiguous *virtual* memory space. Physical memory fragmentation isn't a problem w/ the MMU translating contiguous 64-bit address space to possibly non-contiguous physical pages. On 32-bit hosts, that gets dicey. On 64-bit, you've got plenty of room even w/ ARC.

That said, I'd love to see ARC & the native Linux disk cache functionality either merge or at least have ARC behave more like the normal caching mechanism (IE free up RAM more eagerly), but it's not actually caused me significant problems on 64-bit.

Re:For home users, basically meaningless.

[The-Ixian]

I used MythTV for years as a DVR and I tried a lot of different file systems.

The 2 that always worked the best were JFS and XFS for the sole reason that large file deletes took almost no time at all. Compared to several seconds or even minutes with other file systems.

Re: For home users, basically meaningless.

[Zeromous]

If you are using ZFS, you need to have the offline backup.

So many things can go wrong with ZFS due to failures beyond your control. You use ZFS so you don't have to restore, and keep an offline backup for when ZFS is fucked.

If you can't afford to offline your ZFS data, ZFS is not for you.

BTRFS

[ssam]

I'll stick with BTRFS thanks. It gives me all those features, is GPL and has been trouble free for me on many TB of disks for several years.

Re:BTRFS

[Anonymous Coward]

As a die hard BTRFS user that chases kernel releases like a addict chases crack, I can't help but say that there are still some annoying issues out there.

While none have given me data loss, you'll get the occasional deadlock from a set of kthreads that do compression or a severe slowdown with next to no disk I/O and big WAITIO (usually get 16.xx Load in such cases on a quad core machine). For the slowdown case you'll get a speed drop from 150MB/s to 900~KB/s on spinning rust for a couple of minutes. Happens only after heavy use in the range of 2+TB written with forced compression.

ENOSPC? Not on my end. Trying to copy a file and running out of space results in WAITIO through the roof while BTRFS tries to find free space. I've had a job that stalled and thrashed the hard drive for 9 hours while it tried to recover space. At no point did it simply kill the transfer due to out of space, btrfs usage showed around 1GB of space left with plenty for metadata. It's at 1GB free for data extents and that's what kills the whole deal. You can't use that last 1GB, you'll just deadlock until some space is recovered by deleting files manually. Happens every time, just make sure to transfer something that is larger than the available free space and watch it suffer.

All this with Linux kernel 4.4.2. Looking at the various mailing lists with regular posts from people with obscure problems I've never encountered before, can't really say it's on par with ZFS stability. And ZFS On Linux is still missing a few things last I checked from the true ZFS implementation, but it's usable. Can't comment on ZOL long term stability, but I would feel comfortable enough using it instead of BTRFS for say a production server.

Re:BTRFS

[ssam]

I am using BTRFS on luks on my laptop. Even during a motherboard failure that cause repeated hard poweroffs I did not loose any data (and thanks to data checksumming I know that there is no corruption lurking in the files). BTRFS has developers at Facebook, Fujitsu, SUSE, IBM and still gets patches from people at Oracle. Seems a fairly healthy project to me.

For containers

[DrYak]

More precisely, the blog bost is about using ZFS' copy-on-write (CoW) capabilities in the context of linux containers.
(thin virtualized machines. The guest share the same kernel as the host, but the userspace is separated and compartmentalized using the kernel's cgroup feature.
Similar to BSD Jails and Solaris containers.
Think like a chroot, except extented to all the other concepts beside file system).

The fast and easy snapshoting that come with CoW filesystems like ZFS (or BTRFS for that matters) makes it very easy to spin new virtualized containers simply by snapshoting the subtree holding the empty template, while wasting only minimal resource (only the differences are stored as the two copies diverge over time).

Like a train wreck in reverse

[BaronM]

Every time I see news about ZFS and Linux, it's a little bit less of a mess. Eventually, I expect that all of the major distributions will go this route and sidestep the licensing issue by providing distro-supported modules that are installed by user request, sort of like the way that Nvidia drivers are provided.

Re:Too little, too late...

[LichtSpektren]

I used the wrong tool for the job, therefore it sucks.

Lies

[dlenmn]

It's not quite abandonware, but the central impetus for it's creation and advancement is gone.

I wasn't planning to comment on this thread, but this is too big a lie to let stand -- unless by "not quite abaondonware" you mean "has absolutely nothing in common with abandonware besides being a type of software". Oracle was never the sole developer, and now that Oracle has lost interest, the developers just moved to other companies and kept doing the same thing. Its raison d'etre remains to provide an advanced filesystem that's easily integrated with linux, which for better or worse means being licensed under the GPL or something compatible.

As for encryption, yeah that would be nice to have, but it's not like zfs has all the features btrfs has. I'll take btrfs's online balancing (ability to add and remove drives at will) over built in encryption, but I realize that's a personal choice.

Finally, let's actually quote the FAQ correct only stability:

Short answer: Maybe.

Long answer: Nobody is going to magically stick a label on the btrfs code and say "yes, this is now stable and bug-free". Different people have different concepts of stability: a home user who wants to keep their ripped CDs on it will have a different requirement for stability than a large financial institution running their trading system on it. If you are concerned about stability in commercial production use, you should test btrfs on a testbed system under production workloads to see if it will do what you want of it. In any case, you should join the mailing list (and hang out in IRC) and read through problem reports and follow them to their conclusion to give yourself a good idea of the types of issues that come up, and the degree to which they can be dealt with. Whatever you do, we recommend keeping good, tested, off-system (and off-site) backups.

Pragmatic answer: (2012-12-19) Many of the developers and testers run btrfs as their primary filesystem for day-to-day usage, or with various forms of real data. With reliable hardware and up-to-date kernels, we see very few unrecoverable problems showing up. As always, keep backups, test them, and be prepared to use them.

For all practical purposes, btrfs is stable. Everything they say in the long answer basically applies to linux in general (unless you have a support contract with Red Hat or the likes).

Re:Too little, too late...

[ssam]

Why do you need the closed nvidia driver on a server? Nouveau should be fine or even just the vesa driver. (I could say why do you even need a video card on a server, but I guess some folk prefer that to using ssh or a serial connection from a laptop)