Slashdot Mirror
Oracle Engineer Talks of ZFS File System Possibly Still Being Upstreamed On Linux (phoronix.com)
New submitter fstack writes: Senior software architect Mark Maybee who has been working at Oracle/Sun since '98 says maybe we "could" still see ZFS be a first-class upstream Linux file-system. He spoke at the annual OpenZFS Developer Summit about how Oracle's focus has shifted to the cloud and how they have reduced investment in Solaris. He admits that Linux rules the cloud. Among the Oracle engineer's hopes is that ZFS needs to become a "first class citizen in Linux," and to do so Oracle should port their ZFS code to Oracle Linux and then upstream the file-system to the Linux kernel, which would involve relicensing the ZFS code.
One nice thing about ZFS not being in upstream is that it is currently maintained and updated separate from the Linux kernel.
Now, it would be nice to relicense ZFS under GPL so that it can be included in the kernel. But this should wait until the port is a bit more mature. Right now development is very active on ZFS and we have new versions coming out every few weeks; having to coordinate this with kernel releases will complicate things.
All this said, relicensing ZFS would definitely help Oracle redeem themselves a bit. After mercilessly slaughtering Sun after acquiring them, they have a long way to go to get from the "evil" side back to the forces of good.
ZFS wants to live in a fairly specific configuration. It wants a bunch of drives, a bunch of memory, and not much competition for system resources.
Except for the part where it works with 2 drives, on a system with 4GB of RAM and under constant heavy load just fine.
> The problem with ZFS on Linux is that some aspects of it are redundant with the kernel.
Probably ALL aspects of it. Linux already has a raid implementation in-kernel. It already has filesystems. It already has multiple volume managers, which handle whichever type of snapshots you prefer. It already has IO schedulers. ZFS, or rather something that looks just like it, can be implemented as a few configuration lines for pre-existing Linux components.
Because Linux normally lets you use your choice of file system on top of your choice of volume manager, on top of whichever RAID implementation you choose, with your choice of IO scheduling options, ZFS isn't exactly the best fit. ZFS mashes all those different things into one big blob. That's not really how Linux is designed.
That's the same issue as systemd - it may (or may not) be a good init system. It may or may not be a good logging system. It may possibly be a good DNS server (probably not). But it can't seem to decide wtf it is.
Just as this article popped up I was assembling a JBOD array (twelve 4TB drives) for a new data center project, my first in quite a while. Also self funded so I don't have to defer to anyone in decisions.
When I started I did a bit of reading trying to decide what RAID hardware to get. To make a long story short once I read the architecture of ZFS and several somewhat-polemic-but-well-reasoned blog entries I decided that is what I wanted.
Only two months ago I had an aged Dell RAID array let me down. I have no idea what actually happened, but it appears some error crept in one of the drives and it got faithfully spread across the array and there was just no recovering it. If I didn't have good backups that would have been about 12 years of the company's IP up in smoke. I just thought I'd share.
So I ended up as a prime candidate (with new found distrust for hardware RAID) to be a new ZFS-as-my-main-storage user. I've just recently learned stuff that was well established five years ago and I can't understand why doesn't everybody do it this way.
Wow. snapshots? I can do routine low-cost snapshots? Data compression? Sane volume management? (I consider LVM to the the crazy aunt in the attic. Part of the family but ...) Old Solaris hands are probably rolling their eyes but this is like mana from heaven to me.
Given the plethora of benefits I am sure the incentive is high enough to keep ZFS on Linux going onward. ZFS root file system would be nice but I am more than willing to work around that now.
One nice thing about ZFS not being in upstream is that it is currently maintained and updated separate from the Linux kernel.
And that's actually a huge problem that makes it a major obstacle to its upstream adoption.
Mainly due to code duplication.
ZFS (and its competitor BTRFS) is peculiar, because it's not just a filesystem. It's a whole integrated stack that includes a filesystem layer on the top, but also a volume management and replication layer underneath (ZFS and BTRFS on their own a the equivalent of a full EXT4 + LVM + MDADM stack).
That is a necessity, due to some features in these : e.g. the checksuming going on in the filesystem layer is also useful to determine correct copies in case of bitrot in the replication layer.
But how this is handled is the big difference between ZFS and BTRFS.
ZFS on Linux just packs all the needed bits together with it.
It comes with its own volume management and replication code.
That is a duplicate of functionnality existing elsewhere in the kernel.
And duplication is always bad for maintenance.
BTRFS being developped on Linux tries to leverage as much as possible :
- the Zstd compression currently being introduced to BTRFS, uses the same routines as the Zstd compression being introduced into the kernel loader : both leverage the in-kernel compression facilities of the crypto modules
- the device mapper facilities are used by lvm, mdadm and dmraid but also by btrfs. There was a plan to develop code to support more than 2 parity blocks (more than RAID6), that would have been beneficial to both btrfs and mdadm.
That's why developers complain of boundaries/layers violation with ZFS but not about BTRFS.
ZFS comes with its own tangled mess of layers, BTRFS is just a wrapper around facilities already existing in-kernel.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
In contradistinction ZFS takes a holistic, unified approach:
* Volument Management <--> File Management <--> Block
{...}
ZFS works because it intentionally "Flattened the stack" -- Yes, this runs counter to the layered Unix approach
The problem is that ZFS implement this by rolling everything in the same "rampant layering violation" package.
It is one single "flattened stack".
On the other hand, BTRFS shares as much code as possible with in-kernel facilities (it leverages "device mapper" routines that are used also by lvm, mdadm, mdraid, etc. it leverages in-kernel compression routine that are also used by the kernel loader and the crypto module, etc.)
It's not as much a "rampant layering violation" as a wrapper against layer facilities already existing in kernel.
-- but sometimes that is NOT the best design decision.
So basically, the problem isn't the overall design, but that actual code re-use vs. re-write.
Meanwhile Oracle keeps flailing about with Btrfs.
Btrfs works. It's in kernel, It's a first class filesystem in opensuse, and its copy-on-write facilities are extensively used for versioning with snapper.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]