Apple Discontinues ZFS Project

Zaurus writes "Apple has replaced its ZFS project page with a notice that 'The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.' Apple originally touted ZFS as a feature that would be available in Snow Leopard Server. A few months before release, all mention of ZFS was removed from the Apple web site and literature, and ZFS was notably absent from Snow Leopard Server at launch. Despite repeated attempts to get clarification about their plans from ZFS, Apple has not made any official statement regarding the matter. A zfs-macos Google group has been set up for members of Apple's zfs-discuss mailing list to migrate to, as many people had started using the unfinished ZFS port already. The call is out for developers who can continue the forked project." Daring Fireball suggests that Apple's decision could have been motivated by NetApp's patent lawsuit over ZFS.

Comments

God forbid...

[Anonymous Coward]

God forbid the summary tell us what ZFS is

Re:God forbid...

[zach_the_lizard]

God forbid the summary tell us what ZFS is

It is a filesystem, available in (Open)Solaris and at least FreeBSD, possibly other BSDs as well. It has some interesting features, which you can check out here. I have heard it claimed that ZFS has good performance, but I have not evaluated any of those claims.

Re:God forbid...

[ozmanjusri]

You have a fan in me

That must sting a bit.

Github project taking up the slack

[KillNateD]

Dustn Sallings put the code on Github and has already hacked some basic Snow Leopard support and a minimal installer:

http://dustin.github.com/2009/10/23/mac-zfs.html

Code's here, fork away:

http://github.com/dustin/mac-zfs

Re:The straight dope

[BrentH]

"The flip side is that I've heard that Apple's file systems team is full steam ahead on their own next-generation file system. And, perhaps not coincidentally, they're hiring." from http://daringfireball.net/linked/2009/10/23/zfs

This is pretty shitty because it'll fragment the momentum ZFS had in being the next-gen ubiquitous file system. When it was clear ZFS wasn't coming to Linux, those guys got btrfs going, now Apple is doing their own, while ZFS obviously will stay around too. Microsoft obviously wasnt on board for any of this, and without the momentum behind ZFS it never will. This nonsense isnt helping, and I think the best Oracle could do it release it under all the licenses that'll get it into OSX/Linux and perhaps even Windows. Can Oracle go over Sun's head on this or Sun==Oracle?

Maybe they should look at HAMMER FS

[Lemming Mark]

http://en.wikipedia.org/wiki/HAMMER

Should be under a suitable license for their usage. It's written for DragonflyBSD which has a funny filesystem driver interface but AIUI the developer had ports to other OSes in mind, so it should still be doable. It can do cheap filesystem snapshots so it would support Time Machine-style operation well. The question is whether it could be adapted to fit Apple's uses well enough. Given one of the linked articles suggests Apple are hiring FS developers my guess would be that they've decided they'd rather build a ground-up filesystem that supports all the (slightly odd set of) features MacOS X wants.

Re:The Reason is Probably Technical

[Anonymous Coward]

T
It makes perfect sense. HPFS is simply far too long in the tooth now.

Did you mean HFS?

HPFS is the old OS/2 filesystem created by Microsoft in 1985 (d. in 1987) and now replaced by NTFS and JFS

Hand in your geek card on the way out

Correction

[toby]

A lot of confusion has resulted from labelling ZFS a "filesystem". It actually combines both volume management and filesystem layers to achieve unique levels of performance, manageability, and data protection. Merits close study, as the concepts of ZFS overtake current best practices, conventional filesystems and RAID. You can get this taste of the future today, if you're using Solaris 10/OpenSolaris/FreeBSD.

Re:Correction

[balbeir]

AIX has had a similar "filesystem" for a long time BTW so it's not such a novel idea

Re:Correction

[BitZtream]

Warning: I have become a ZFS fanboy, my statements may not be entirely rational.

Use ZFS for a week on something like a file server, you'll be more than willing to ignore traditional Unix logic (which I firmly believe in) in order to use the goodness that is ZFS.

I'll admit, I'm a ZFS fanboy now, and I've only used it on FBSD 7.2, which is using a much older version of ZFS (v6, current is 13 I think). In 7.2 its not considered production ready and its still awesome to work with for me.

I shoved 4 PATA and 4 SATA drivers in a case, 2 gigs of ram (really the minimum for FBSD and ZFS on a 64 bit machine, which is where it sings). The 4 SATA drivers are in the zraid vdev, 2 of the PATA drives are in another mirrored vdev, and the final 2 in a second mirror vdev. So all the drives have redundancy and they are all in one big zpool with the total space available as one big chunk.

Now thats all well and good, but heres where it gets awesome. Want more space? Add some drives, create a new vdev, add it too the pool, instant space available. Okay, so thats not that impressive in and of itself. I can also add in a SSD as a 'cache' drive, which the ZFS system will then populate with a read only copy of data that gets accessed often in a random way, for a speed increase for your random IO needs.

Okay, so I've got a few terabytes of data available, but I need a to create a share for backup using TimeMachine on my Mac. Well time machine will be happy to consume all the space on the drive. No problem, create a new mount point in the zpool, limit it to 1TB. It will only consume up to 1TB, IF space is available in the zpool. I can also reserve the space if I want to ensure its there for the backups. You can use the same system for quotas of individual users.

I also have a bunch of ISOs with uncompressed data on them that I mount from virtual machines for various reasons, full of essentially text data. Welp, for that create another mountpoint on the same zpool, turn on compression, now my 5TB of text gets compressed into a few hundred gigs automatically and transparently. Its read only, and very important. I have backups, but they are in another state and transfering them back would be a painfully slow process which would cost me a lot of time. So I set the mount point to read only and set copies=2. Now the mount point is read only, and the data is stored twice, on 2 different vdevs. So not only is the data on a raid or a mirror, its on BOTH. If I was really worried I could set copies=3 and it would be on all the vdevs, so as long as one is usable I have my data, assuming all the vdevs have enough space to store the data. One of them doesn't, so copies=2 is the only useful option to me.

I also have a software archive of all my commercial windows software, I want to keep it safe from any sort of infection or modification, so I set that mount point readonly.

So far, I've done nothing that can't be done already with existing methods, but what I've done it across a common shared data pool. Free space is shared across all of them.

I thought ZFS was a waste of time until I started using it. I am by no means a ZFS master, but I've learned that it can do some pretty powerful things. My setup is small, I only use it at home until FBSD 8 is released, which will have what is considered to be a production ready implementation, but I will be moving to it at that point in our internal office servers.

I know I haven't listed any life changing reasons to use it, but its turned me into a fanboy.

Technically, it IS modular, even if it doesn't have well defined zones. If you look at ZFS in a slight different way, it can be just one layer of the system. You can create virtual devices in a zpool and treat them as a block device (its just another /dev entry). Just to see how it worked, I created a virtual device on top of the zpool, formatted it with UFS, put some files on it, expanded the virtual device, ran growfs and had a larger UFS mount point.

Re:Correction

[TheRaven64]

ZFS is modular, it just doesn't have layers in the same places at the conventional block device, filesystem, vfs layers.

At the bottom layer, you have the pooled storage layer. This is comprised of several modules. The lowest layer is responsible for combining virtual devices into storage pools. This handles things like stripping and mirroring, but it does so with dynamically sized block ranges, rather than static ones (i.e. partitions). On top of this you have the transactional I/O layer. This handles things like checksums, encryption and compression, and provides support for transactions. Every disk access goes via this layer. That means that every single block device access from the upper layers gets transaction support, so you can guarantee that the block devices are always in a recoverable state; every group of low-level I/O operation either worked or failed, it never partially worked. Above this is the adaptive replacement cache, which handles caching and, with the L2ARC can also perform caching in Flash as well as RAM.

On top of that, you have the transactional object layer. While the pooled storage layer works with a flat 128-bit address space, the transactional object layer works with objects and sets of objects. Any higher-level I/O is expressed in terms of transactions containing modifications to sets of objects. This is a very powerful abstraction. An object can be a complete filesystem, some metadata, or a file; there is no difference at this layer. Anything that you can do with one, you can do with any of the others. There are a few other things in this layer, like the ZFS intents log, which handles a higher-level (finer-grained) transactional model.

On top of this is the user interface layer. There are two things here, the ZFS POSIX Layer (ZPL) and ZVOL. The former produces something that looks like a UNIX filesystem, with NFSv4 ACLs and a few other nice features. This is close to a filesystem in the traditional model, but also a bit like a VFS. It provides something that looks like UFS on top of lower-level features, but these are not on-disk structures. ZVOL is simpler; it provides something that looks like a block device. You can export this via iSCSI, for example, and put any kind of filesystem you want into it. Unlike a real block device, this one supports copy-on-write, snapshots, transactions, and so on. You can, for example, have a Windows VM (or real system) with a FAT or NTFS filesystem in a ZVOL mounted via iSCSI. You can snapshot it with the same O(1) snapshots that the rest of ZFS has, and then restore it later. You can also use zfs send and zfs receive to stream the changes across the network to another machine, irrespective of whether you're using ZPL or ZVOL + some filesystem. You can also use ZVOL for things like UDF images for burning, creating a skeleton image, cloning it, and using the native copy-on-write support to make small changes to it.

Re:Correction

[TheRaven64]

Depends on what you need from quotas. From the start, ZFS has allowed volumes with limits. That means that you can create a new volume for each user's home directory and give it a quota. It will then be allowed to grow until this limit is reached. Per use and per group quotas on volumes were only implemented in March, so they aren't available on older versions of ZFS.

Not sure about clustering, but you can export ZVOLS via iSCSI, so if you wanted to distribute the load that way, you can.

Your third point makes no sense. VDEVs can be combined in arbitrary ways, including hierarchically, in the pooled storage layer. Perhaps you are confusing storage pools with VDEVs. With the L2ARC, you can use faster disks (e.g. flash) as a cache for the slower ones.

Re:The straight dope

[setagllib]

You really need to subscribe to the mailing list. The rate of development is only growing -- it's just now moved on to a lot of smaller features and improvements, now that most of the work is already done.

Re:With SSDs, who needs it?

[supertux]

People don't do enough research and buy the wrong stuff. If you need fast writes, you get the Intel X25-E. If you need fast reads, the Intel X25-M is fine. If you need the SSD to take a punishing amount of writes over the years, you get the X25-E again. If you aren't planning on punishing it with writes, the X25-M is again fine. If you need cheap, then you get an extra helping of crappy write I/O. :-)

If you want to have monstrously fast storage, you build a raid with zfs, and use one or two X25-Ms for the L2ARC cache and a mirrored (through zfs) X25-M pair for the zil cache.

I'd rather use SSD to supplement traditional storage rather than to run straight off it. But that said, I have been running my mythtv box off of an 8GB Transcend compact flash for the OS for over two years now with great results. It is a gentoo system, so I compile stuff on it all the time. Of course it has a mysql database that gets updated daily because of the schedules it pules down for mythtv. No problems there, and no regrets.

But more to your point, the problems that plagued crappy SSD controllers and designs are being worked out, and probably somewhat soonish won't be a relevant issue anymore for most people.

Re:The Reason is Probably Technical

[4iedBandit]

I'm sure it will, but I'm afraid that doesn't mean that made it practical for Apple to integrate into OS X or that it fitted the use cases they needed for many desktop scenarios.

Um, the technical work was already done. It could have shipped with Snow Leopard. Again, the reason it didn't has nothing to do with the technical feasibility of it.

Ultimately, the only way to deal with silent data corruption or 'bit rot' is to have multiple levels of redundancy several times over for your data - which ZFS has and deals with. No desktop Mac can ever have that.

Why? Because you say so?

Anyone who thinks that is anywhere near being practical to deal with on a desktop system is an idiot

While I may be an idiot, you have to convince me that ZFS is not practical for a desktop. Again, just because you say so is not reason enough. I stand by my statement that ZFS is the only file system with enough benefit to make me explicitly choose it for building servers. You may argue that there's a difference between a server and a desktop but those really are nothing more than abstract concepts. A file system that has too much overhead for my desktop has too much overhead for my servers. Performance matters. ZFS may not be the fastest, but it is no slouch either and the other benefits it brings to the table far outweigh miniscule performance concerns.

By no stretch of the imagination does ZFS handle this 'magically'. There is a severe price to be paid.

What exactly is this severe price? Can you spell it out? Exactly? "Any sufficiently advanced technology is indistinguishable from magic." In that respect yes I will say it is magic because it is head and shoulders more advanced than anything else I've had the pleasure of working with. File systems have not had this kind of improvement in decades.

I'm afraid that hardware, bad sector and disk issues are far, far more prevalent problems than data corruption at an OS level...but I'm afraid it's just not a primary concern for everyone else or for those developing desktop operating systems.

How do you know? It's not a significant problem till the data you need is unavailable when you need it. At home my own modest media library sits on just 500 Gig with no guarantee that any of it will still be whole in 6 months. Sure I back it up. Routinely. But until you access the file you don't know if it's been corrupted. Then how long as it been corrupted? Do your backups go back far enough to compensate? Yes you can checksum everything routinely and maintain a database of checksums to validate file change. Part of the beauty of ZFS is it does this with every thing you put in it, at the block level, and it validates the checksum every time you read the data. If a block fails the check, it not only sends you the valid block from the mirrored copy (You do have redundancy right? Even ZFS won't save you if you only have one copy.) but also replaces the bad block with a copy of the good one.

Storage capacity is skyrocketing. Going to backup to fix problems is a real problem in itself. Are the tapes on-site? Do we have to go to the vault to find an uncorrupted copy? Did the media pool get recycled and now there is no uncorrupted copy? Do you enjoy explaining to an executive why the data they want is unavailable despite spending millions on enterprise class storage and backup solutions? The problems of enterprise storage are becoming problems of home users. I have three terabytes of storage just to backup my home system in a replication layout I'm okay with, but I really would have loved the protection ZFS offers against bit rot to top it off. Stick your head in the sand if you want, but I consider my data and it's availability a little more important. ZFS handles it elegantly, in the background, with negligible performance hit.

Likely in the works

[Anonymous Coward]

I'm sure Apple is planning something, OS X's file system is showing its age. Ars had a very good article about it here: From BFS to ZFS

Re:This is devastating...

[mysidia]

Because 'df' shows mounted filesystems in the more traditional physical fashion. It's best to inspect dataset properties to see their status

zfs get available pool/path/to/dataset

zfs get all pool/path/to/dataset | grep used

zfs list

Another thing that can throw you off, if you have compression enabled (lzjb) and do dd if=/dev/zero of=blah.txt

And ^C it later... you can have a 50gb file using 0 bytes on disk

du -m blah.txt

Will show 0 bytes, but ls -l blah.txt will show its logical size.

It drives people unaware of zfs compression nuts.

Yes, there's a learning curve to some of the things in zfs... but it's all so totally worth it.

There are some things that are harder in zfs than otherwise. Most things are a hell of a lot easier, especially volume management with pools VS traditional volume managers.

ZFS is overhyped and not a good fit for desktop OS

[Anonymous Coward]

If you take a look at look at the features and limitations of ZFS: http://en.wikipedia.org/wiki/ZFS#Limitations

You can see that it has missing functionality such as encryption support and it does not seem to be a good fit for systems with removable media. It is also counterintuitive for the average user to grasp the concept that drives are part of a "pool". Finally, it is prone to fragmentation.

Given all of the problems and lack of completeness, I cannot understand how anyone in their right mind would advocate using ZFS for any system at the time. I think that ZFS, should it continue to be developed, could be a useful FS for a SAN device but I don't see it as ever being a good fit for a desktop OS.

Re:This is devastating...

[mysidia]

I tried the zfs commands you suggested but there aren't any of those available in my path,

The two programs used to work with zfs are 'zfs' and 'zpool'. Both of them are normally located in /usr/sbin

If your PATH is broken so you can't easily access the tools, it doesn't matter what filesystem you're using, you'll have a bad time. I would suggest adding /usr/sbin to your PATH if it's not, or else use "/usr/sbin/zfs list" etc.

the user's, life is made harder?

The user's life is not made harder. It's no harder to learn "Zfs list" than to learn "df"

What is all so totally worth it? I haven't seen any advantages of zfs over hfsplus.

Copy on write filesystem, performance is excellent..

Unlimited impact snapshots which have no I/O performance impact, and has the ability to rollback to most recent snap, clone snapshot, etc; makes backup, replication type tasks easy. The ability to implement transactional unbreakable system upgrades, ala apt-clone.

RaidZ, to protect against drive failure

Ditto blocks and checksums to ensure data integrity, protect against silent data corruption, heal bad disk blocks.

No need to 'fsck'

No need to decide the size of each file system at creation time, 'quotas' are soft and can be changed, instead of having to re-format/fdisk to change sizes of a dataset.

No need to manage individual disks. No limits on number of disks imposed by the filesystem layers.

No arbitrary limits in zfs. ZFS can scale to (in theory) 256 quadrillion zettabytes. Other filesystems have a max file size limit, and a max filesystem size limit.

Sharing a folder is a simple invokation of a "share" command.

Re:The straight dope

[Anonymous Coward]

That's EXACTLY right. Sun's management loved the idea. Sun's lawyers ruined it for everyone.

Worse, ZFS was basically ready to ship. Unfortunately, the latest version available publicly is over a year old.

Fortunately, any developer who got ANY developer seed with any newer version of ZFS (assuming newer versions were seeded) has a legal right under the CDDL license to send a letter to Apple and demand a copy of newer sources, and they cannot legally keep it under an NDA. I would encourage members of the Apple developer community to do so. It would probably save the newly created public fork's developers a lot of time.

Sincerely,
A friend of ZFS

Re:Did ZFS ever support FIleIDs?

[Anonymous Coward]

Case insensitivity support was specifically added to later editions of ZFS to help out Apple.

Re:The Reason is Probably Technical

[jcr]

They have core developers from the HFS+, BeFS, and ZFS teams, not to mention all the work they've done in-house on the other filesystems they support.

-jcr

Re:With SSDs, who needs it?

[MartinSchou]

How expensive are they?

If you're looking at hundreds of thousands of writes a day to your database, you really only need somewhere between 10 and 100 IO/second (there are 86,400 seconds in a day). Most hard drives handle that somewhat decently, especially if you use a good RAID configuration.

Looking at 100,000,000 updates a day (1,158 writes/second)? Intel's X25-M is rated at more than 4 times that

Iometer* Queue Depth 32
Random 4 KB Write:
80 GB - Up to 6.6 K IOPS
160 GB - Up to 8.6 K IOPS

Let's compare that to a 15k.2 Seagate Savvio harddrive. Oh, right, they don't list their IOPS ratings. Let's look at what they do have though:

Not including controller overhead (msec):
Single track, typical: 0.2 (read) 0.42 (write)
Average, typical: 2.9 (read) 3.3 (write)

Intel lists these figures:

Latency Specification:
- Read: 65 micro seconds
- Write: 85 micro seconds

In other words, for a single track, the Intel drive will be almost 5 times as quick to start the write, and on average the Intel drive will be 38 times faster.

Or looking at it in another way, the absolute best case scenario where we simply ignore actually writing something, the Seagate drive can achieve 205,714,286 write operations per day (86,400 seconds/0.42 milliseconds). The Intel drive will hit 1.016.470.588.

While I can't find anyone benchmarking Intel's SSD offerings directly against the Savvio, I can find a mix of tests. From Tom's Hardware we see that SAS drives tops out at about 400 IOPS for any given task.

Using Tom's Hardware for a comparison, their review of the X25-M had it bottoming out at around 900 IPOS, making it perform 225% better at its worst, compared to the SAS drive's best.

Prices:
Newegg.com doesn't have the Savvio, so I'm using Google instead:
Seagate Savvio 15k.2 146 GB edition: US$ 226.44 or US$1.55/GB
Intel X25-M 160 GB edition: US$ 439 or US$ 2.74/GB

Considering the performance advantage of at least 225%, you'd have to spend at least US$ 509.49 just to get the same kind of performance as you'd get from the US$ 439 drive from Intel. And that's just their mainstream edition. AND we're talking SSD's worst case scenario vs. SAS's best case scenario. Realistically we're talking much greater advantages for the SSD.

And you keep talking about "commodity SSDs" but refer to datacenters. A commodity harddrive is a 7.200 RPM 8 MB SATA drive, and they aren't suitable for a datacenter either. Duh! So why the fixation of comparing commodity hardware from one technology to enterprise hardware from another? Stop buying commodity hardware for your datacenter needs.

Sorry, the FACT that SSD has had write performance problems, wear leveling, and write endurance issues is by no means 10 year old information.

And yet you haven't caught on to the fact, that this isn't that big of a problem. Anandtech wrote an excellent paper on write performance problems, and his benchmarks are based on used drives (the drive has to perform deletes before writing), and he got these performances:

4KB Random Write Speed
Intel X25-E 31.7 MB/s
Intel X25-M 23.1 MB/s
Western Digital VelociRaptor 1.63 MB/s

The VelociRaptor i