Apple Removes Nearly All Reference To ZFS

Roskolnikov writes "Apple has apparently decided that ZFS isn't really ready for prime time. We've been discussing Apple/ZFS rumors, denials, and sightings for some years now. Currently a search on Apple's site for ZFS yields only two hits, one of them probably an oversight in the ZFS-cleansing program and the other a reference to open source. Contrast this with an item from the Google cache regarding ZFS and Snow Leopard. Apple has done this kind of disappearing act in the past, but I was really hoping that this was one feature promise they would keep. I certainly hope this isn't the first foot in the grave for ZFS on OS X."

Comments

Well fuck it, we're going to 128 bits

[QuantumG]

cross-meme joke completed.

Larry effect again?

[G3ckoG33k]

Could this be a Larry effect?

Re:Larry effect again?

[rsmith-mac]

Larry Ellison, the Oracle CEO. Oracle just recently purchased Sun (makers of ZFS), so the OP is postulating whether Apple pulling ZFS is a product of Cisco not working on/opening up ZFS to Apple like Sun did.

Re:Larry effect again?

[ildon]

I'm thinking Balki effect.

Re:Larry effect again?

[mrsteveman1]

No, Google "2 CEOs, 1 filesystem".

Re:Larry effect again?

[Anonymous Coward]

One
Raging
Asshole
Called
Larry
Ellison

Re:Larry effect again?

[ThePhilips]

It's actually more interesting than that.

Oracle currently sponsoring development of Btrfs for Linux, which is equivalent to recently acquired ZFS. I wonder if that played any role.

P.S. Though my personal opinion is more pragmatical. ZFS doesn't seem to be extensible and obviously doesn't support all featlets of HFS/HFS+ (streams, aliases, case insensitivity) which are required for Mac OS and its applications. I guess that should have been a major bummer. Meaning that even if Apple would add ZFS support, it would very likely have different name and it would be incompatible with Sun's ZFS.

Re:Larry effect again?

[AttilaSz]

It's widely known that Steve Jobs and Larry Ellison are good friends, see this from http://en.wikipedia.org/wiki/Larry_Ellison

"On 18 December 2003, Ellison married Melanie Craft, a romance novelist, at his Woodside estate. His friend Steve Jobs, Apple, Inc's CEO, was the official wedding photographer."

So, no, Larry's company becoming ZFS owner ain't the reason Steve's company would drop it.

Death knell

[siloko]

I certainly hope this isn't the first foot in the grave for ZFS on OSX.

More like the last nail in the coffin . . .

Re:Death knell

[Macrat]

Or it's not ready for a consumer OS yet.

Re:Death knell

[udippel]

More like the last nail in the coffin . . .

Which is what I hope. Having tried forth and back over the last years, trying to convince myself, that it would fulfill its promises (and it promises a lot! and all beautiful things) one day or another.
It simply didn't. Which is a shame, since if it did, ZFS would be last file system mankind would have ever needed.
But even in 2009, it suffers from serious problems, just read the ZFS list in OpenSolaris. Basic things, that is.
Like boot corruption; like unusable system, if you pull the power, and pull the power again while it is restarting; Like slowness under specific conditions; like rendering the file system unbootable, reproducibly, when using a specific setup of snapshots.
The latter, not addressed on the mailing list, killed our interest immediately.
Not to forget some arrogance of the Sun engineers when it turned out that you cannot simply unplug a USB-drive. And it won't be enough, to umount it, neither. If you want the data to be there, sure, after the removal, you have to export the drive. Now tell this to Aunt Tilly. Or me, when I stumble over a USB-cable and out it is. And my data, as confirmed on the mailing list, potentially gone forever; with, confirmed, no tool available for recovery.

My last hope for it, had been that the engineers at Apple were able to give it the life-line needed to provide reliable Time-Machines (the snapshots of ZFS are just perfect therefore), but obviously, they have given up just as well.

I bet that something like ZFS will resurrect, one day or another. It simply has to. But ZFS as of today is more like Leonardo's drawings of a copter, compared to an Apache.

Re:Death knell

[Anonymous Coward]

What ZFS does have that typical Apple Consumers would like to see it on desktops and not just on servers?

Almost every ZFS oriented discussion, there just comes one point up. ZFS is not miracle what is not possible to gain already with other kind setup with RAID and other filesystem.

Re:Death knell

[KonoWatakushi]

This is almost entirely nonsense. I have been following the zfs-discuss list for years, and almost no one has lost data. There have been a few bugs which could in rare cases render your data inaccessible, but they almost always have workarounds, and do get fixed.

The data loss and corruption that the parent is talking about is the fault of crap hardware. In almost every case, USB is involved, or more rarely the lack of ECC ram. It is true that ZFS is less tolerant of bad hardware. Note, faulty good hardware is not considered bad; that is reserved for garbage which (for instance) lies to the OS about flushing the disk cache. With such hardware, it is impossible for any filesystem to function reliably.

USB and Firewire bridges are notorious for this. If you care about your data, you should run the other way if you happen upon one. ZFS works great on good hardware though. With directly attached disks and ECC ram, there is no cause for concern.

Re:Death knell

[speedtux]

The data loss and corruption that the parent is talking about is the fault of crap hardware. In almost every case, USB is involved, or more rarely the lack of ECC ram. It is true that ZFS is less tolerant of bad hardware.

What good is a fault tolerant file system if it isn't tolerant of faults?

With such hardware, it is impossible for any filesystem to function reliably.

Quite incorrect.

USB and Firewire bridges are notorious for this. If you care about your data, you should run the other way if you happen upon one.

Well, golly, those only happen to be the way 99.999% of Apple's customers attach exernal drives, not to mention 99.9% of all of the rest of the world.

Re:Death knell

[ishobo]

Windows and Solaris fall into the first category, UNIX and Linux into the second.

Are you sure you do not want to correct that statement?

Re:Death knell

[udippel]

I have been following the zfs-discuss list for years, and almost no one has lost data.

That's not good enough for the likes like me.
For the rest of your post, I am simply too lazy to prove you wrong. For a beer each I could fiddle out those that were confirmed to lead to data loss, including unrecoverable data loss, as I mentioned in my post.
But I won't do this (except for that beer each), because you know that best yourself:
The data loss and corruption that the parent is talking about is the fault of crap hardware. In almost every case, USB is involved, or more rarely the lack of ECC ram
Because this is exactly, word for word, the usual excuse given in the mailing list.
And I didn't add the one in my original post, when it was 'confirmed' that you need RAID if your data are valuable to you; and now, read this in bold: irrespective of hardware failure. I for one accept the need for RAID, in case of a hard disk really and effectively dying. Not for manhandling the data. Read the postings carefully.

Of course, the other person answering your flawed arguments about 'crap hardware' is right to the point: What good is a fault tolerant file system if it isn't tolerant of faults?
May I remind you, the premise and promise of ZFS was the atomic write, the always consistent state on the drive. I do think and believe this is true, and all blocks are either written and confirmed or just not. As far as I can make out, the problem has only been shifted: to the problem of metadata. Again, refer to the mailing list. Those exist in four-fold. Why? It seems the consistency of blocks on the drive being guaranteed, the layer of actually having the links to those correct data is more vulnerable. Think of a pool: if you jank the structure of a pool by janking a USB, you have 100% correct data (contrary to any other file system, I agree), but alas no more structured access to reassemble them (compared to inodes).

(The mods opting for 'informative' of your post obviously don't read the ZFS mailing list, and nobody blames them.)

Re:Death knell

[KonoWatakushi]

What good is a fault tolerant file system if it isn't tolerant of faults?

This is not about faults; ZFS handles those fine. This is about hardware which behaves badly.

You should not talk so authoritatively when you are so obviously ignorant of the subject. What you are implying, is that a filesystem does not care about the ordering of writes, and that is absolutely absurd. The ordering of writes is more critical for copy on write filesystems like ZFS, but in neither case is your data safe on bad hardware.

Like you point out, there is lot of bad hardware out there. What you overlook is that existing filesystems have no facilities to catch, much less correct such errors or corruption; that is why it is called silent data corruption. Even filesystems like HFS+ only journal metadata, so this is a lot more common than you realize.

ZFS can be improved on misbehaving hardware, but that still won't fix bad hardware. Those improvements will only allow the possibility to recover to a consistent state, but data will still be lost.

Re:Death knell

[ThePhilips]

Of course, the other person answering your flawed arguments about 'crap hardware' is right to the point: What good is a fault tolerant file system if it isn't tolerant of faults?

Or in layman's terms: if shit happens, system shouldn't help to spread it, but to localize it.

(The mods opting for 'informative' of your post obviously don't read the ZFS mailing list, and nobody blames them.)

What you describe is a standard problem faced by all journaling and/or distributed file systems. Or for that matter any distributed ("shared data") system. You simply cannot guarantee anything (efficiently) when many asynchronous agents are involved. And it all depends where would designers cut the compromise with inefficiency (force sync of all the agents).

Judging that it took Veritas a decade to make such system (at least from reading their 5.0 papers I get the impression), I think ZFS needs much much more time to mature.

Re:Death knell

[wereHamster]

Every disk will corrupt eventually, it's just a matter of time. Not even the best hardware will help you there. So the question is, how well does the filesystem catch these errors and correct them. It turns out, ZFS is really bad at this, as it can get into a state where you can't even import the pool (where zpool either stops with an error and in worse cases causes a kernel panic). There have been numerous bug reports on the zfs mailing list and the opensolaris bug tracker. So far nobody seems interesting in fixing those. My pool got corrupted in such way. I had to manually poke around the filesystem and invalidate metadata until zpool was able to import the pool. Something that a 'fsck' could have easily done, but Sun refuses to create such tool because, according to them, ZFS is robust enough. All credits go to this guy who had the idea to invalidate the uberblocks directly on the disk: http://opensolaris.org/jive/message.jspa?messageID=318457#318457

Re:Death knell

[JoeMerchant]

In 2006 I was given a MacBook Pro to work with... it had been over 10 years since I computed on the fruity side and I decided to trust that Apple had "done the right things" with the features they included. One feature of interest was their "FileVault" which gave a "here, try this:" howto on encrypting your home directory. I thought with all the "just works" hype that surely Apple would have made such a thing secure and reliable... it wasn't 2 weeks before my MacBook Pro was useless, had to re-install OS-X from DVD, as confirmed by an Apple Tech support guy who actually sounded like he knew what he was talking about... see, in 2006 the MacBook Pro wouldn't always shut down cleanly, so a hard power-off was required every couple of days... this hard power off glitched the FileVault, rendering everything in it (my one and only home directory on the system), inaccessible, and therefore the system unbootable past the login prompt.

Lessons learned:

Even today, encrypting files necessary to boot the system is a bad idea - the promises of ZFS would cure this, but only if they are true.
Always create a back-up account, or three, so you don't have to brain-wipe the system when your primary account goes dead (which mine never has since then, of course it is also not encrypted anymore).
"Just Works" is fanboi speak for "I really like my shiny computer and I'll say anything to make you think it's better than yours.", OS-X is on-par with other modern OSs in terms of reliability and usability, at least in my experience of the last 3 years, they (OS-X, Vista, ubuntu) each have strengths and weaknesses, but overall, OS-X isn't a shining star in any area that matters to me.

If ZFS were "ready for prime time" I expect that it would be in ubuntu already. If I were in corporate cost cutting strategy at Apple development, I'd say "let the Linux world sort that one out" and pick it back up if/when it is ready for prime time. Because, if Apple sorts it out on their own nickel, they'll only have the geek-head advantage for a little less than a year before useful ZFS is picked up by the Linux world, and I doubt that one year of geek-head advantage translates into enough extra shiny boxes sold to cover the (apparently high and growing) cost of ZFS development.

Re:Death knell

[toby]

What ZFS does have that typical Apple Consumers would like to see it on desktops

Pretty much all of it applies equally to consumer systems.

ZFS is not miracle what is not possible to gain already with other kind setup with RAID and other filesystem

You need to study ZFS more, as you clearly know little about it. Almost no RAID systems can do what ZFS does. Hints: end to end checksumming; self-healing; copy on write; ...

Hint: The extra capability largely comes from integrating both the "filesystem" and "volume manager" layers, which are separate modules in traditional configurations. Calling ZFS a "filesystem" seems to mislead a lot of people that it can be compared to other "filesystems"; and the fact that ZFS implements RAID-like redundancy leads people to think that it can be compared to other "RAID" systems. Sure, it can be compared, but conventional systems will generally lose (notably in data integrity, but also in performance, manageability, etc).

Re:Death knell

[wereHamster]

Even if you have a 100% bugfree drive firmware, silent data corruption is still possible (resilience against silent data corruption is one of ZFS's selling points!). Filesystems that can't handle that simply have no place in todays world. The problem is that one flipped bit can cause ZFS to think that the whole pool is unusable - even though it keeps redundant copies of the metadata which it then completely ignores! What for does it keep the copies then?

Re:Death knell

[Charan]

What good is a fault tolerant file system if it isn't tolerant of faults?

Any time you read about a product that guarantees perfect fault tolerance, there is always a list of constraints that must be met for that claim to hold. You probably won't ever see this list marketed, but it's there somewhere.

I haven't looked into this, but it sounds like ZFS is fault-tolerant given a system model where data can change once it's on-disk, but otherwise system components are fail-stop. So if you ask a hard disk to perform a write barrier and flush its data to disk, the disk will either do so and report success, it will issue an I/O error, or it will catch on fire. Any way, ZFS will handle the situation correctly.

Of course, the immediate next question is whether that failure model is realistic. Turns out it isn't. Some hardware will report 'write barrier complete' when it still has unwritten data in its buffer.

If you can't count on your hard disk to flush its caches or even order writes correctly, I don't think it's possible to build any fault-free file system on top of the drive.

Re:Death knell

[Sancho]

The problem is that one flipped bit can cause ZFS to think that the whole pool is unusable - even though it keeps redundant copies of the metadata which it then completely ignores! What for does it keep the copies then?

Where are you getting this from? It's not in the link you posted.

In the link you posted, the admin found three uberblocks (there are supposed to be four). ZFS correctly made multiple uberblocks, per design. It appears that all three were corrupt. Why? Who the hell knows. Could be that the disk was going bad. Could be the problem I mentioned. Could be a ZFS bug. There's not enough information in that thread to say with certainty.

Now I'd say that ZFS "isn't ready for the desktop" no matter what the reason for the faulty uberblocks. I'm not supporting ZFS by disagreeing with what you're saying--rather, I'm railing against drives that do bad things and developers not being willing to work around them. One of these is the fault of Sun--and it's a major one. The ability to recover from hardware misbehaving is important.

However, the posting you're discussing doesn't have anything to do with "silent corruption"--it's explicitly about an unexpected halt, which causes highly unpredictable results when using drives that misbehave. Hence my bringing up that possibility in the first place. As I said, I don't know how the blocks got corrupted--but neither do you.

Take it from another perspective. If all of the copies of the superblock on an ext3 filesystem were simultaneously corrupted (it doesn't matter how), what would happen? Won't the drive fail to mount, or even fsck? (fsck requires a good superblock, I think.)

Re:Death knell

[diegocgteleline.es]

I have been following the zfs-discuss list for years, and almost no one has lost data.

That must be a different list than the one I've been reading....

http://mail.opensolaris.org/pipermail/zfs-discuss/2009-April/027748.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-April/027765.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-January/025601.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-March/027629.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-March/027365.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-March/027257.html

Re:Death knell

[wereHamster]

In the link you posted, the admin found three uberblocks (there are supposed to be four). ZFS correctly made multiple uberblocks, per design. It appears that all three were corrupt.

ZFS keeps a history of the last 256 uberblocks in four different places in the pool. So even if all copies of the most recent uberblock got corrupted, it could still fall back to one of the older ones. You'd maybe loose the last few minutes of work, but that's not nearly as catastrophic as loosing the whole pool. It could fall back, but it doesn't, it rather panics the kernel. This is where a userspace fsck would help, for examply by giving you the choice to safely invalidate the last uberblocks. I was not feeling very comfortable when I wrote the dd/ud script that automated that task, but I had nothing to loose at that point.

The silent corruption was just an example. It doesn't matter what causes the corruption. But _if_ you end up in a situation like the admin or me, you have to resort to such ugly tricks to recover your pool. And that is something I'm not willing to accept on a production system - or any system at all for that matter.

Perhaps it will BE ZFS just not BE CALLED ZFS

[KenCrandall]

WIth the impending purchase of Sun by Oracle, I'm thinking it could be one of 2 things:

1) ZFS will be killed and/or de-emphasized and/or re-licensed in such a way that Apple is not comfortable/happy with putting it into Mac OS

2) It will still be ZFS just not called ZFS anymore (either re-branded or forked by Apple or re-named by Oracle/Sun)

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[Bill, Shooter of Bul]

1) Oracle hasn't publicly said anything of that nature, nor is even any rumors to that effect.
2) They aren't mentioning the features that zfs provides under any kind of name

Most likely, they've been focusing too much on the embedded space with the iphone and didn't have the man power to integrate a complex third party FS into their OS. As it was only going to be for the OSX Server for "production servers", they probably thought that was the easiest thing to drop. I mean, lets be honest no one really uses OSX Server for anything really mission critical that relies on it for the kind of storage capabilities ZFS would provide. Do they? Feel free to correct me with real world usage senarios of OSX Server ( I haven't heard of much).

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[beelsebob]

I'm fairly confident of what it is, having actually used zfs on OS X.

1. The implementation still has some major bugs -- I managed to get a kernel panic with it just by writing to a raid-z.
2. There are some unresolved issues just with the way zfs behaves, for example, pulling a USB device with a zfs volume on it *must* cause zfs to shit its pants, because it's guarenteeing that writes to it will work.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[MichaelSmith]

At my workplace we migrated to a brand new sun NFS server with ZFS and hit a critical bug in the first two weeks. If Sun can't get it right I don't expect others to.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[isorox]

I mean, lets be honest no one really uses OSX Server for anything really mission critical that relies on it for the kind of storage capabilities ZFS would provide. Do they? Feel free to correct me with real world usage senarios of OSX Server ( I haven't heard of much).

I guess XSans may use it. I don't know much about them to be honest, another department at work has a small one for FCP editing. Seems to me that it's the same as any old san.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[MichaelSmith]

Its an nis + automount + zfs + nfs problem. I am a user of the system (not an administrator) so my information is incomplete. Basically we lost the ability to export deep subdirectories (say /path/to/user) from our zfs file system (which is physically on a raid array) via nfs.

We retained the ability to export /path/to as a workaround. The way nis (yp) works in our setup is that it exports individual user directories as required.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[Anonymous Coward]

Mac OS X Server has a few features that are hard to replicate well on other servers, basically coming down to specific Mac management (Open Directory, NetBoot, Software Update), and in particular AFP file services. There are a lot of design/production companies out there with a lot of Macs who need a reasonable amount of storage, and AFP still tends to work better for Mac clients than things like SMB. We've got a few clients with a few hundred Macs and and ZFS would have been a good additional option to have for backend storage. The snapshot and scrub features alone would be a big benefit.

Xsan is great for certain situations but Apple's tools tend to target that towards video production, and not everyone needs or can afford a full SAN.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[prockcore]

The iTunes store uses Akamai. So it uses Linux, not OSX at all.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[LKM]

Apple uses Akamai for mirroring some of their stuff. They use Xserves as the main source.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[totally bogus dude]

I wouldn't think Akamai would be doing any of the actual work behind the iTunes store. I seem to recall they do have that capability, but it would be really hard to take advantage of unless you designed for it from the start, and even then I doubt anyone, especially a company as large as Apple, would be happy to give their content distribution network access to any of their actual user data.

Our website is served by Akamai as well, but nearly all the content is served by Windows web servers. If you do a simple GET and the page is in the cache of the Akamai server you're using, then you could maybe say it was served by Linux or whatever. If you do a search or anything that requires actual work, your request will be getting funneled back to our Windows servers.

I would say it's extremely unlikely iTunes works any differently.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[beelsebob]

Sure they have http://zfs.macosforge.org/trac/wiki#ZFSDocumentation you just don't know where to get apple's current head of their zfs implementation.

Re:Perhaps it will BE ZFS just not BE CALLED ZFS

[carton]

I think NFS is a good alternative to AFP. I use it at my Mac shop, and it works really well once you figure out the automounter tricks, which in short is, on Mac OS X 10.4 and 10.5 the automounter works oddly much better if you use 'net' and let it pick where to mount the share.

Once you get that worked out, Macs get along quite nicely with standard NFS servers which gives you a huge complicated market from which to draw.

There are some huge high-performance vendors in this space too like Bluearc and Isilon, and in addition to the already pretty intimidating (separate data/metadata cluster) NFS to SamFS/QFS gateway above, Sun is working aggressively (albeit slowly) on a new generation of cluster-backed (high-availability and not-limited-to-the-bandwidth-of-a-single-CPU) NFS stuff like pNFS, and NFS-to-Lustre gateways. It is not a mistake to make a commitment to NFS.

it'd be nice, though, if Apple would push all their netboot, LDAP, and software update cache tools as open source packages and get them integrated into CentOS or Ubuntu, the way they did with CUPS which works amazingly well. It's like they think they're doing some Microsoft Small Business Server thing with OS X Server, and it's just not on.

One less "feature"

[Lank]

With most of the emphasis on performance and stability, this was probably the one "feature" I was looking forward to with Snow Leopard. At $29 I'll still upgrade. Grand Central and OpenCL sound fairly impressive but I was really looking forward to a file system that never needed to be upgraded... I guess I'll keep on waiting.

Re:One less "feature"

[moosesocks]

I was really looking forward to a file system that never needed to be upgraded...

ZFS might be the holy grail of filesystems in terms of capacity, flexibility, and data integrity, which have traditionally been the limiting factors for filesystems. However, it's not particularly fast, and I'm sure that we'll come up with better ways to store data in the future.

If Apple have their own "ZFS killer" in the works, and choose to release it under a permissive license that's compatible with the GPL, they might very well be able to displace ZFS, given that the Linux community's refusal to support it has been an enormous thorn in its side.

I want a universal filesystem

[Anonymous Coward]

Which one can you mount on Linux, MacOS and maybe even Windows without precarious hacks, and with journaling, long filenames, and maybe extended attributes? So far FAT and HFS+ without journaling seem to be about the only choices. ZFS would have been it if MacOS and Linux both ended up supporting it, but now neither of them do (without precarious hacks!)... so Solaris is off in the corner by itself again. Bah humbug.

When I dual-boot my Mac (Linux & Leopard) I'd like to have the same partition for home directory on either system. A better FS for thumb drives than FAT would be nice, too.

The situation is utterly pathetic.

Re:I want a universal filesystem

[JohnFluxx]

Why, it's almost as if Microsoft don't want to inter-operate. Ext3 is fully documented with viewable code, yet MS don't implement it. NTFS on the other hand has to be reverse engineered.

Re:I want a universal filesystem

[dzfoo]

What about a pony?

            -dZ.

Re:I want a universal filesystem

[superposed]

I'm in the same boat. For years I've been looking for a file system can hold files larger than 2GB and can be mounted from Windows and Mac OS X (and maybe Linux). That would allow me to store all my work on one partition, and access it from Mac OS X, from Windows via Boot Camp, or from Windows inside Parallels or VMware Fusion. It would also allow me to transfer large files back and forth between my Mac and other Windows computers. I was hoping ZFS would be that file system.

The last time I checked (the middle of 2008), the only way to do this was via NTFS, and the only read-write support for NTFS on OSX was the MacFUSE NTFS driver, which was pretty slow.

I just saw that MacDrive 7.2 now allows Windows Vista x64 (my Boot Camp OS) to read HFS disks, so maybe I'll give that a try. There are also rumors that Snow Leopard's Boot Camp utility will include drivers for Windows to read HFS disks, so maybe that will help too.

Re:I want a universal filesystem

[drsmithy]

Ext3 is fully documented with viewable code, yet MS don't implement it.

What's in it for them ?

Re:I want a universal filesystem

[JohnFluxx]

Exactly how they think. They don't care about making it easy for users to have a decent cross-platform filesystem. They want lock in and control.

Integration issues

[henrikba]

The Known Issues and Features in the Works page for ZFS on MacOSforge explains the situation pretty well. Integrating ZFS into MacOSX isn't just a matter of creating a device driver. Time Machine, Finder, Spotlight and other core OS products needs to support ZFS features explicitly, since ZFS behaves a lot differently from HFS+.

That would be a new feature

[code4fun]

Snow Leopard is about performance and optimization. A new file system would fall under new features.

ZFS still needs more miles under the belt

[BlackSabbath]

I've played around with ZFS on the Mac a little bit. I've also played with ZFS at work (Sun UltraSPARC platforms) where we went from true believers to backing away rapidly (let's just say that there are certain Oracle workload profiles for which ZFS causes some massive performance hits especially when the disks are close to full).

I'm guessing that ZFS failed to meet at least one of (what I imagine are) Apple's criteria:
1. has to be simple to use
2. has to be rock solid

There's a good chance it failed at both. I'm not saying that ZFS is crap. Personally I think its a brilliant design, however it needs a bit more sunlight before its ready for the Steve.

Re:ZFS still needs more miles under the belt

[0x000000]

Okay, first and foremost it is well known that if you are running a database engine on top of ZFS you have to tune it to that specific database engine. This is well documented, and well described in the ZFS manuals, including steps to be taken to resolve these issues.

As for the performance degradation when the disks are close to full are being worked on, while this can cause issues (especially if you have a lot of snapshots) any IT worth their salt would not have let the disk get that close to full that it causes issues (I've seen this error once on my production servers, when the disk was at 95% capacity, I was brought in as a contractor). Replacing and upgrading disk capacity is as simple as pulling one drive from the RAID Z, placing a new one in, and letting it resilver, then pull the next one, until you have pulled all of them, after which you will get the full space the new disks can provide, so going from 1 TB drives to 1.5 TB drives will at the end of replacing all of them (so that they are now all 1.5 TB or bigger) give you the extra space.

As for 1, ZFS is extremely simple to use. gvinum from FreeBSD, or Linux's LVM are complicated, unnecessarily so, and 2, ZFS has so far proven far more reliable. It has been extremely fast, and has already saved a whole lot of trouble when a disk started failing by giving us a warning that ZFS reads were failing and letting us replace the disk before disaster strikes. Since we started using it in the last year we have had not yet had to resort to finding the backup tapes for a server because a disk went bad in Linux's LVM and bad data was written to other disks and files were lost.

I don't believe the issue is that ZFS is not ready yet, I don't think that Apple has had the time to make sure that everything fits in with their way everything has to work, certain features that HFS+ can offer are not possible on ZFS yet. Certain tools are relying on very specific HFS+ mechanics and workings (Time machine for example) which would complicate work to replicate that on ZFS.

While I was looking forward to seeing ZFS in Mac OS X, I doubted that it would be anytime soon, especially since it is a large undertaking making sure that the various parts of the system are all tuned for ZFS, this includes the way the OS caches, the amount of memory it can use for ZFS arc cache, and things along those lines. FreeBSD has slowly been working through those exact issues.

Re:ZFS still needs more miles under the belt

[BlackSabbath]

We never got a reproducible cause/effect explanation from the Sun engineers (which is one of the main reasons we started backing away).

Our particular problem seemed to occur when free-space shrank to below 20% and we had workloads with large numbers of connections doing lots and lots of very small write transactions in Oracle (using Oracle AQ as the backing store for our ESB/BPEL implementation). It wasn't 100% reproducible but seemed to be linked to those configurations more often that not.

Having said all that, we never used ZFS for production systems (we are far too conservative a company). We used it for dev/test/UAT environments where the ability to clone large numbers of test environments cheaply, quickly and with very little disk space cost was of great benefit. Its still used in some circumstances, just not all of them. Horses for courses.

Re:ZFS still needs more miles under the belt

[BlackSabbath]

> you are running a database engine on top of ZFS you have to tune it to that specific database engine

Been there, done that.

> any IT worth their salt would not have let the disk get that close to full that it causes issues

I don't consider 80% a threshold which any FS should start to cause issues

> As for 1, ZFS is extremely simple to use

For unix admins perhaps. For the remaining rather large subset of Mac OS X users perhaps not.

> I don't think that Apple has had the time to make sure that everything fits in with their way everything has to work

Totally, 100% agree.

Re:ZFS still needs more miles under the belt

[chefmonkey]

Wait... what? You open by telling us that you have to perform custom tuning on the filesystem for certain applications, and later assert that "ZFS is extremely simple to use."

[cue the car analogy]

That's like claiming you have to open the hood on a car and tinker with the engine depending on what kind of road you're driving on, and then asserting that the car is "extremely easy to use." If you mess with your engine on a regular basis, it might seem that way -- but if you're a normal user, it's an unspeakable pain in the ass.

Re:For those who are wondering:

[Omestes]

Hmm.. karma whore much?

I'm sure 99.9% of the people on Slashdot, who care enough to open the discussion know what ZFS is, and those who don't are perfectly capable of entering the term "ZFS" into Google.

But hell, lets see if I can do this too:

Apple:

Apple Inc. (NASDAQ: AAPL) is an American multinational corporation which designs and manufactures consumer electronics and software products. The company's best-known hardware products include Macintosh computers, the iPod and the iPhone. Apple software includes the Mac OS X operating system, the iTunes media browser, the iLife suite of multimedia and creativity software, the iWork suite of productivity software, and Final Cut Studio, a suite of professional audio and film-industry software products. The company operates more than 250 retail stores in nine countries[2] and an online store where hardware and software products are sold.

Sorry for trolling, have a six pack and a day off.

ZFS? What ZFS?

[commodoresloat]

There never was a ZFS. And Oceania was always at war with Eurasia.

I see no problem with that

[erroneus]

If something isn't "good enough" to make a solid product, then don't include it. This is how Vista got whittled down the way it was. The list of features that were pulled is longer than those remaining by my estimation.

mod parent up

[commodoresloat]

tequila really burns when it comes out your nose.

Re:For those who are wondering:

[wolf12886]

I'm sure 99.9% of the people on Slashdot, who care enough to open the discussion know what ZFS is, and those who don't are perfectly capable of entering the term "ZFS" into Google.

Alright fair enough, I mean that's what I did, but alot of slashdoters like myself whould first grumble about there not being a link to said article in the story. So I figured near the top of the comments was the next best thing.

Also, I've already got excelent karma. Once they come out with a +2 Godlike-Karma-bonus you can legitimatly troll me for karma-whoring.

Re:For those who are wondering:

[porl]

Slashdot

Slashdot, sometimes abbreviated as /.,[1] is a technology-related news website owned by SourceForge, Inc. It features user-submitted and editor-evaluated current affairs news with a "nerdy" slant.

(for those that got here by accident... you can't leave them out).

We do

[theolein]

We use Mac OSX Server for our infrastructure. It's a royal PITA and I now wish we hadn't done it, but there have been a number of media companies in recent years that have moved to Mac OSX Server because all their clients are OSX.

My view is that Apple is just jealous of Microsoft and said to itself that if Microsoft can drop promised new features in Vista like the DB based file system, then why can't Apple drop ZFS? ;-)

Re:For those who are wondering:

[Anonymous Coward]

Wikipedia

Wikipedia is a free,[5] multilingual encyclopedia project supported by the non-profit Wikimedia Foundation. Its name is a portmanteau of the words wiki (a technology for creating collaborative websites, from the Hawaiian word wiki, meaning "quick") and encyclopedia. Wikipedia's 13 million articles (2.9 million in the English Wikipedia) have been written collaboratively by volunteers around the world, and almost all of its articles can be edited by anyone who can access the Wikipedia website.[6] Launched in January 2001 by Jimmy Wales and Larry Sanger,[7] it is currently the most popular general reference work on the Internet.

(for those who got freaked out and wondered were they were after they clicked your link).

KILL HFS+ WITH FIRE

[argent]

OK, when they updated UFS in Panther I was all ^_^ because I was tired of HFS+ turning up x_x, and then they decided to make Spotlight dependent on HFS+ and I was all o_O and half the guys on Slashdot were telling me that UFS was -_+ and ZFS was coming and they were all :) over that, well guys, what kind of emoticon are you mainlining now?

ZFS or Btrfs

[r45d15]

Afaik Btrfs, which is roughly the Linux version of ZFS, has been started by Oracle (developers) and then embraced by Red Hat and alikes. So I'm wondering what are Oracle's plans about Btrfs after acquiring ZFS through Sun?

Snow Leopard *Server*

[R.Mo_Robert]

If you read the linked page (from Google cache), you'll see that this feature was slated for Snow Leopard Server, not the consumer version. I do not recall Apple ever advertising fll ZFS support as a feature for the consumer verison of 10.6, and neither does Wikipedia.

(Yes, consumer 10.5 does have read-only support for ZFS from the command-line; I imagine this would be still present in 10.6. In any case, it's not like this project is a secret, as Apple has released it open-source.)

ZFS not ready?

[ggendel]

Well, I've been using ZFS for several years on several different machines with mixes of mirrored and RAID-Z configurations. Since that time, I've never lost one bit of data. It has survived power-supply failures, lightning strikes that fried the motherboard, flaky I/O cards, and human error. I understand that the implementation on Mac OS/X may be buggy, but it's not inherent to ZFS. I've several Macs doing time-machine to networked ZFS drives. It's definitely the filesystem I'd like to have everywhere.

Re:ZFS not ready?

[ggendel]

You're right on the button. I created a sparse file for each machine image using diskutil so I could fix maximum size (I'd hate it to take over my entire 2.5 TB pool). The trick is to figure out the name that each machine wants, but worse comes to worse, you cancel it quick on the first sync if it's wrong and then rename the file and start it again.

Then I used the native CIFS service that comes with OpenSolaris for the connection. I started with Samba, but the native CIFS service had 1000X better throughput.

There is an option that enables mounting "foreign" disks for time machine. This may explain it better:

http://www.macosxhints.com/article.php?story=20080420211034137

Parent Wrong.

[toby]

Shame I just blew my mod points by posting.

But parent is completely wrong. ZFS root/boot is fully supported by Sun, and ZFS itself is used in production in thousands of installations.

Uhm... please mod parent "entirely f'ing wrong"

[Gary W. Longsine]

It's a widely known fact that Apple uses Mac OS X Server to host the iTunes Music Store. No, I'm not going to provide you with a link. Learn to use Google.

BTRFS vs. ZFS

[DrYak]

So, no, Larry's company becoming ZFS owner ain't the reason Steve's company would drop it.

Unless you keep in mind that Larry's got his own filesystem under the hood : BTRFS was Oracle's GPLed answer to Sun's ZFS (BSD licensed).

So perhaps Oracle is thinking that developing 2 competing filesystems with the same feature-set is maybe too much ? (Specially since Oracle tends to target slightly more often Linux than BSD - and thus could make sens to put more resource into a file system with a GPL-friendly license rather than a files system whose license makes it incompatible with Linux)
And perhaps they would like to drop ZFS in the long term ? So suddenly there's less incentive for Apple to support ZFS.

Or maybe, indeed, it has nothing to do with Oracle acquiring Sun and perhaps ZFS is to Apple what WinFS is to Microsoft: an eternal "Sorry, we didn't have time to implement it in the current version of the OS, but we promise we will put it into the next iteration. Trust us, that time it'll be for REAL !" vaporware.

Who's Larry?

[fm6]

If you mean, has Oracle management quietly told Sun to back off the ZFS evangalism, I kind of doubt it. It's hard to see why they would even care, at least not enough to risk getting caught doing something that could have nasty consequences — Oracle's acquisition of Sun still hasn't had federal approval, and illegally interfering with Sun's management would be just the thing to get it turned down.

The whole ZFS-on-MacOS thing is part of Sun's broader efforts to fight the marginalization of its technologies by open-sourcing them and then evangelizing everybody in sight to adopt them. This has happened not just with ZFS, but also with Solaris, the Sun implementation of Java, and even the Sparc CPU.

One aspect of this effort has been to push OpenSolaris and ZFS at desktop users. Pushing Apple to fully support ZFS (right now, they only provide a read-only driver) is part of this, as is a big push to get CS students and other hackers to download and use OpenSolaris on their personal PCs.

There's a certain amount of wishful thinking here. Solaris and ZFS do have very real and important technical advantages over their alternatives. But for a desktop user these advantages are pretty minimal. And to get them, you have to pay a big price in learning to use more complex tools and in not being able to participate in in bigger user communities.

Apple's response to Sun's ZFS evangelism was initial polite interest, but little positive effort over the long term. Not at all surprising: what use is ZFS to the typical Mac user? If servers were a bigger part of Apple's business it might be different.

Akamai is only a network

[raddan]

The backend, for the vast majority of their customers, is up to the customer to decide. I had the pleasure of taking a grad CS course with an Akamai engineer, and I specifically asked him about Apple, which is one of the customers he works with. He said Apple provides their own backend.

What about the other side?

[HumanEmulator]

Just playing devil's advocate: All the posts here seem to be trying to figure out what's wrong with ZFS to cause Apple to yank it out, but what if ZFS is fine and there's some big feature they're working on for HFS+ that they couldn't duplicate in ZFS?

I admit it's much more likely they just don't want to maintain full support for multiple filesystems, which is what they'd have to do because there's no way they're putting ZFS on iPhones and iPod Touches anytime soon.

Either way, the really telling thing is they aren't talking about ZFS in Mac OS X Server. If they had any plan for a ZFS future, it would start there much like the way HFS+ Journaling was initially a Mac OS X Server feature. (Introduced in OS X Server 10.2.2 and rolled out to non-server OS X in 10.3.)