Apple Looking at ZFS For Mac OS X

Udo Schmitz writes "Apples Filesystem Development Manager, Chris Emura, is looking into porting Sun Microsystems' file system ZFS to OS X. At least this is what Sun's Eric Kustarz states on the ZFS mailing list. Is this a glimpse of hope for all those of us who think HFS+ isn't up to par for a 21st century OS? Next thing you know and they'll rewrite the Finder ..."

Comments

Comparison of Filesystems.

[Whiney Mac Fanboy]

Have a look at wikipedia's Comparison of file systems page to see the difference between ZFS & HFS+.

The main advantage for HFS+ users (I mean who's really going to need a 16,000,000 Gigabyte file) would be the introduction of journalling beyond metadata (and even this is unlikely to be useful to most people).

Re:Comparison of Filesystems.

[lokedhs]

I think the major advantage is the fast snapshotting and cloning. It uses copy-on-write so that it doesn't take more space than what you actually change.

Imagine being able to take really fast working copies of whatever you're doing and be able to simple use the old versions by cd'ing to the old clone.

That's certainly what I would use ZFS for. The rest of the stuff, pooling and mirroring and stuff is less interesting in my laptop. :-)

Re:Comparison of Filesystems.

[captnitro]

Since OS X.3, I believe the kernel has defragmented files under 20 MB on the fly.

Re:Comparison of Filesystems.

[clifyt]

"HFS+ is subject to fragmentation (but Apple, like MS, provides no tools to help you deal with it)"

Talking in depth to one of the original OS X engineers (there were 4 or 5 depending if you count Jobs as one of them -- they all claimed Jobs gave as much input to the original porting of Next to the new OSX as anyone else did), his claim was that fragmentation isn't a problem.

Apple specifically doesn't offer tools because it defrags files as it makes sense to the operating system -- and generally doesn't defrag at all except for tiny files because modern drive and multiple independent read / write heads on drives today make a bit of entropy a good thing. If I remember later conversations correctly, he also mentioned that Apple had several graphic based disc tools that could do the same things that the OS does on an individual file basis, but didn't see the point in releasing them because this was something that should be left up to the OS and not up to the user. I argued that the user should have control and he countered with the fact that unless you had intimate knowledge about the drives physical features as well as the OSs specific needs, you are more likely going to slow things down in your quest to align the pretty colors together on your defrag program.

What was interesting was that he also recommended that you never fill a drive past 60 or 70 %. The claim was that having a huge chunk of empty space allowed the OS to do its thing without having to resort to smoke and mirrors.

Note -- defragging is an IMPORTANT part to my audience. I deal with musicians and engineers working on digital audio workstations. I remember using specific defraggers that were used solely for our industry (i.e., would write audio files to areas of the disc that were claimed to be the fastest read / write). I followed this skeptically -- until my contact forwarded me to a counterpart of his at Microsoft that essentially said the same thing -- in a MODERN OS using modern hardware, this does more harm than good.

Do I believe that a user couldn't get more optimized use out of defragging their own drives? I don't really know...but I'm going to trust these guys. Do your own research though. For all I know, I was told a line of BS that is intended to keep people like me from poking around under 'modern os`s' :-)

Slashdot is Getting Better Again

[LakeSolon]

A story that consists of a link to wikipedia and a mailing list posting about an OS possibly (maybe, potentially) switching filesystems.

Beats the heck out of story about a blog posting that's just a regurgitation of an MSNBC article that doesn't know what the frack it's talking about.

This is meaningless

[squiggleslash]

Here's a listing of the file systems currently supported on OS X Panther (it may be more for Tiger, I don't know):

$ ls -l /System/Library/Filesystems/
total 0
drwxr-xr-x 8 root wheel 272 14 Mar 12:46 AppleShare
drwxr-xr-x 7 root wheel 238 12 Apr 2005 URLMount
drwxr-xr-x 6 root wheel 204 14 Mar 12:47 cd9660.fs
drwxr-xr-x 3 root wheel 102 22 Dec 2004 cddafs.fs
drwxr-xr-x 4 root wheel 136 14 Mar 12:48 ftp.fs
drwxr-xr-x 5 root wheel 170 14 Mar 12:47 hfs.fs
drwxr-xr-x 4 root wheel 136 14 Mar 12:47 msdos.fs
drwxr-xr-x 4 root wheel 136 14 Mar 12:47 ntfs.fs
drwxr-xr-x 4 root wheel 136 14 Mar 12:47 udf.fs
drwxr-xr-x 4 root wheel 136 14 Mar 12:46 ufs.fs
$

HFS and UFS are the official choices of file system for installing your bootable OS X or Darwin system. The rest are either network based file systems or are specific choices for interoperability with other operating systems.

There are many reasons why Apple might be looking at ZFS. Only one is that Apple intends to actually make Mac OS X use it as a home filesystem.

Now, here's a reason the write-up author didn't think of: Apple is rumoured to be working on a virtualization layer for OS X, with the intent being that OS X will run in parallel with multiple operating systems. Even if that rumour is false, it's clear that with BootCamp, Apple is taking the idea of Macs running multiple operating systems (albeit not at the same time...) seriously. Solaris and GNU/Linux are the two most popular Intel platforms save for Mac OS X and Windows.

Isn't it more likely that Apple wants Mac OS X its multi-OS Macs to "just work" with the other operating systems, able to achieve a high degree of interoperability without forcing the other platforms to support HFS+?

I'm not saying a move to ZFS would be a bad thing, though it doesn't, so far as I can see, support arbitrary metadata so it'd be as practical as UFS in its current form, which is barely used by Mac users. I just think a port of the main Solaris file systems is, in practice, something Apple would be doing anyway, as part of the Intel OS-agnostic direction they're going in.

Re:This is meaningless

[Anonymous Coward]

ZFS actually is a ver good file system.

Here is the ars technica low-down on what ZFS does differently and why that's such a good thing.

arstechnica.com/news.ars/post/20051117-5595.html

What Apple Is Looking For

[rpk]

There are probably two things that Apple would be looking for in ZFS: a shiny feature they can point to for their enterprise and video production markets, and for the consumer market, the promise of a simple, reliable way to back up and grow the storage of a Mac without have to worry about mounting/copying/moving volumes, managing backups, etc.

Re:HFS+ vs. UFS vs. ZFS

[rsmith-mac]

From the AirPort page:

This issue is resolved in Mac OS X 10.1 and later.

It's the same deal with the problem with Classic. All 3 items you link to are for OSX 10.0 and have been fixed since then. The number of UFS problems now is minute compared to then.

that's nice. now fix network file systems.

[dschuetz]

I just wish we could come up with a network file system that's worth the trouble. Right now, I'm using a Linux server with three Macs (two Tiger, one Panther), and everything is over NFS. Most of the time, it works fine, but if there's a weird hiccup, then the Mac will freeze solid and has to be hard power-cycled. Also, some apps simply won't run from a network share (or they'll run, but one thing or another won't be right). Install that app to a local drive, and it works fine. And this isn't even to mention security issues.

I've looked at AFP, but that essentially mounts the remote system as if it were an external drive, and assigns everything to the logged in user, so ownership, permissions, etc., are all really screwy. Plus that gets even worse if you use fast user switching -- now two people are independently trying to mount the same network drive, each claiming to own it outright. And it doesn't look as seamless as, say, simply going to /Server/Shared or /Server/Apps.

SMB isn't much better.

There's always AFS, but that's so bloody complicated that I'd take a lot of convincing before I seriously considered it.

This isn't even to mention the problems that most apps have in working in a networked environment -- applications simply aren't designed for, say, networked home directories, and *especially* aren't designed to be running simultaneously on multiple systems. So if I've got Mail.app running in the den and I log in upstairs to check mail just before I go to bed, things could get messed up.

I'm not sure there's even been a new network file system since the mid 90's, has there? Certainly, nothing with broad support that fixes some of these issues? All I want is UNIX filesystem features -- simple locking (I guess), owners, regular permissions. Doesn't even need to do ACLs. Transparently mounted so it looks like it's part of the local filesystem. And at least reasonably tolerant of network glitches, so a momentary drop at the server (or whatever else happens to screw NFS connections to the wall) doesn't put all apps which have even heard of the mount point into an uninterruptible kernel-level deep-freeze (what's the point of kill -9, dammit?). Is that so difficult?

HFS is big endian

[KonoWatakushi]

One of the reasons proposed over at Arstechnica has to do with byte ordering. Currently on intel macs, all disk IO has to be byte swapped, degrading performance. ZFS on the other hand will store data in the machines native format.

Even so, all of the other features of ZFS are worth much more than this. If Apple is anything more than a consumer widget company now, ZFS should definitely be under consideration.

ZFS is far from "just another filesystem," and comparing it to existing filesystems indicates a lack of understanding. Take a look at this presentation for more information.

Re:HFS is big endian

[BurntNickel]

Currently on intel macs, all disk IO has to be byte swapped, degrading performance. ZFS on the other hand will store data in the machines native format.

While the non-native byte ordering does slow performance this only applies to metadata and not the contents of the files.

Apple should just buy SUN

[adam1101]

The more I think about it, the more it makes sense for Apple to buy SUN. Their products nicely complement each other. Apple is strong in the consumer market and in the creative sector, SUN has good presence in the enterprise, tech and finance sectors. Apple has great brand value and knows marketing like no other computer vendor, SUN has technical excellence, but it's been struggling in the last years to actually sell their stuff. Their products portfolios have little overlap, and together they offer a very complete spectrum of computer products.

Mac OS X is a great consumer OS, but performance at the high end is sub-par. For servers, Solaris is fast and scalable, has nifty features like ZFS and DTrace, but the UI is pretty crude. Imagine a merger of these. Looking at their market caps, Apple can afford it.

Re:Apple should just buy SUN

[mhollis]

I like your comment. And the reason why I like it so much has to do with my (past) experience on a University system. Universities developed servers and file sharing with Macs using Sun's servers because Apple really didn't have a server. I mean you could put a Mac (usually an older one) on a network and tell it to share files with everyone but it lacked lots of stuff you would expect to have in a server and it tended to be pretty slow.

I would argue that it was the University exposure that lead Apple to offer Ethernet on Macs. Appletalk was great and people hooked themselves up very quickly with Appletalk (you could buy cabling at your local Radio Shack or use almost any twisted-pair cabling, including electrical cables) but Ethernet was a lot faster and more reliable. I'll bet the folks who developed 10 Base-T Ethernet were thinking Appletalk when they came up with the design for the connector and the twisted pair.

But I digress...

I did a fair amount of work with a hard Science department and they all had Suns as servers. They were strictly Sun Unix for the geeks and they developed systems and applications on that model. But for those who actually had to function in an office environment, the Macs were standard. They used Microsoft's Office for memos, reports and spreadsheets and TeX for document publishing. Everything you did worked.

Frankly, I think this legacy is part of the reason why Apple got fascinated with Unix again (that, and Jobs' NeXt company). It would be a good marriage. Apple's X-Serve RAIDs with Sun. Sweet!

Re:Think you'll get it?

[booch]

I agree that there was a lot in VMS that the world has "lost". I think that modern UNIX implementations should look at what VMS had, to reuse some of the good ideas that we still have not replicated. My favorite is the security system -- various small capabilities that each user (or program) could be granted. And the super-user only had one capability by default: the ability to grant privileges. I also appreciated the automated versioning, with the ability to pull up a previous version from the filesystem without having to use any special programs.

And yes, I know that Windows NT is sort of descended from VMS. But I've not seen many of the concepts make it up to userland cleanly implemented.

And I'm also aware that VMS is still around. It may not be on life-support yet, but it's clearly in the nursing home already.