Apple Looking at ZFS For Mac OS X

Udo Schmitz writes "Apples Filesystem Development Manager, Chris Emura, is looking into porting Sun Microsystems' file system ZFS to OS X. At least this is what Sun's Eric Kustarz states on the ZFS mailing list. Is this a glimpse of hope for all those of us who think HFS+ isn't up to par for a 21st century OS? Next thing you know and they'll rewrite the Finder ..."

Comments

Comparison of Filesystems.

[Whiney Mac Fanboy]

Have a look at wikipedia's Comparison of file systems page to see the difference between ZFS & HFS+.

The main advantage for HFS+ users (I mean who's really going to need a 16,000,000 Gigabyte file) would be the introduction of journalling beyond metadata (and even this is unlikely to be useful to most people).

Re:Comparison of Filesystems.

[captnitro]

Since OS X.3, I believe the kernel has defragmented files under 20 MB on the fly.

Re:Comparison of Filesystems.

[araemo]

Go try and defrag a windows drive with less than 20% free, it'll give you a warning. :)

Most modern filesystems do some amount of 'defrag' automatically over time. Windows XP w/ NTFS does this, I would bet HFS+ is designed to do this. Of course, if there isn't a lot of free space to play with, the automatic 'opportunistic' defrag has a lot less chance of moving a large file to a bit of contiguous free space. If you can manage it, don't fill your drives to the brim. It will hamper performance, and it will make defrags take MUCH longer if you seriously fragment your files.

The way I understand it, sometimes when you overwrite a file, instead of reusing the same blocks, the FS marks those as free and writes to some currently free blocks, 'defragging' that file.

Re:Comparison of Filesystems.

[clifyt]

This was actually addressed.

If you keep around 70% of your drive free, the machine will be able to make large enough chunks that given a combinations of other factors it was meaningless.

It was said with multiple independent read write heads, you can actually fill the buffer faster by spreading the load out in noncontiguous sections...and even if it did need to read sequential sections from the same head in different areas, both that the drive can read files nonsequentially and load these chunks into the cache while the other read head is catching up -- and that it takes less than a ms to jump from one sector to another these days.

The clue that was beaten into me was to think of this as sorta a spanned raid within a single drive and that's entirely how these work these days (and then told its entirely not like a raid so I shouldn't use that metaphor lest some nerd that thinks with his head instead of his gut tells me that I'm wrong -- ok I made up the last part, but its essentially what I was told).

But all in all, as other have mentioned, HSF+ likes to defrag on the fly non-contiguous chunks of less than 20 megs (and it will also do this in the background after the CPU is more free after seeing these) -- and given that the average cache on a drive is around 16 Megs, even when this inevitably doubles in the next year or two, the logic remains that this is still good enough.

But you are entirely right -- if drives didn't employ caching and multiple independent read write heads (i.e., early multiple platter systems required that the read/write heads all be driven by the same motor and thus killing any attainable speeds).

Blah blah blah...its all pseudoscience and phrenology to me. I'm just mouthing everything that was sent to me without understanding a word of it. I'm a musician (and technically a pseudoscientist by trade) so making up words and using them incorrectly by mirroring others comes naturally and might even make sense to those that don't know any better :-)

Re:Comparison of Filesystems.

[Kjella]

Look up any decent hard disk review site and compare sequential read vs random reads, and you'll see that it suffers quite badly. As for the operating system handling that, I don't know about OS X but Windows certainly doesn't. The "never fill it past 60 or 70%" is simply because than the OS can pick large open chunks, avoiding fragmentation but it does nothing to fix fragmentation. Once your drive has been close to 100%, the last files will be in a zillion fragments all over the disk. They will sit around like little road bumps making sure all other files will be fragmented too.

That said, most desktop users will not notice a big difference between a fragmented, quick-defragment (defrag files, but don't consolidate free space) and full defragmented disk. A typical modern HDD has a 35-40MB/s minimum transfer rate. DV, probably the most resource intensive any normal person bothers with has a measly 25Mb/s = 3,2MB/s. Unless you're suffering from really horrible fragmentation, that should be no problem. Same goes for analog capture with hardware/on-the-fly compression. Yes, there are fringe areas like raw analog video or scientific data but audio capture isn't part of it anymore. And if you're that specific, using a separate tool isn't that big a deal. Servers OTOH might be something, but I imagine most of that is handled by other parts than the OS disk I/O.

Re:HFS+ vs. UFS vs. ZFS

[minus_273]

ufs does not work with all software especially stupid applications made by microsoft

Re:HFS+ vs. UFS vs. ZFS

[Whiney Mac Fanboy]

Doesn't OS X already have UFS? ZFS is nicely buzzword compliant, but Mac users can migrate to a standard Unix filesystem today if they want to.

Yes, OS X allready has UFS, but according to Apple:

UFS is case-sensitive

With a UFS-formatted volume, you may have the two "My File" files in the same location as well as other similarly named files -- "My file," "MY file," "My FiLe," "my File," and so forth.

AirPort and UFS

AirPort does not function when Mac OS X is installed on a UFS formatted volume. For more information, please see article 106252: " Mac OS X 10.0: AirPort Does Not Work From UFS Partition "

Customizing Hard Disk Volume Name

You can permanently customize a hard disk volume name when using Mac OS Extended, but not (currently) when using a UFS-formatted hard disk volume. For more information, please see article 106191: " Mac OS X 10.0: Startup Volume Is Named '/' Instead of 'Mac OS X' "

Mac OS X Classic Environment and UFS

The Mac OS X Classic environment does not function the first time it is opened on a UFS formatted volume. For more information, please see article 106277: " Mac OS X 10.0: Classic Does Not Work From a UFS Disk On First Use "

Mac OS 9 and UFS

When you start up your computer using Mac OS 9.x, UFS hard disk volumes do not appear on the desktop and cannot be used.

(unfortunately, zfs would not fix many of these issues)

Re:This is meaningless

[Anonymous Coward]

ZFS actually is a ver good file system.

Here is the ars technica low-down on what ZFS does differently and why that's such a good thing.

arstechnica.com/news.ars/post/20051117-5595.html

Re:HFS+ vs. UFS vs. ZFS

[rsmith-mac]

From the AirPort page:

This issue is resolved in Mac OS X 10.1 and later.

It's the same deal with the problem with Classic. All 3 items you link to are for OSX 10.0 and have been fixed since then. The number of UFS problems now is minute compared to then.

Re:YAY!

[Anonymous Coward]

HFS+ supports multiple file forks (just like NTFS, except they are called streams under NTFS), but at the moment they are only used for Data and "Resource"

So yeah, Mac's still do have Data and Resource forks, nothing bad about them (OS X is less reliant on them though)

And if you ZIP a file, the Resource fork gets carried along, just with a . added to the beginning of the file name to hide it on UNIX systems, and the hidden attribute set on FAT32 drives (like USB thumbsticks) and such.

Intel Macs good oppurtunity to make a clean break

[saha]

I very much wish for an updated filesystem for Mac OSX. I know that HFS plus (with journaling and meta-data searching where added later), I feel HFS + is showing signs of age. I was hoping when Apple first developed Mac OSX it had used the UFS system and then made a separate HFS+ partition for people who wanted to use a Mac OS9 on the PowerPC based Macs, but that didn't happen. Perhaps for the best at the time. Wilfredo Sánchez Vega wrote a whitepaper on the reasoning for HFS + at the time

So now with the Intel Macs and no need for Mac OS 9 support, Apple can tell all their developers that all Universal apps must be able to run on UFS. That way should Apple decide to adopt ZFS it should be a painless transition. Holding on to HFS + with the Intel Macs for this long will hamper any transition into a future filesystem. This will prepare Adobe and Microsoft to write their new Universal versions to be able to accept any type of filesystem and not rely on the resource fork of HFS

That's my 2 cents.

rewriting the Finder

[Aram Fingal]

I know this is just a little comment at the end of the story and not the main topic but the Finder really does need to be rewritten. It has a surprising lack of multithreading, even compared to Mac OS 9. This is most apparent (and most annoying) when you are navigating a slow network volume in the Finder. Quite often, you just can't do anything with but wait for the network to time out.

Re:HFS is big endian

[BurntNickel]

Currently on intel macs, all disk IO has to be byte swapped, degrading performance. ZFS on the other hand will store data in the machines native format.

While the non-native byte ordering does slow performance this only applies to metadata and not the contents of the files.

Re:Reiser4 would be a better choice

[alonso]

I think that you have to inform better, Raiser has journaling. From namesys site: "Reiser4 is an atomic filesystem, which means that your filesystem operations either entirely occur, or they entirely don't, and they don't corrupt due to half occuring. We do this without significant performance losses, because we invented algorithms to do it without copying the data twice"

Re:HFS+ vs. UFS vs. ZFS

[Azarael]

It helps to have knowledgeable moderators, but posts still have to be moderated to be useful for a general audience. In this case, the post in question doesn't tell you much if you don't happen to be very familiar with different file systems for OSX and the compatibility of OSX software with those file systems.

Is the grandparent post flamebait? maybe not. Without minus_273's though, its probably not useful enought to be modded up either. Whether the moderation system is right or wrong, isn't the point here, but the as the guidlines say http://slashdot.org/moderation.shtml/ in the FAQ, question 5:

What is a Good Comment? A Bad Comment?
* Good Comments are insightful. You read them and are better off having read them. They add new information to a discussion. They are clear, hopefully well written, or maybe amusing. These are the gems we're looking for, and they deserve to be promoted.
* Average Comments might be slightly offtopic, but still might be worth reading. They might be redundant. They might be a 'Me Too' article. They might say something painfully obvious. They don't detract from the discussion, but they don't necessarily significantly add to it. They are the comments that require the most attention from the moderators, and they also represent the bulk of the comments. (Score: 0-1)
* Bad Comments are flamebait. Bad comments have nothing to do with the article they are attached to. They call someone names. They ridicule someone for having a different opinion without backing it up with anything more tangible than strong words. Bad comments are repeats of something said 15 times already making it quite apparent that the writer didn't read the previous comments. They use foul language. They are hard to read or just don't make any sense. They detract from the article they are attached to.

By the above def, the grandparent is no more than an average comment that maybe leans a bit towards flamebait and probably shouldn't have been modded up or down.

Re:DeRez, RezWack, SplitForks, etc.

[JambisJubilee]

There's a bunch of resource fork utilities in /Developer/Tools/. Just add it to your $PATH

Re:What ever happened to BeFS?

[drewness]

Yes, Dominic Giampaolo works at Apple and is in charge of filesystems there.

Re:Think you'll get it?

[booch]

I agree that there was a lot in VMS that the world has "lost". I think that modern UNIX implementations should look at what VMS had, to reuse some of the good ideas that we still have not replicated. My favorite is the security system -- various small capabilities that each user (or program) could be granted. And the super-user only had one capability by default: the ability to grant privileges. I also appreciated the automated versioning, with the ability to pull up a previous version from the filesystem without having to use any special programs.

And yes, I know that Windows NT is sort of descended from VMS. But I've not seen many of the concepts make it up to userland cleanly implemented.

And I'm also aware that VMS is still around. It may not be on life-support yet, but it's clearly in the nursing home already.

Re:Reiser4 would be a better choice

[be-fan]

Reiser4 is transaction-oriented, just like ZFS. The two actually use a similar principle (not journaling) to maintain consistency, based on COW'ing blocks in a tree, then committing the change atomically by swapping the pointer to the root of the tree in the parent node. Reiser4, however, instead of using the traditional block tree ZFS does, uses "dancing trees", which is kind of a B*-tree with ideas from log-structured filesystems mixed in.

beyond RAID in data integrity, self-healing

[toby]

Apart from those pluses mentioned by lokedhs (snapshotting is no trivial feature to have, if you're running databases, for example, or want admin abilities like rollback) - What ZFS offers that no other Linux filesystem offers, let alone HFS+, is end-to-end data integrity and self-healing. That's why I picked Solaris 10 for a high-integrity database app recently. Nobody else could offer the integrity guarantees (apart from some SAN vendors perhaps).