ZFS Confirmed In Mac OS X Server Snow Leopard

number655321 writes "Apple has confirmed the inclusion of ZFS in the forthcoming OS X Server Snow Leopard. From Apple's site: 'For business-critical server deployments, Snow Leopard Server adds read and write support for the high-performance, 128-bit ZFS file system, which includes advanced features such as storage pooling, data redundancy, automatic error correction, dynamic volume expansion, and snapshots.' CTO of Storage Technologies at Sun Microsystems, Jeff Bonwick, is hosting a discussion on his blog. What does this mean for the 'client' version of OS X Snow Leopard?"

Comments

What does this mean for 'client'?

[daveschroeder]

Nothing, in particular. It means that ZFS isn't going to be officially supported and/or promoted on client. But, since Mac OS X and Mac OS X Server are essentially the same OS with some different/additional pieces on the top of Server, and like other filesystems that were exposed via the GUI tools and supported on Mac OS X Server, but not on Mac OS X, in the past -- such as Mac OS Extended (Journaled, Case-Sensitve) -- it will likely be available via the command line tools, and usable by people savvy enough to work with other boot devices to format the volume in the desired fashion, etc.

Re:What does this mean for 'client'?

[Goaway]

You should perhaps consider taking the internet less literally.

Re:What does this mean for 'client'?

[Just Some Guy]

From what I understand, ZFS is fast not memory efficient.

Nitpick: it definitely likes a lot of RAM, but it doesn't necessarily use it inefficiently. Car analogy: a semi truck is fuel efficient, even though it's gas mileage is a lot lower than a sedan's.

Re:Finaly

[jandrese]

8TB is rapidly becoming "not that much stuff" these days. You can already buy 1TB HDDs, so we're just three doublings away from hitting the limit with a single drive (not to mention RAID arrays).

Re:How will I benefit?

[fishdan]

This is very informative

Re:How will I benefit?

[profplump]

What is it with you people and filesystem-level snapshots?

I'd much rather have volume or block level snapshots, like with LVM and other similar systems. Those systems provide RO and RW snapshots, dynamic partitioning, drive spanning, etc., and can be easily layered with other block-level components to provide compression, encryption, remote storage, etc. as well. All that without tying you to a single file system (though that may be a moot point on OS X, as it will only boot from HFS/HFS+ AFAIK).

If you really wanted to you could even write a script that takes no arguments other than a path name and automatically created a series of volumes of an appropriate size for the folder you selected, setup software raid to mirror them into a single device, mount the device with a compression filter, format it (with any file system) mount it normally, move the data over, drop the old data, rebind the mount point to the old path name, and update fstab. The only thing you miss here that ZFS may be able to do (I didn't check) is avoid closing the files that are moved.

I'm not saying the features ZFS has are useless -- I think they are great -- they just aren't all that new and exciting. They might be new OS X, or repackaged in a way that's easy to consume, but they are things that anyone with big disks has been doing for years.

Re:How will I benefit?

[Cyberax]

SMART sucks. That's just a fact - very often it kicks in when your drive has failed.

Also, there are lot of real cases where malfunctioning drive can silently write incorrect data. ZFS will help you in this case.

Re:How will I benefit?

[Lars512]

For home users, it would let you simply plug a new drive in your Mac, press a button, and have it just add space to your main drive. You wouldn't need to specifically setup a RAID. No resizing. No "external drive" if you don't want it that way. Just buy a drive, plug it in, and it's all handled for you.

I'm not sure you'd want it to work this way for external drives. Will they be available at crucial parts of boot time when some important files are striped across them? Even if they are, you're basically unable to ever remove the external drive again. If there's a problem with the drive, all your data is lost. Probably the way these drives work now is better. Maybe mirroring onto an external drive would work ok, but it would then be an undesirable write bottleneck.

Re:Standardizing file systems

[larry bagina]

1. The GPL prevents ZFS integration (without a complete and total reimplementation of all the code... which won't happen since everybody prefers to write their own filesystem)
2. MS DOS/FAT is the universal file system for operating systems.

Re:How will I benefit?

[MrMacman2u]

Actually S.M.A.R.T. is an amazing tool and is utterly invaluable in monitoring drive health... IF and ONLY IF you have the appropriate software (windows: google it, *nix: smartmontools) AND know how to read the resulting output.

The reason many people think SMART sucks and I say to check SMART manually is because 95% of drive manufactures set the threshold or "fail" values WAAAAY too high or low!

I use SMART constantly (about once every other week) to "check in" on how healthy my drives are and knowing how to read the values the software returns has saved my data many times. In fact, due to certain SMART values, I KNOW my Thinkpad hd is on currently on it's way to failing even though it is still working fine, for the moment. The SMART information has let me know it's on it's way out the door and therefore I have taken precautions to safegaurd the data on that machine and have the drive replaced.

Granted, SMART can't inform you ahead of time about sudden and complete mechanical or electronic failure, it can warn you if your drive is slowly the kicking the bit bucket. (With a small amount of know how until drive manufactures wake up and set more appropriate values)

Don't knock something you know nothing about! Kthnxbye!

Re:How will I benefit?

[Cyberax]

Maybe I'm unlucky, but I had three notebook HDs die on me without any warning. Even though I'm using 'SmartMon' program which should warn me about worsening drive condition.

Also, Google's on hard drive survey seems to come to the same conclusion: "One of those we thought was most intriguing was that drives often needed replacement for issues that SMART drive status polling didn't or couldn't determine, and 56% of failed drives did not raise any significant SMART flags"

http://www.engadget.com/2007/02/18/massive-google-hard-drive-survey-turns-up-very-interesting-thing/

Re:No, I am understanding just fine.

[gutter]

It is pretty hard to tell from your tirades whether you are talking about the ability to support pluggable filesystems, or the availability of those pluggable filesystems. You seem to be conflating the two. You start by complaining that OS X lacks the ability to support pluggable systems, but the first link from the AC's post proves you wrong:

http://developer.apple.com/qa/qa2001/qa1242.html

In fact, every filesystem OS X supports is written using this mechanism, out of the box:

[gutro:~/] gutter% ls -1 /System/Library/Filesystems/
AppleShare
URLMount
afpfs.fs
cd9660.fs
cddafs.fs
ftp.fs
hfs.fs
msdos.fs
nfs.fs
ntfs.fs
smbfs.fs
udf.fs
ufs.fs
webdav.fs
zfs.fs

Your most recent tirade seems to be a complaint about the lack of available filesystems, which I guess is a reasonable complaint, but that's not what you orignally asked for. Then you asked for a simple package you could download and install, and again, the original reply contained one (MacFUSE). Granted, that's a poor example, because it hides OS X's native pluggable FS support behind the FUSE pluggable FS support, but that doesn't mean that the AC was wrong. You can go and download the MacFUSE package, and the sshfs package, install them using the standard installer, and begin using a filesystem that works over SSH, no compiling necessary. (Incidentally, that one is super handy).

In short, the original reply by the AC was 100% correct, and you were 100% wrong, (and seemingly unable to comprehend his reasonable explanations) and somehow by sheer bluster, you seem to have convinced everyone of the opposite.

Re:ZFS is fast, but not lightweight

[OrangeTide]

hehe.. very funny, WAFL is what is used in NetApp filers. You can read the patents on it and get a pretty good high level idea of how it works, the patent documents lays it out in pretty easy to understand terms. As for the mystery filesystem (starts with a D), it's like XFS but has more features to make it worth comparing to ZFS. The performance is similar to XFS too (a little better when it can hold transactions in NVRAM).

Even XFS is on par with ZFS (with lower cpu utilization) for nonclustered performance. XFS is so close to the theoretical maximum performance of a raw unclustered disk group (when tuned for the workload) that there is no "an order of magnitude or more" in those cases. When get get 99.5% of the raw disk I/O performance you're not going to reach an order of magnitude by making it .5% faster :)

Of course for a distributed cluster of disks the free version of XFS is nothing compared to ZFS, because XFS can't even really do clustering. And few people benchmark SGI's commerical CXFS, despite my experience with XFS even I haven't used CXFS.

Also it's hard to compare most filesystems to ZFS because most filesystems that have RAID just live on top of a hardware/software RAID implementation. XFS (and others) do a little bit of optimizations when they are on a RAID, but it doesn't really reflect in the performance numbers in a major way. ZFS has the advantage of RAID-Z, which is faster(usually) and far more flexible.

ps - I like to think of XFS as the base line of what a good filesystem should be, and you're either better than XFS or worse. I'm not trying to claim XFS is the fastest filesystem in the world or anything. Although if tuned for your workload the performance is quite impressive, when not tuned there is a lot of head scratching why all your RAM is gone and your performance is worse than FAT.