ZFS Confirmed In Mac OS X Server Snow Leopard

number655321 writes "Apple has confirmed the inclusion of ZFS in the forthcoming OS X Server Snow Leopard. From Apple's site: 'For business-critical server deployments, Snow Leopard Server adds read and write support for the high-performance, 128-bit ZFS file system, which includes advanced features such as storage pooling, data redundancy, automatic error correction, dynamic volume expansion, and snapshots.' CTO of Storage Technologies at Sun Microsystems, Jeff Bonwick, is hosting a discussion on his blog. What does this mean for the 'client' version of OS X Snow Leopard?"

Comments

Finaly

[bobwrit]

We can finaly fill up more than 8 TB on this FS. Anyone up to try?(with what?)

Re:Finaly

[jabuzz]

News to me, double checking the Sun pages tells me that two or more servers cannot mount the same pool at the same time. It is allegedly coming with Luster 1.8, but it ain't here now.

How will I benefit?

[The Ancients]

Ok, I'm reasonably technical, but not savvy with the intimate workings of a file system. What will this mean for the average user with an iMac or MacbookPro, when ZFS finally appears as the default FS of OS X? Will it be faster, more error-resistant, or...?

Re:How will I benefit?

[countSudoku()]

You'll also be able to create a pool of drives that acts as a single drive, like you can with the RAID setup now, but far faster to setup. Growing your pools is a breeze and if they can tie TimeMachine into the zfs snapshots, my god, what can't we do?! Seriously, this will be a nice advanced file system for Mac OSX. We've been using it on Solaris for a year now for zone root/usr file systems, and zfs is AWESOME!!! Except that even Sun is not recommending we use it for zone root file systems until they hit update 6 of Solaris 10. Whoops! That's in November, so we're just sitting tight until they support Solaris root/OS zfs file systems. Then we upgrade. Then ? Then we profit!

Ob. Apple Joke referencing earlier /. artice:
Of course, the delay for the consumer OSX support of zfs will have to wait until they code in skipping backups of your iTunes library! ;)

Re:How will I benefit?

[Lally Singh]

Wow, it's such a major leap it's hard to describe.

Imagine having an external HDD on your mac. Whenever you plug it in, it automatically starts mirroring the internal drive.

Take atomic snapshots of your entire filesystem, send it over scp to back up your drive as a single file. Or, send over the difference between two snapshots as an incremental backup.

Have more than one drive, want mirroring? 2 steps on the command line.

Have a directory you really care about? Make it a sub-filesystem (this doesn't involve partitioning, etc, just a command that's almost identical in syntax and performance to mkdir) and tell ZFS to store 2 or 3 copies of it.

Have a directory you'd like auto-compressed? Tell zfs to compress it. New data to it is automatically, and transparently compressed. Completely transparent to the user and to applications.

And I'm just getting started. Trust me on this, google it.

Re:How will I benefit?

[MBCook]

For one thing it would make the implementation of Time Machine much simpler. No more directory tree full of hard links and such. If they put it on other boxes (like Time Capsule) they could unify the format (it uses a different storage method). Then you could pull the Time Capsule drive, stick it in your Mac, and be all set.

For servers, it has all the standard ZFS benefits (easy storage adding, redundancy, performance, etc).

For home users, it would let you simply plug a new drive in your Mac, press a button, and have it just add space to your main drive. You wouldn't need to specifically setup a RAID. No resizing. No "external drive" if you don't want it that way. Just buy a drive, plug it in, and it's all handled for you.

Re:How will I benefit?

[ArbitraryConstant]

I'd much rather have volume or block level snapshots, like with LVM and other similar systems. Those systems provide RO and RW snapshots, dynamic partitioning, drive spanning, etc., and can be easily layered with other block-level components to provide compression, encryption, remote storage, etc. as well. All that without tying you to a single file system (though that may be a moot point on OS X, as it will only boot from HFS/HFS+ AFAIK). ZFS shits all over LVM:

-Say I want to take hourly snapshots, and retain them for a month. When the parent data for a ZFS snapshot changes, ZFS merely has to leave the old data alone. OTOH, LVM must copy the block to every snapshot before it can change it in the parent. My hourly snapshots will quickly cause my disk to thrash to a halt with LVM and using much more space, while ZFS incurs a negligible penalty.

-LVMs allow dynamic partitioning, but they can't share capacity on the fly. If I delete a file on an LVM-hosted filesystem, that space becomes available to the filesystem but not all the others. Unless I shrink the filesystem, generally requiring that I take it offline for a while.

-Another layer could potentially handle checksums on LVMs, but in practice Linux can't do this properly by itself.

-ZFS can use other layers, there's just a substantial benefit to letting it run the show.

The only reason this won't turn out to be a huge disadvantage for Linux is that BTRFS will provide most of the same features. Layering can be a very helpful design tool, but there are times it becomes a hinderence. It's important to be flexible when there's benefits to integrating stuff into a single layer.

Re:How will I benefit?

[Fweeky]

SMART sometimes works, very often it doesn't. Manufacturers have been progressively crippling it, to the point at which some barely even monitor anything, because they're perceived by marketing as being bad for business.

e.g. Seagate are one of the few vendors who are honest about ECC correction and seek error rates, and their SMART counters are correspondingly huge and read rather poorly (50-60/100 is a common value); you can even graph them and see the rates sweep up and down as the drive moves the heads over the platters every hour or so. Nobody else does this, and occasionally you'll see someone on a forum asking if it means their disk is failing.

Raw read and seek error rate on my Western Digital drives? 0, corrected values: 200/200. Right, I'm sure WD have magical drive heads which read every bit perfectly every time, and never miss seeking to a track, ever.

That's not to say SMART can't be useful, but as Google's disk failures USENIX paper demonstrates, they're not as reliable as one might hope.

I dunno if I trust it yet.

[boxless]

I've lurked a bit in the opensolaris forums, and there's a whole bunch of scary things with this FS. Like the RAM requirements for starters.

More efficient backups.

[pavon]

One feature of ZFS is copy-on-write file snapshots, which allow you to "copy" a file, but the common portions of the file will be shared between the two copies, decreasing disk space.

This is great for backing up large files containing frequent but small changes. For example encrypted disk-images, parallels windows disk images, database files, the Entourage email box, or home videos you are in the process of editing etc.

Right now Time Machine creates an entire copy of the file each time it changes, making it unsuitable for backing up these types of files, and so you are encourage to exclude those files from backup. ZFS could fix that.

It could also make adding disk space more seamless, if desired. Slap on an external Firewire drive or even airport, click the "Add to storage pool button", and suddenly it just acts like part of your system drive. You don't have to worry about what is stored where.

Standardizing file systems

[failedlogic]

I don't know a whole heck of a lot of the technical details on ZFS. What I have read and understood, it sounds like what ZFS offers is something that every OS should include in its file system. Since, as I understand the BSDs and many Linux distros are starting to include (albeit limited/beta/alpha) ZFS support, and the long-rumored OS X inclusion being confirmed, could this be a universal file system for Operating systems? I would definitely like to see ZFS as a bootable Windows file system.

Say I have a portable USB hard drive or a dead motherboard in one system and want to retrieve the data off a hard drive. One computer has Windows and the other is Nix or OSX. Generally, the file system one could use that *should* work between Windows, Mac and 'Nix was Fat32. There are some issues with FAT32, the least of which is lack of support for large hard drives. The only other ways I can think of transferring the data are via Network or using a OS hook to read the data.

I just switched from Apple to Windows. I've been using an app to read my HFS+ file system on Windows to get data off the hard drive. It works well, but its not build-in. Nor is read/write NTFS access in other OSes. In any case, getting the data has been a bit of a pain. A standard file system I can just plug in a drive no problem would be awesome.