Slashdot Mirror
Sun CEO Says ZFS Will Be 'the File System' for OSX
Fjan11 writes "Sun's Jonathan Schwartz has announced that Apple will be making ZFS 'the file system' in Mac OS 10.5 Leopard. It's possible that Leopard's Time Machine feature will require ZFS to run, because ZFS has back-up and snapshots build right in to the filesystem as well as a host of other features. 'Rumors of Apple's interest in ZFS began in April 2006, when an OpenSolaris mailing list revealed that Apple had contacted Sun regarding porting ZFS to OS 10. The file system later began making appearances in Leopard builds. ZFS has a long list of improvements over Apple's current file system, Journaled HFS+.'"
One file system to rule them all.
But they killed that project.
Mmmm... Boiled Oceans!
More
Well, not in THIS forum. But elsewhere.
/ 06/sun-ceo-jonathan-schwartz-zfs-to-be-the-file-sy stem-in-leopard
5:1 that it's not the default root file system in Leopard.
The first bootable release of ZFS (not "BUILD," but "RELEASE") isn't even due until the Fall.
I'm not alone in this skepticism. See this Ars story, for example.
http://arstechnica.com/journals/apple.ars/2007/06
When ZFS was first mentioned in the same breath as OS X it was pointed out that at the time you couldn't boot off ZFS file systems, so people were thinking it would power external (or secondary) timemachine devices. If it's replacing everything, I'm assuming you can now boot from a ZFS drive? When was this functionality added?
He's already taken it back, more or less:
"I don't know Apple's product plans for Leopard so it certainly wouldn't be appropriate for me to confirm anything. [...] There certainly have been plenty of published reports from various sources that ZFS is in Leopard, I guess we will all have to wait until it is released to see if ZFS made it as the default, or if they simply announce that it will become the default in a future release."
"There is no time, sir, at which ties do not matter," Jeeves, (Jeeves and the Impending Doom)
Not anymore, it ain't... Now, Apple will go with NTFS just to spite them...
"I like systems, their application excepted", George Sand (French)
On my first read, I was wondered why Mark Hamil was making announcements about OS X file systems...
"There is no time, sir, at which ties do not matter," Jeeves, (Jeeves and the Impending Doom)
I have a particular gift for breaking things that are supposed to be reliable. (Admittedly, mostly through ignorance, but I digress)
Besides the worrisome concepts of delayed writing and an always consistent file system, I can imagine never being able to bring a zfs back pretty easily. Which, the snapshots are supposed to solve, but pretty soon, my hardware storage budget just went through the roof because I'm storing terabytes of snapshots pretty quickly.
Got Trader Joe's? friendwich.com RSS feeds work now!
does the user get a choice? Having access to more filesystems is a plus, but I wouldnt want to replace HFS+ for ZFS personally. So hopefully you can choice which one you want. Similiar to how Windows lets you choice FAT32 and NTFS, or OS X you can do HFS or HFS+, or Linux with ext2, ext3, reiser, etc.
Jobs is probably not happy about his thunder being stolen right before for the June 11th keynote
I strongly doubt he didn't know about it. This is Jonathan Schwartz, not a OS X rumors blogger. At any rate, ZFS in OS X is Sun's thunder; Time Machine is Apple's thunder, and that's already announced. How many OS X users (other than slashdot readers) will care in the slightest about the underlying filesystem? What they care about are the features, like Time Machine, that it enables.
Yes I'm sure it will be worth it in the long run but I'm not looking forward to yet another hiatus in which: no industrial-strength disk-recovery tools are available, in which accidentally running the wrong disk-repair tool on the wrong partition hoses it instead of fixing it, and in which yet more legacy software suffers breakage due to subtle incompatibilities in implementation.
(Yesyesyes, I know, ZFS is reliable that disk-recovery tools are not needed. And if you believe that, then you probably believed Microsoft when they said NTFS volumes never needed defragmentation).
Dear Apple:
Please let HFS+ still be an option.
Please let Classic still run on Power Mac processors.
Please let reasonably well-behaved software that uses resource forks still work.
Please let it be case-insensitive and case-preserving.
"How to Do Nothing," kids activities, back in print!
Maybe some in the know (not me) could fill us (people like me) in... Are there other benefits that will come from moving to ZFS? I'd guess that for the average consumer any performance gain, or loss, won't really make a difference, but what about those running servers or doing heavy video/audio work? Or are there other aspects of this filesystem that will make it that much better than HFS+?
...mentions that ZFS is from Sun?
-- Boycott Shell
Once we're sure it's stable, because it looks like a massive improvement over the 1970s-style file systems we're using now. ZFS is now part of FreeBSD, Solaris will have ZFS "soon" and many Linux distros are also considering it. Good. Let's get to a common standard that's excellent and forget the tedium of these past, less effective file systems.
technical writing / development
If ZFS is the default file system, it will mean that Time Machine (i.e. the snapshot feature) of 10.5 will be able to take snapshots without requiring a secondary file system to keep the copied (recoverable) blocks, as it does now with HFS+. To me, the secondary filesystem requirement makes Time Machine essentially useless on a laptop.
I think this is true. ZFS is maybe the biggest of Leopard secret features that will be revealed next week at the WWDC. I believe this is the real reason Leopard was delayed.
If it's really the default file system on OS X 10.5 it really is big news.
We will see.
I think Slashdot would benefit from adopting some of K5's approach to story submissions. The Firehose is a great start, but instead of simply saying yes or no, users should be able to give feedback to the submitter. The summary for this article is a great example. The submitter typed "build" instead of "built," resulting in an annoying distraction in an otherwise concise description of the story.
Newspapers have Copy Editors (at least they used to; most seem not-too-bothered by spelling these days). It would be nice if interested Firehose users were given the opportunity to help make sure the summary was fit for publication before it hits the front page.
I guess this should have been a journal entry, but it seemed like an opportune time to bring this up.
I don't care why you're posting AC
I know its not on Mac but this shows how easy and powerful ZFS is. I have heard directly from Sun that by Solaris 10 will soon have bootable ZFS either in update 4 or update 5. Remember that the big problem with Sun hardware is that they need firmware support for bootability and that it may be much easier on OS X to make ZFS bootable. http://video.google.com/videoplay?docid=8100808442 979626078
This will make going from earlier versions of OSX to the new one more of a pain because the whole disk will have to be reformatted.
Back, nazi, back!
How will ZFS work with Bootcamp?
When is Leopard coming out? I know it's been delayed, maybe they're testing it with ZFS?
Democrats or Republicans. They are both taking us to the same place and they are not afraid of us anymore.
Performance:
Suppose I want to access a file.
First, the filesystem looks it up. This operation takes time proportional to the log of the directory size. Maybe you do better with hashes.
On a case-sensitive (POSIX-compliant) filesystem, you're done. You have the file, or you can return an error code.
On a case-insensitive filesystem, your done if you're lucky. If not lucky, you need to do a linear scan of the whole damn directory. Many places have a directory with some insane amount of files. Intentionally or not, it's common to go into the tens of thousands. A few places (running XFS mainly, sometimes Reiserfs) get into the millions.
Because of the way directory listings are done (read then look up stats) you can generally square the above numbers. Ouch.
I18N:
Then there is the issue of internationalization. For example, consider "I" and "i". Some places have an uppercase with the dot, and other places have a lowercase without the dot. The rules for uppercasing and lowercasing differ from what most people are used to. Oh crap! This issue doesn't exist on a case-sensitive filesystem.
Safety:
App needs to make a file. App sees that file does not seem to exist. App writes file. Complex international case rules mean that no, the file DOES exist, and it gets clobbered.
There is no need to 'stick the knife' in! ;-)
I'd never heard of ZFS before, but the summary made it pretty obvious to me that it was from Sun (an announcement from the Sun CEO, and a prior request that Sun port it for them).
This is just total utter complete speculation on my part, but one of the features of ZFS I like is the pool concept where you just add disks to the total storage space and then carve it up any way you see fit. I tested this with Solaris 10 on VMware and it really is just that easy...add the disk to the machine, then register it with the pool. Done. No muss, no fuss. I was shocked at how fast and easy it was.
Anyway, where I'm going with this is: Why should we still have disks on the desktop? If all the physical disks are just part of one big pool, you do away with needing to see the various disk icons and instead have something maybe similar to "This computer" with a different organizational structure; you see Applications as a group (a la the original Windows Program Manager) while your home directory stays the same in the nested-folder style.
What I'm really wondering is if ZFS could be an integral part of a totally redesigned Finder interface that allows for Apple to get away from the traditional Disk/Directory paradigm. I know Unix in general really doesn't support a disk paradigm at all (which is why I've always preferred Unix over Windows...none of the C: or D: business) and it'd be great to have the same thing on the Mac with Apple's capability for superior interfaces.
...The Schwartz meant "THE" as in "it's THE filesystem all the COOL KIDS will be using." He didn't necessarily mean the ONLY filesystem.
Oh, and can you imagine a beowulf cluster of these things?
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
can volumes be dynamically resized, which would be needed in this case
The Boot Camp setup does exactly that (shrink existing partitions on Intel macs).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
It's easy to add custom filesystems to ZFS, so you could have something like a filesystem that represented a remote photo sharing site as a directory (that's kind of a lame example, but I hope it serves to illustrate the ability).
Also ZFS supports automatic low level compression, so disk reads/writes can be faster since you are reading compressed data off a disk (disk reads are a lot slower than decompression).
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I'm a new ZFS user, and so far I'm very happy with it.
But the bigger picture here is that this is NOT a YRO.slashdot.org article. Its just a news article.
No patent disputes. No lawsuits. Just technological progress and cooperation.
Why can't this be more of the status quo?
I've been reading stuff about predictions by scientists, and its still universal that hardware will increase at the current rate of about doubling in power ever 12-18 months.
Software performance is about 1/2 of that at best. If more cooperation like this could happen, then there is no reason for software to lag behind hardware, but one large software company keeps hindering this progress...
I think ZFS on Mac is a great idea, and I'm not even an Apple Fanboy. What I want is ZFS ported to Linux, (not via FUSE) but that would require Sun to change or dual license ZFS to allow it. (Currently under the CDDL license I beleive)
Seems like Apple take some of the best ideas from the Unix world. Really shows the potential of Unix systems if the people who wrote them thought a little more about usability.
I'm interested in hearing if ZFS would effectively replace Xsan. Anyone know?
I am using ZFS-FUSE right now. On my gentoo system, many partitions are zfs, including /home, /var/tmp, /usr/share, /usr/portage, and /opt
Because I have suffered some random corruptions in the past, even with ext3 ("This mp3 didn't used to have a skip there!"), I wanted the checksumming so that I can tell when I need to restore something from a backup.
As a filesystem, it works completely, including creation of new filesystems, compression, checksums, etc. However, I've noticed a decrease in my system's general performance since installing zfs (probably due to it holding my home directory). Memory usage and mysterious CPU usage (I don't think it's checksumming) are the current disadvantages, but the author says it's still completely unoptimized.
Should you try zfs-fuse? Definitely. But right now don't expect a performance gain.
A cat can't teach a dog to bark.
You have know way to know if bits in your movie files are being corrupted with FAT32 unless you manually keep around checksums. ZFS does that for you. It will detect silent corruption of your data.
Apple is "well known" for massive backwards compatibility updates... except they aren't... They always handle transitions over a couple of versions, intelligently bringing people along. They swapped processor architectures twice and each time brought people along with emulators, in the Intel case it wasn't faster than the fasted G5 machines, but those of us upgrading 3+ year old machines (Powerbook G4 1Ghz -> Macbook Pro in my case) found our PPC apps running faster and Intel code flying.
We all expected the Intel migration to happen with 10.5, they shocked us when they did it off the 10.4 base.
While they did abandon Mac OS to move to OS X, they provided a migration strategy (Carbon) and a compatibility layer (Classic). Classic support shipped with 10.0/10.1, 10.2, and was supported in 10.3 if you already had it, as well as 10.4 I think, but they kept classic for around 5 years, which gave everyone time to migrate to Carbon. Its unfortunate that there is no long-term Classic via Rosetta just from a classic application point of view, but they didn't leave anyone in the lurch.
I expect 10.5 to introduce this OS, which will be useful for new installs, or for external drive arrays, especially for the Video market, but I wouldn't expect it to be the default. OS X has supported a Unix filed system, but defaulted to HFS+, because HFS+ was compatible with Mac OS, so you could dual-boot OS 9 and OS X for a good 2 years on new hardware to maintain compatibility. If they hadn't done that, they would have lost the Pro-Audio and Pro-Video markets that took a few years to get native OS X applications.
Getting it in the wild and for professionals would help that market, while not breaking ANYONE's compatibility. Sometime in 10.5's lifetime they may ship new computers with it, or they may wait for 10.6 in two years. But giving everyone two years is plenty of time to get utilities and applications compatible with the new file system.
The flashy consumer features are touted for the OS, but the underlying architecture has always followed a 2-cycle release. If you've used OS X Server for 10.2/10.3/10.4, you'd notice that they introduced stuff in one version with limited exposed functionality (with the rest via the Unix layer), enhanced the functionality in the next rev, and polished thereafter.
The Apple Mail Server -> Cyrus migration was someone poorly handled, but mostly because AMS was garbage. But the 10.4 Mail tools are night and day beyond the 10.3 ones.
They are actually far more careful than people give them credit for.
The different is, they don't keep backward compatibility as a long-term goal, they do a two-stage migration, giving people 2-4 years to transition.
Discuss.
Website Hosting
Did Apple use ZFS under CDDL or did they negotiate a custom license with Sun, i.e. is Apple's ZFS implementation open source or not?
So now if I were to get a Mac and triple boot Windows OSX and Linux...
Linux will only be able to read the Windows NTFS partition.
Linux won't be able to do anything with the OSX partition (unless I use FUSE).
Windows won't have access to anything other than itself.
And I'm not sure about OSX, I've never used it.
Is that correct?
go back to Digg
I'm not sure a boot device / standard first partition for a laptop is where ZFS shines, or where Apple would put a lot of focus. However, look at the server space. Lots of people like XSan. Products like Final Cut Server demand huge piles of storage, often XSan, and could use a lot of the properties of ZFS file systems. They could also be exported directly as file systems and mounted on desktops and laptops.
"Default" in this case may mean the default for your XServe RAID, or XSan, but not for your laptop.
I agree. I am the submitter of this story btw. Something weird happened. Only the first half of the description is mine, the other half was taken form another firehose article. My closing line was "Jobs probably doesn't like someone stealing his thunder for the keynote" and it doesn't appear in the current summary. But oddly enough some people in the discussion are referring to that line, so this line must have been taken out after the story was already up... ;-)
I did make the d/t error though. I am a dyslexic and a non-English speaker, so you could say I'm double handicapped.
This sig is just as redundant as the rest of this posting
For something that's only a year or so old (production wise), I don't trust it worth shit.
/dev/null > /path/to/file', then, once you have some blocks free, rm works (so does unlink).
We run Netbackup Enterprise on Solaris 10 - during our last round of upgrades we installed ZFS on our disk staging storage units. It replaced VxFS. The way disk staging storage units (DSSUs) work in Netbackup, the disk is always near 100% full form a unix perspective. Basically, any time more disk is needed, the oldest image that has been copied to tape is expired from disk, thus freeing up more room. However, ZFS's most prominent bug from our perspective is that during periods of high activity, if all blocks become allocated, it becomes impossible to unlink(2) a file. This causes the application to no longer be able to make space for new backup images.
Going down the shell, try to rm a file and it comes back: rm failed, disk is full.
Well, if the disk is full, and you can't rm because the disk is full, how do you free up space?
Sun's response, truncate an unnecessary file using 'cat
Ok - so how do you tell a compiled application to truncate an unnecessary file before unlinking it? You can't! How can you determine what an unnecessary file is? If you delete the image before expiring it from the catalog you get errors when you try to expire, so you end up with catalog corruption.
All in all, this is a problem that should never have been introduced, let alone still exist after months of sending trace outputs and reproducing it in multiple environments. ZFS isn't ready for the real world.
ZFS is not ready for prime time - at least not on Solaris.
I setup ZFS on some SAN storage in a new system. The internal boot disks were mirrored UFS. When one of the HBAs fried, the SAN storage disappeared - and the system panic'd.
Every reboot thereafter stopped in a panic. The ZFS subsystem panic'd the system at every boot when it couldn't find all its volumes. After calling Sun support, I found out that they need to do a massive code redesign to catch that issue, and it wouldn't be out for at least 6 months.
I'm sure ZFS will be great - once they clean up these type of showstopper bugs.
Jeremy Baumgartner
I think this is a bad decision for Apple. ZFS cannot be incorporated into the Linux kernel due to Sun's choice of license, and Solaris isn't widely used compared to Linux.
Regardless of whether ZFS is actually "better" than other file systems (I don't think it is), choosing ZFS makes Macintosh even less compatible with the mainstream FOSS world than it already is.
Time Machine is already fully functional (apart from a few gui glitches) in the current leopard developer builds, but ZFS isn't even available in Disk Utility (yet?). This doesn't mean ZFS won't be added at the last minute, but it certainly isn't required for Time Machine.
... booting OS X on a Mac and booting Solaris 10 on a *** are completely different questions.
I don't see why Apple couldn't complete this functionality on that schedule: They know their operating system and integrated hardware intimately, are not burdened by Sun's qualification requirements, and have had considerable time (including the post-iPhone extension) to work on it.
5:1 they ship with ZFS boot.
you had me at #!
Why would you possibly prefer HFS+ to... well... anything else that OSX might support? Except maybe FAT32...
Don't thank God, thank a doctor!
they shocked us when they did it off the 10.4 base.
It did not shock those of us who know that NEXTSTEP was transparently portable to at least four architectures.
you had me at #!
After enough years of use, don't you think someone would've noticed if ZFS didn't come with the equivalent of fsck?
I'll bet money that ZFS will be both more reliable, and easier to recover than HFS+, except that it may take a little while before someone has a recovery GUI for you. Probably not, though -- it'll probably be in Disk Utility in the first release.
Don't thank God, thank a doctor!
..."Steve says 'It seems like a good idea' "
... be like, to allow all the various ZFS configuration options to be managed, and yet be simple and (relatively) idiot-proof?
Seems to me that the design and implementation of THAT would take longer than to insert ZFS as the primary filesystem.
I wonder whether this is really more than a research project within Apple. There is evidence that Apple has been investigating ZFS, but you have to take any third party claims about the design of future OS X releases with a fist sized salt crystal. Don't be surprised if Apple waits a few days and issues a 'clarification' on the matter. Jonathan Schwartz has set himself up for an embarrassment and, frankly, he deserves a slap for being so naive. They'll deny it even if incorporating ZFS is their intent. The economy sized egos at Apple will not tolerate being both up-staged and obligated by others. Whatever amount less of a splash Apple gets to make on its own as a result of Jonathan's public demonstrations will be accounted for in full, plus interest.
Lurking at the bottom of the gravity well, getting old
I really dont understand the hype about ZFS. Isnt it just the basic LVM + filesystem that weve had in UNIX (well, not Solaris) for years? Say for example in True64 (advfs/lvm) or AIX (jfs/lvm). I can stripe and mirror at a PP level any way I want. Snapshots and copies. What does ZFS provide that LVM + JFS doesnt? Can someone in the know summarize for us lazy? :)
You'll probably be hashing filenames. I guess you can do that all lowercased, which may be OK for Apple.
That doesn't work for Windows, which supports both Win32 and POSIX. You need something two-level, first case-insensitive then case-sensitive. (there are worse alternatives) Ugh.
All of this is with per-filesystem case conversion, because filesystems can come from different locales.
In one locale, the lowercase of "I" is the same as the lowercase of "I" with a dot. Both have dots. In another locale, they are distinct.
In one locale, the uppercase of "i" is the same as the uppercase of "i" without a dot. Both are a plain "I". In another locale, they are distinct.
Thus:
In one locale, two filenames are distinct. In another locale, they are not distinct. Oh crap. Now try sharing removable media or a network filesystem between the two locales.
Leopard installer asks right at the start if you want to create a backup of your disk.
So just plug in an external Firewire/USB disk, do the backup, install Leopard with a reformat of disk to ZFS. Then when the Installer asks if you want to copy data from another computer/disk, just point at the backup. You end up with a clean install on ZFS with all your users, apps, data copied over...
Much simpler and less error prone than trying to come up with a way of converting HFS+ volume to ZFS
If Apple wanted a better file system they would ALREADY be using UFS.
I'm really sick of HFS+. Three times now I've had it eat its brains so bad that FSCK wouldnt fix it. I have NEVER, not in 20 years that I've been working as a network administrator, had anything even vaguely resembling a modern UNIX file system get so messed up that fsck *refused to work*, not unless the disk was completely destroyed. That hasn't been a problem since ncheck and clri got replaced with fsck back in the '70s!
And in three years, HFS+ ate its own brains three times... and that's just with my own personal hardware.
That's worse than Windows.
When they announced that Panther was getting an updated version of UFS from FreeBSD, I was hopeful. They *almost* had all the HFS+ extra functionality working over UFS in Jaguar... including aliases. But NO, they decided they were going to make the system even more HFS+-specific by putting the Spotlight hooks in the file system instead of the vnode layer.
So until I actually see this come out from Apple, I'd take it with a grain of salt. They've had an appalling tendency to hold onto old bad ideas long past their freshness date.
On a case-insensitive filesystem, your done if you're lucky. If not lucky, you need to do a linear scan of the whole damn directory. Many places have a directory with some insane amount of files. Intentionally or not, it's common to go into the tens of thousands. A few places (running XFS mainly, sometimes Reiserfs) get into the millions.
And if you don't use paths, none of this applies at all. Paths are evil, slow, and fragile. Abstracted references, ftw! One for non-persistent data, one for persistent data. Yayness.
No, no and no.