Slashdot Mirror
Apple Discontinues ZFS Project
Zaurus writes "Apple has replaced its ZFS project page with a notice that 'The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.' Apple originally touted ZFS as a feature that would be available in Snow Leopard Server. A few months before release, all mention of ZFS was removed from the Apple web site and literature, and ZFS was notably absent from Snow Leopard Server at launch. Despite repeated attempts to get clarification about their plans from ZFS, Apple has not made any official statement regarding the matter. A zfs-macos Google group has been set up for members of Apple's zfs-discuss mailing list to migrate to, as many people had started using the unfinished ZFS port already. The call is out for developers who can continue the forked project."
Daring Fireball suggests that Apple's decision could have been motivated by NetApp's patent lawsuit over ZFS.
Now if you're using zfs on Mac OS, you can't complain if it loses your data. You already knew it was forked.
Please hand in your geek card as you leave.
http://en.wikipedia.org/wiki/ZFS
Do what thou wilt shall be the whole of the Law
Posting anon, lest someone guess who my sources are.
The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money for giving it to Apple under different terms and the amount they wanted was in the range of "hell, we could do it ourselves for that".
Add to that, the Oracle buyout and Sun going into management paralysis, and Apple decided to go it alone.
Apple's CoreOS team includes several of the lead engineers from the ZFS project (who fled the remnants of Sun in the Schwartz melt-down), and the architect of the BeFS. I'm expecting Apple to do their own next-generation file system, probably in the 10.7 timeframe.
My sense of entitlement demands that everyone hand everything to me on a silver platter as well! I shouldn't be required to click on a link, much less do a Google search. You and I are in agreement. BTW, you owe me for that.
God forbid the summary tell us what ZFS is
It is a filesystem, available in (Open)Solaris and at least FreeBSD, possibly other BSDs as well. It has some interesting features, which you can check out here. I have heard it claimed that ZFS has good performance, but I have not evaluated any of those claims.
SSC
I doubt that it's a legal issue as the primary reason that this has happened, especially considering that the project seems to have stagnated steadily in successive versions of OS X. There just doesn't seem to have been the will within the OS X development group to make this work and to support and fully integrate ZFS into the inner workings of the OS. Given the pretty extensive functionality and plumbing of ZFS its probably been too much of a big ask to integrate a filesystem like that into a desktop. They might well have come to the conclusion that ZFS was simply complete overkill on a desktop and that it just wasn't possible.
However, they still desperately need a next generation filesystem and according to the linked article they're hiring filesystem engineers. I don't see any evidence that this was anything other than a technical avenue that they've explored that has fallen by the wayside as so many have before.
Dustn Sallings put the code on Github and has already hacked some basic Snow Leopard support and a minimal installer:
http://dustin.github.com/2009/10/23/mac-zfs.html
Code's here, fork away:
http://github.com/dustin/mac-zfs
All this time, I thought folks were talking about file management with a phony French accent!
Save zee file to zee FS and you will see that zee bytes go ...
It's NOT me! It's the meds! I'm on 1000mg of Fukitol.
THIS is an example of what deserves to get a "+1, Funny" mod, not the ten thousandth retelling of an "in Soviet Russia" joke or some other tired meme.
Of course, there's also no better way to prove his point than to get offended by it.
It is a miracle that curiosity survives formal education. - Einstein
Interesting - we're chugging happily along in Linux / Windows / Mac / Unix land having a load of competing filesystems where all the popular ones have *roughly* similar capabilities. Then ZFS appears in OpenSolaris and filesystem design becomes cool again. Everyone starts either porting ZFS or making filesystems with similar features ... Now a major player that actually *had* ported ZFS (somewhat) is seemingly deciding to go it alone. It seems as though the next-gen filesystem space is also going to have a variety of competing filesystems.
I generally think this is a good thing, lets just hope that a reasonable degree of interoperability becomes possible anyway.
When SSDs come down A LOT in price, and up in size, maybe.
Go do a search on Newegg. Biggest they've got is 256GB, of those, the cheapest is $595. You can get several terabytes for that price with a magnetic hard drives.
SSDs have a place, but as a general replacement for magnetic hard drives they are too expensive with too little capacity.
There is also more to the file system than access speed.
Well, technically, they do, I don't think WinFS has been nuked yet.
It might as well be. Better odds of seeing Duke Nukem Forever.
Hearing that ZFS support was upcoming in Snowleopard is one of the things that encouraged me to switch my desktop from Windows XP to MacOS.
It is an understatement to say i'm disappointed to see Apple abandoning this.
Support for ZFS is not just a little feature checkbox, it's a major component of the OS.
It'd be like if Microsoft dropped/cancelled support for Solitaire from Windows....
No shit. I can't wait to switch to Windows 7 with the totally awesome WinFS.
Do you even lift?
These aren't the 'roids you're looking for.
Leaving aside all the crazy storage pool stuff (great for servers, not necessarily that useful for desktops), there are some interesting features in ZFS that I hope make their way into Mac OS X in some filesystem.
Snapshots and Copy-On-Write filesystem clones seem like a great way to improve the Time Machine backup feature, and would make it easy for applications to provide backup-on-save very efficiently.
The compression and encryption features would likely be useful for some people. I don't think the increased filesystem limits (number of files, size of files) would matter for most folks.
I thought it referred to "Zaphod, For Sure!" which is Zaphod Beeblebrox's Re-election slogan.
... and then they built the supercollider.
Don't forget the power savings. I just got a new 24" LG panel with LED backlighting that, I believe, uses only 26W at full power. Compared to the old 19" CRT it replaced, it's a huge savings (that one used over 140W I think). Not only does this affect my power bill, but also the temperature in my poorly-ventilated office. Now it doesn't get so darn hot in there. (Yes, I could turn up the A/C, but then the rest of the house would be freezing, so that doesn't make much sense.)
The picture is nice and bright, and as you'd expect with an LCD, not distorted at all, as I've found most large CRTs to be. Yes, CRTs definitely have better color (at least better than the 6-bit TN panels which are common for LCDs), but ones supporting 1900x1080 or better resolution are downright gigantic, and use a ton of power, and the picture seems to have distortion problems too. I really don't miss having to muck around with all the image adjustment (centering, size, moire, trapezoid, etc. etc.) controls that I had to on every CRT I used.
Steve Jobs commented on HAMMER FS inclusion in 10.7 during WWDC 2009. He said "due to legal and technical constraints we can't touch that"
A lot of confusion has resulted from labelling ZFS a "filesystem". It actually combines both volume management and filesystem layers to achieve unique levels of performance, manageability, and data protection. Merits close study, as the concepts of ZFS overtake current best practices, conventional filesystems and RAID. You can get this taste of the future today, if you're using Solaris 10/OpenSolaris/FreeBSD.
you had me at #!
Too bad for Apple, not for ZFS. OpenSolaris and FreeBSD support ZFS just fine. I do think it's best suited to servers, and OpenSolaris and FreeBSD are greatly superior server operating systems anyway.
Thanks for breaking it down. Too bad Apple had to stop HAMMER time.
Yeah, but a lot of people don't need terabytes worth of storage. I just built a machine for my wife with a 64 Gb SSD, which cost $180. She's currently only using about 25% of the space. The good thing is that the machine is really fast on some tasks she does that require a lot of hard-disk access.
Find free books.
SATA attach SSD has achieved price parity with enterprise SAS, the density is almost there, and the performance completely blows it away. We're not at the end of spinning disc, but you can see it from here.
The new performance tier of storage is PCIe attach SSD. At two terabytes of storage and 1.5GB/s per slot, we're getting close to what we used to get from Ramdisk in performance and adequate density at 3TB per rack unit including server (HP DL785 G5 or equivalent). Yes, this is expensive right now, but the performance tier always has been. This is for trading platforms, HPC and such. These are approaching 2M IOPS and 40TB per 7U server.
The second tier is 2.5" 256GB SATA SSDs. You get 3TB per rack unit including the server. About the same cost as SAS for 10x the performance. Software options enable you to scale this to infinity in both bulk and performance. Great for databases, VMDK files and iSCSI. Get the hot-swap version and leave some open bays so that when the 1TB 2.5" SSDs come out you can migrate your LUNS with no downtime.
The third tier is SAS spinning disk. At something like 20TB/Rack unit (excluding servers) you can use this to serve frequently used files.
The fourth tier now is SATA spinning disk. At roughly the same density as SAS spinning disk for one-fourth the cost, this is a good candidate for deduplicated targets like virtual tape libraries or deduplicated NAS. It's also a good place to store your snapshots. With modern snapshot technologies there's no good reason to not store snaps every 15 minutes or so. Typically you would park this storage offsite for DR purposes so you can avoid the Premium Microsoft danger eXperience(**).
Storage pros probably would note that I neglected to mention tape and Fiber Channel. That's neither accident nor ignorance. The only reason for tape is legally mandated tape backups, and I consider this the IT equivalent of legally mandated hitching posts outside every business (which laws persist in some places) - if you gotta, you gotta, but there's no reason any more to consider it a necessary or good practice. As for Fiber Channel, it just doesn't fit in the model any more. I know this hurts the feelings of folks who just dropped a million bucks for a single rack of SAN storage with 100TB, or worse - popped for the new 8GBit stuff complete with a converged ethernet/FCoE solution, but it's true. There's just no reason for fiber channel any more. It just doesn't have the bandwidth to support a modern storage solution and it costs too much. Sure, it's got redundancy from the disc to the file server, but so what: modern file servers use redundant storage and clustered redundancy and don't need the diminishing returns of embarassingly expensive drives, head nodes, capacity licensing and annual support contracts. By the time you figure in oversubscribed ports in your FC network, you've lost the supposed reliable performance benefit of the whole thing. This isn't bad news for Cisco - they're going to sell a lot of 10Gbit Ethernet ports before they get cheap and they haven't lost anything by being also compatible with FC. It really bites to be EMC this week, but they'll figure it out.
Check the specs on this server, this card, this drive and this array. This is off-the-shelf stuff, not pie in the sky. The interconnect people need to get off their butts, but this is all doable right now. The compute side becomes an almost trivial cost of what it takes to maintain this storage bandwidth and capacity. If you like proprietary solutions HP sells a thing called the LeftHand Virtual San App
Help stamp out iliturcy.
If you're looking at hundreds of thousands of writes a day to your database, you really only need somewhere between 10 and 100 IO/second (there are 86,400 seconds in a day). Most hard drives handle that somewhat decently, especially if you use a good RAID configuration.
Looking at 100,000,000 updates a day (1,158 writes/second)? Intel's X25-M is rated at more than 4 times that
Let's compare that to a 15k.2 Seagate Savvio harddrive. Oh, right, they don't list their IOPS ratings. Let's look at what they do have though:
Intel lists these figures:
In other words, for a single track, the Intel drive will be almost 5 times as quick to start the write, and on average the Intel drive will be 38 times faster.
Or looking at it in another way, the absolute best case scenario where we simply ignore actually writing something, the Seagate drive can achieve 205,714,286 write operations per day (86,400 seconds/0.42 milliseconds). The Intel drive will hit 1.016.470.588.
While I can't find anyone benchmarking Intel's SSD offerings directly against the Savvio, I can find a mix of tests. From Tom's Hardware we see that SAS drives tops out at about 400 IOPS for any given task.
Using Tom's Hardware for a comparison, their review of the X25-M had it bottoming out at around 900 IPOS, making it perform 225% better at its worst, compared to the SAS drive's best.
Prices:
Newegg.com doesn't have the Savvio, so I'm using Google instead:
Seagate Savvio 15k.2 146 GB edition: US$ 226.44 or US$1.55/GB
Intel X25-M 160 GB edition: US$ 439 or US$ 2.74/GB
Considering the performance advantage of at least 225%, you'd have to spend at least US$ 509.49 just to get the same kind of performance as you'd get from the US$ 439 drive from Intel. And that's just their mainstream edition. AND we're talking SSD's worst case scenario vs. SAS's best case scenario. Realistically we're talking much greater advantages for the SSD.
And you keep talking about "commodity SSDs" but refer to datacenters. A commodity harddrive is a 7.200 RPM 8 MB SATA drive, and they aren't suitable for a datacenter either. Duh! So why the fixation of comparing commodity hardware from one technology to enterprise hardware from another? Stop buying commodity hardware for your datacenter needs.
And yet you haven't caught on to the fact, that this isn't that big of a problem. Anandtech wrote an excellent paper on write performance problems, and his benchmarks are based on used drives (the drive has to perform deletes before writing), and he got these performances:
The VelociRaptor i