Slashdot Mirror
Apple Discontinues ZFS Project
Zaurus writes "Apple has replaced its ZFS project page with a notice that 'The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.' Apple originally touted ZFS as a feature that would be available in Snow Leopard Server. A few months before release, all mention of ZFS was removed from the Apple web site and literature, and ZFS was notably absent from Snow Leopard Server at launch. Despite repeated attempts to get clarification about their plans from ZFS, Apple has not made any official statement regarding the matter. A zfs-macos Google group has been set up for members of Apple's zfs-discuss mailing list to migrate to, as many people had started using the unfinished ZFS port already. The call is out for developers who can continue the forked project."
Daring Fireball suggests that Apple's decision could have been motivated by NetApp's patent lawsuit over ZFS.
Now if you're using zfs on Mac OS, you can't complain if it loses your data. You already knew it was forked.
Please hand in your geek card as you leave.
http://en.wikipedia.org/wiki/ZFS
Do what thou wilt shall be the whole of the Law
Posting anon, lest someone guess who my sources are.
The long and short of it was, Apple and Sun couldn't come to terms on the licensing. Sun wanted a lot of money for giving it to Apple under different terms and the amount they wanted was in the range of "hell, we could do it ourselves for that".
Add to that, the Oracle buyout and Sun going into management paralysis, and Apple decided to go it alone.
Apple's CoreOS team includes several of the lead engineers from the ZFS project (who fled the remnants of Sun in the Schwartz melt-down), and the architect of the BeFS. I'm expecting Apple to do their own next-generation file system, probably in the 10.7 timeframe.
My sense of entitlement demands that everyone hand everything to me on a silver platter as well! I shouldn't be required to click on a link, much less do a Google search. You and I are in agreement. BTW, you owe me for that.
God forbid the summary tell us what ZFS is
It is a filesystem, available in (Open)Solaris and at least FreeBSD, possibly other BSDs as well. It has some interesting features, which you can check out here. I have heard it claimed that ZFS has good performance, but I have not evaluated any of those claims.
SSC
I doubt that it's a legal issue as the primary reason that this has happened, especially considering that the project seems to have stagnated steadily in successive versions of OS X. There just doesn't seem to have been the will within the OS X development group to make this work and to support and fully integrate ZFS into the inner workings of the OS. Given the pretty extensive functionality and plumbing of ZFS its probably been too much of a big ask to integrate a filesystem like that into a desktop. They might well have come to the conclusion that ZFS was simply complete overkill on a desktop and that it just wasn't possible.
However, they still desperately need a next generation filesystem and according to the linked article they're hiring filesystem engineers. I don't see any evidence that this was anything other than a technical avenue that they've explored that has fallen by the wayside as so many have before.
Dustn Sallings put the code on Github and has already hacked some basic Snow Leopard support and a minimal installer:
http://dustin.github.com/2009/10/23/mac-zfs.html
Code's here, fork away:
http://github.com/dustin/mac-zfs
All this time, I thought folks were talking about file management with a phony French accent!
Save zee file to zee FS and you will see that zee bytes go ...
It's NOT me! It's the meds! I'm on 1000mg of Fukitol.
THIS is an example of what deserves to get a "+1, Funny" mod, not the ten thousandth retelling of an "in Soviet Russia" joke or some other tired meme.
Of course, there's also no better way to prove his point than to get offended by it.
It is a miracle that curiosity survives formal education. - Einstein
Interesting - we're chugging happily along in Linux / Windows / Mac / Unix land having a load of competing filesystems where all the popular ones have *roughly* similar capabilities. Then ZFS appears in OpenSolaris and filesystem design becomes cool again. Everyone starts either porting ZFS or making filesystems with similar features ... Now a major player that actually *had* ported ZFS (somewhat) is seemingly deciding to go it alone. It seems as though the next-gen filesystem space is also going to have a variety of competing filesystems.
I generally think this is a good thing, lets just hope that a reasonable degree of interoperability becomes possible anyway.
When SSDs come down A LOT in price, and up in size, maybe.
Go do a search on Newegg. Biggest they've got is 256GB, of those, the cheapest is $595. You can get several terabytes for that price with a magnetic hard drives.
SSDs have a place, but as a general replacement for magnetic hard drives they are too expensive with too little capacity.
There is also more to the file system than access speed.
...who don't even *have* a next-generation filesystem project to cancel.
(see - I can troll, too!)
Hearing that ZFS support was upcoming in Snowleopard is one of the things that encouraged me to switch my desktop from Windows XP to MacOS.
It is an understatement to say i'm disappointed to see Apple abandoning this.
Support for ZFS is not just a little feature checkbox, it's a major component of the OS.
It'd be like if Microsoft dropped/cancelled support for Solitaire from Windows....
No shit. I can't wait to switch to Windows 7 with the totally awesome WinFS.
Do you even lift?
These aren't the 'roids you're looking for.
Leaving aside all the crazy storage pool stuff (great for servers, not necessarily that useful for desktops), there are some interesting features in ZFS that I hope make their way into Mac OS X in some filesystem.
Snapshots and Copy-On-Write filesystem clones seem like a great way to improve the Time Machine backup feature, and would make it easy for applications to provide backup-on-save very efficiently.
The compression and encryption features would likely be useful for some people. I don't think the increased filesystem limits (number of files, size of files) would matter for most folks.
http://en.wikipedia.org/wiki/HAMMER
Should be under a suitable license for their usage. It's written for DragonflyBSD which has a funny filesystem driver interface but AIUI the developer had ports to other OSes in mind, so it should still be doable. It can do cheap filesystem snapshots so it would support Time Machine-style operation well. The question is whether it could be adapted to fit Apple's uses well enough. Given one of the linked articles suggests Apple are hiring FS developers my guess would be that they've decided they'd rather build a ground-up filesystem that supports all the (slightly odd set of) features MacOS X wants.
I thought it referred to "Zaphod, For Sure!" which is Zaphod Beeblebrox's Re-election slogan.
... and then they built the supercollider.
Don't forget the power savings. I just got a new 24" LG panel with LED backlighting that, I believe, uses only 26W at full power. Compared to the old 19" CRT it replaced, it's a huge savings (that one used over 140W I think). Not only does this affect my power bill, but also the temperature in my poorly-ventilated office. Now it doesn't get so darn hot in there. (Yes, I could turn up the A/C, but then the rest of the house would be freezing, so that doesn't make much sense.)
The picture is nice and bright, and as you'd expect with an LCD, not distorted at all, as I've found most large CRTs to be. Yes, CRTs definitely have better color (at least better than the 6-bit TN panels which are common for LCDs), but ones supporting 1900x1080 or better resolution are downright gigantic, and use a ton of power, and the picture seems to have distortion problems too. I really don't miss having to muck around with all the image adjustment (centering, size, moire, trapezoid, etc. etc.) controls that I had to on every CRT I used.
A lot of confusion has resulted from labelling ZFS a "filesystem". It actually combines both volume management and filesystem layers to achieve unique levels of performance, manageability, and data protection. Merits close study, as the concepts of ZFS overtake current best practices, conventional filesystems and RAID. You can get this taste of the future today, if you're using Solaris 10/OpenSolaris/FreeBSD.
you had me at #!
I suggest you drop MacOS like a hot potato, send a nastygram to Apple giving them a piece of your mind, and check out both OpenSolaris and FreeBSD. They both support ZFS, OpenSolaris because Sun invented ZFS, and FreeBSD because they have competent management AND engineering. Unlike certain others (and I'm not pointing the finger at linux).
Once again, FreeBSD has shown the fools in Cupertino how it's done.
Too bad for Apple, not for ZFS. OpenSolaris and FreeBSD support ZFS just fine. I do think it's best suited to servers, and OpenSolaris and FreeBSD are greatly superior server operating systems anyway.
Grab a small SSD for apps/games and a 5400RPM terabyte disk for all your music/movies/series/home video/whatever. 64/80GB disks seems to be the sweetspot now, which even leaves some space for apps after installing Win7 ;)
Live today, because you never know what tomorrow brings
For the reason I stated: That using Sun's ZFS left them without control of development, and tracking an outside codebase has reputational risks to which Apple in particular is averse. Having ex-Sun people work on a new filesystem is great, but they still need to navigate the patent minefield that Sun has sown around ZFS.
Interesting that Sun non-competes did not stop their engineers walking down the street to work on directly competitive technology... (First I heard that engineers left Sun for Apple, actually. I thought the ZFS team was quite small, and it is obvious from the list that the key people remain at Sun.)
you had me at #!
Must not be as good as what you are smoking. You are being simplistic.
Total IOPS on SSD's are higher, but that is mostly READ IOPS and not WRITE IOPS. SSD's have typically performed very poorly with random writes. Obviously you don't understand that they mostly refer to TOTAL IOPS because it sounds good. What about the write IOPS? You are conveniently leaving that out. That and the fact that in some cases read IOPS and write IOPS on some SSD's are not anywhere near each other. This creates problems in a server environment depending on what you want to do. Since we are talking about ZFS here, I doubt we are talking in the context of a home user.
Quite recently write endurance and wear leveling were major concerns. There is no way that a generation 1 SSD could ever compete against SATA in the datacenter when you take everything into consideration. AFAIK, they still have not fixed the write performance in commodity SSD's on the market. FusioIO and OCZ have taken steps, expensive steps, to bring their offerings with much better write performance.
Theoretically, if SSD was that much better we would see people making RAID setups with them and seeing stellar results. Yeah.... Quite a few people report that the performance was really slow when logic told them it was going to be really fast.... I wonder why.
RAID can benefit you depending on what RAID you are using and WHY you are using it. You need to consider what you need first. Read performance? Fault tolerance? Write performance? Application requirements? You just can't throw an SSD at the problem and say, "Uhhh.. Solid State will solve the problem dawg".
If you are trying to create a database server that is going to be doing hundreds of thousands of updates a day... you are NOT going to succeed doing it with SSD drives.
Bottom line, if all you want is really high READ IOPS, and don't need the fast write capability, than yeah.. go with an SSD. Just don't blow smoke up people's butts by claiming its performance alone can negate the need for newer and higher performance filing systems. Puhleeeze.
People don't do enough research and buy the wrong stuff. If you need fast writes, you get the Intel X25-E. If you need fast reads, the Intel X25-M is fine. If you need the SSD to take a punishing amount of writes over the years, you get the X25-E again. If you aren't planning on punishing it with writes, the X25-M is again fine. If you need cheap, then you get an extra helping of crappy write I/O. :-)
If you want to have monstrously fast storage, you build a raid with zfs, and use one or two X25-Ms for the L2ARC cache and a mirrored (through zfs) X25-M pair for the zil cache.
I'd rather use SSD to supplement traditional storage rather than to run straight off it. But that said, I have been running my mythtv box off of an 8GB Transcend compact flash for the OS for over two years now with great results. It is a gentoo system, so I compile stuff on it all the time. Of course it has a mysql database that gets updated daily because of the schedules it pules down for mythtv. No problems there, and no regrets.
But more to your point, the problems that plagued crappy SSD controllers and designs are being worked out, and probably somewhat soonish won't be a relevant issue anymore for most people.
Yeah, but a lot of people don't need terabytes worth of storage. I just built a machine for my wife with a 64 Gb SSD, which cost $180. She's currently only using about 25% of the space. The good thing is that the machine is really fast on some tasks she does that require a lot of hard-disk access.
Find free books.
I'm sure Apple is planning something, OS X's file system is showing its age. Ars had a very good article about it here: From BFS to ZFS
SATA attach SSD has achieved price parity with enterprise SAS, the density is almost there, and the performance completely blows it away. We're not at the end of spinning disc, but you can see it from here.
The new performance tier of storage is PCIe attach SSD. At two terabytes of storage and 1.5GB/s per slot, we're getting close to what we used to get from Ramdisk in performance and adequate density at 3TB per rack unit including server (HP DL785 G5 or equivalent). Yes, this is expensive right now, but the performance tier always has been. This is for trading platforms, HPC and such. These are approaching 2M IOPS and 40TB per 7U server.
The second tier is 2.5" 256GB SATA SSDs. You get 3TB per rack unit including the server. About the same cost as SAS for 10x the performance. Software options enable you to scale this to infinity in both bulk and performance. Great for databases, VMDK files and iSCSI. Get the hot-swap version and leave some open bays so that when the 1TB 2.5" SSDs come out you can migrate your LUNS with no downtime.
The third tier is SAS spinning disk. At something like 20TB/Rack unit (excluding servers) you can use this to serve frequently used files.
The fourth tier now is SATA spinning disk. At roughly the same density as SAS spinning disk for one-fourth the cost, this is a good candidate for deduplicated targets like virtual tape libraries or deduplicated NAS. It's also a good place to store your snapshots. With modern snapshot technologies there's no good reason to not store snaps every 15 minutes or so. Typically you would park this storage offsite for DR purposes so you can avoid the Premium Microsoft danger eXperience(**).
Storage pros probably would note that I neglected to mention tape and Fiber Channel. That's neither accident nor ignorance. The only reason for tape is legally mandated tape backups, and I consider this the IT equivalent of legally mandated hitching posts outside every business (which laws persist in some places) - if you gotta, you gotta, but there's no reason any more to consider it a necessary or good practice. As for Fiber Channel, it just doesn't fit in the model any more. I know this hurts the feelings of folks who just dropped a million bucks for a single rack of SAN storage with 100TB, or worse - popped for the new 8GBit stuff complete with a converged ethernet/FCoE solution, but it's true. There's just no reason for fiber channel any more. It just doesn't have the bandwidth to support a modern storage solution and it costs too much. Sure, it's got redundancy from the disc to the file server, but so what: modern file servers use redundant storage and clustered redundancy and don't need the diminishing returns of embarassingly expensive drives, head nodes, capacity licensing and annual support contracts. By the time you figure in oversubscribed ports in your FC network, you've lost the supposed reliable performance benefit of the whole thing. This isn't bad news for Cisco - they're going to sell a lot of 10Gbit Ethernet ports before they get cheap and they haven't lost anything by being also compatible with FC. It really bites to be EMC this week, but they'll figure it out.
Check the specs on this server, this card, this drive and this array. This is off-the-shelf stuff, not pie in the sky. The interconnect people need to get off their butts, but this is all doable right now. The compute side becomes an almost trivial cost of what it takes to maintain this storage bandwidth and capacity. If you like proprietary solutions HP sells a thing called the LeftHand Virtual San App
Help stamp out iliturcy.
That must sting a bit.
"I've got more toys than Teruhisa Kitahara."
If you're looking at hundreds of thousands of writes a day to your database, you really only need somewhere between 10 and 100 IO/second (there are 86,400 seconds in a day). Most hard drives handle that somewhat decently, especially if you use a good RAID configuration.
Looking at 100,000,000 updates a day (1,158 writes/second)? Intel's X25-M is rated at more than 4 times that
Let's compare that to a 15k.2 Seagate Savvio harddrive. Oh, right, they don't list their IOPS ratings. Let's look at what they do have though:
Intel lists these figures:
In other words, for a single track, the Intel drive will be almost 5 times as quick to start the write, and on average the Intel drive will be 38 times faster.
Or looking at it in another way, the absolute best case scenario where we simply ignore actually writing something, the Seagate drive can achieve 205,714,286 write operations per day (86,400 seconds/0.42 milliseconds). The Intel drive will hit 1.016.470.588.
While I can't find anyone benchmarking Intel's SSD offerings directly against the Savvio, I can find a mix of tests. From Tom's Hardware we see that SAS drives tops out at about 400 IOPS for any given task.
Using Tom's Hardware for a comparison, their review of the X25-M had it bottoming out at around 900 IPOS, making it perform 225% better at its worst, compared to the SAS drive's best.
Prices:
Newegg.com doesn't have the Savvio, so I'm using Google instead:
Seagate Savvio 15k.2 146 GB edition: US$ 226.44 or US$1.55/GB
Intel X25-M 160 GB edition: US$ 439 or US$ 2.74/GB
Considering the performance advantage of at least 225%, you'd have to spend at least US$ 509.49 just to get the same kind of performance as you'd get from the US$ 439 drive from Intel. And that's just their mainstream edition. AND we're talking SSD's worst case scenario vs. SAS's best case scenario. Realistically we're talking much greater advantages for the SSD.
And you keep talking about "commodity SSDs" but refer to datacenters. A commodity harddrive is a 7.200 RPM 8 MB SATA drive, and they aren't suitable for a datacenter either. Duh! So why the fixation of comparing commodity hardware from one technology to enterprise hardware from another? Stop buying commodity hardware for your datacenter needs.
And yet you haven't caught on to the fact, that this isn't that big of a problem. Anandtech wrote an excellent paper on write performance problems, and his benchmarks are based on used drives (the drive has to perform deletes before writing), and he got these performances:
The VelociRaptor i
One small problem is that HFS+ sits on top of HFS and thinks like block allocations fall apart after 1TB. Apple has to switch filesystems. While the problems aren't severe at 2TB or 4TB at 50TB they are going to devastating.