Slashdot Mirror
ZFS Shows Up in New Leopard Build
Udo Schmitz writes "As a follow-up to rumours from May this year, World of Apple has a screenshot showing Sun's Zettabyte File System in "the most recent Build of Mac OS X 10.5 Leopard". Though I still wonder: If it is not meant to replace HFS+, could there be any other reasons to support ZFS?"
Isn't the term 'Zettabyte File System' actually inaccurate now? I thought they dropped that and ZFS now only remains as a pseudo initialism
I never get used to these constant resurrections
Now that Vista is finalized, expect Apple to show more and more of the 'secret' features of leopard!
--jeffk++
ipv6 is my vpn
Because if Apple showed them before, there was a risk that Microsoft tried to announce them as future features in their soon-to-be-released perfect Windows Vista ?
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
"It's too bad that stupidity isn't painful." - Anton LaVey
That would be nice, but since ZFS can't be used as boot partition even in Solaris (they'll fix it). it's better to let it stabilize a couple of releases (ZFS is a young FS even in Solaris, after all) and then switch.
"Though I still wonder: If it is not meant to replace HFS+, could there be any other reasons to support ZFS?"
Duh... It's called compatibility.
ZFS = snapshoots = Time Machine5 /4995/
http://arstechnica.com/staff/fatbits.ars/2006/8/1
I will be soon converting my Linux server to Solaris just for ZFS. Although ZFS may not terribly useful on a normal desktop, on a server, it's very powerful.... The idea of parity data actually being used actively to ensure data isn't corrupted is brilliant imho. So is the idea of on-the-fly recovery (I remember a video of some guy writing 30 megs of junk to a partition using dd, ZFS detecting it, and repairing it). *ends rant since all this can be read up about online*
In undeveloped countries, the consumer controls the market. In capitalist America, the market controls you.
...the words "Time Machine" are jumping up and down in front of my face trying to attract my attention. I can't think why that might be.
It's not you: I'm just this horrifically socially awkward with everybody.
Never ask for directions from a two-headed tourist! -Big Bird
A clicky to the Wiki article on ZFS.
- They had implemented everything I thought they should, and
- That only accounted for about 40% of the features of ZFS.
Calling it the last word in filesystems might be hyperbole, but I expect ZFS to last a good 10-20 years, which is quite respectable for a filesystem, and I wouldn't be surprised if it lasted longer. Is it a replacement for HFS+? Not yet.HFS+ is a very nice filesystem for single user systems with a single disk. It implements journalling, has reasonable performance, and has good metadata support. For the average users at the moment, the only real advantage of ZFS would be snapshots, and these are not too difficult to implement for other filesystems.
ZFS, however, is much better when you have multiple physical disks. At the moment, only the top-end Macs have more than one disk. This is likely to change in two ways:
- Cheap flash,
- Network storage
For a home user, ZFS could handle backups trivially by plugging in a large flash drive and adding it to the pool. I suspect this will be one mechanism Time Machine will use. Due to the way ZFS works, you can just mirror a part of the directory tree (e.g.ZFS is not needed as a replacement for HFS+ in 2007, but it probably will be in 2008-9. ZFS is a 128-bit filesystem, which means it is designed to last for a long time. We will probably never need a 128-bit filesystem (unless we actually want to build hard drives the size of planets with single-atom sectors), but we will need a 65-bit filesystem once we get to around 10 Exabytes. This won't happen with single drives for a while, but it will with RAID arrays.
I am TheRaven on Soylent News
The tech behind ZFS at least sounds very impressive, but I have to wonder how useful it is for workstation drives.
I've never found plain-Jane posix permissions to be all that useful on anything other than the most basic of server environments.
HFS has going for it all the fun stuff we've come to love apple for, such as transparent file customization like icons, labels, meta data, and whatnot through resource forks. I assume that these can be made to work with ZFS by making hidden files.
What I'd really like to see is both that kind of functionality along with NTFS's really excellent ACL permission system implemented. ACL permissions are a godsend for people responsible for running a file store that's used by humans as opposed to automated processes. NTFS also has a great deal of capacity for meta-data, although not to the same level as HFS.
NTFS is one of the few worthwhile things that's ever come out of Redmond. I wish more people would spend a bit learning from it without throwing it away simply because it's MS bloat.
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
So it's about the snapshot ability of ZFS, and that's exactly what will be needed for Time Machine.
"I use a Mac because I'm just better than you are."
With ZFS we might be able to get some very powerful backup features into OSX. Most binaries files don't change most of their content, ZFS makes it possible to due meaningful differential backups on large binary files. So for example 200 versions of a word doc with sounds and pictures that got revised over 6 months get stored in maybe 3x the space of the last revision. Emails with the same attachments get stored in just a few k rather than taking a meg each.... If Apple has this all working together by 10.5 then TimeMachine will work far far better then people currently expect it to. A 50g drive will be backing up a terabyte of worth of files.
The answer is that it probably is meant to replace HFS+, but since ZFS is not bootable yet (including for Solaris 10) Apple can take the time to introduce ZFS, build tools for easier management, and let people get familiar with the FS before they have to drop HFS+.
HFS' lifetime has already stretched far beyond what it should have, it's time for Apple to think of its next generation FS, and ZFS is an extremely promising FS with heaps of amazing features Apple has already started to integrate into its UIs with Leopard (Time Machine + ZFS Snapshots anyone?).
ZFS also shows strong promises as both a home and a server FS.
"The way we can tell it's C# instead of Haskell is because it's nine lines instead of two." -- wadler
Actually, you CAN use ZFS for everything except boot...so all you need is a tiny little grub boot partition and your golden. This is a tried and true method of booting NIXen with other filesystem formats.
-- Given enough time and money, Microsoft will eventualy invent UNIX.
They already have UFS and don't make it really usable, even after making a big deal about it being updated to the latest version from FreeBSD in Panther. It's a shame, too, because while HFS+ has a lot of nifty features all of them could be emulated over UFS or ZFS or any other file system (by putting the hooks for applications like Spotlight in the vnode layer rather than the file system - the vnode layer already has most of the hooks Spotlight needs), it falls far behind UFS in terms of reliability.
In fact HFS+ is *so* bad that if it wasn't for a couple of apps that absolutely freak out if they don't have their pet un-emulated feature I would have gone to UFS long since... even if I lost Spotlight completely. Until my Mac I had never run into a file system that wasn't so badly damaged as to be unbootable that coudn't be repaired by fsck... but apparently with HFS+ just running it "too full" can trash it, and I lost my system disk on my old G4 three months running because of that!
So I wouldn't hold out any expectations of ZFS being implemented in any useful way. They already have better file systems than HFS+ and they're not using them.
Zed's dead, baby.
There are no tiger attacks in my area and it's all because this rock I'm holding keeps the tigers away.
I've been wandering about this and am insanely curious: if ZFS really does intelligently copy on write how far can it take it?
e tc.) then this could probably happen quite easily.
for starters, does the FS "know" that i've just clicked "Save As" in my word processor? what about copy and pasting a file back into the same directory to make a local copy? Also? is it just within variations on the same file? if i have a particular setup exe on my system but forget, and download it again to the desktop surely the FS has no initial way of knowing that they are one and the same, does some funky heuristic happen?
basically: does the OS's read/write/copy/delete functionality have to invoke copy-on-write via a FS API or is it built in for every single sector-sized chunk that gets stuffed into the FS?
the next question is the one in my subject: how therefore do you define "capacity"? if i've got a bunch of files that take up 700mb on a ZFS device and try to back up to a (Joliet) CD will i get a message telling me that the CD doesnt have room? i can imagine this scenario being unlikely with optimised binary data (jpegs and mpegs) but if i'm backing up a dev environment with autobackups (main.c,main.c.bak.001,main.c.bak.002,etc.) and manually created and dated directory tree "snapshots" (dev,dev_backup_2006-12-18,dev_backup_2006-12-01,
If you don't risk failure you don't risk success.
I can imagine that using zfs send/receive to export/import pools would be an extremely efficient/safe method of replicating data. Perhaps some sort of ".mac mirror" could work. This would make Time Machine exceptionally useful, and I'd definitely commit extra $ for .mac services (if reasonable) for this.
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
Whatabout sun cluster 3? I particular, for the global filesystem?
Last I looked, they said, "NO! NO ZFS! IF YOU TRY, THE MOONS OF URANUS WILL CRASH INTO PLUTO!"
Do daemons dream of electric sleep()?
It sounds good, but I think migrating for it just a tad extreme given that it will be implemented for Linux pretty quickly. We're talking about neat new features here, it'll re-enforce or make easier backups and redundancy, but it's not a to-die-for solution that will solve all your problems. There's no way I'd drop a fully configured server installation which does what I need for a new filesystem.
By the way it's nice to see dtrace, open source Java, and now ZFS coming out of Sun recently. I almost feel sorry for how little they get out of a lot of their innovations, they remind me of Bell Labs just before they died.
// MD_Update(&m,buf,j);
I agree that ext3 isn't the best thing out there for Linux, and I don't even use it myself. However, I would suggest that the reason for so many people still using ext3 is that most other filesystems aren't better enough to encourage people to move away from ext3. Look at some of the posts under this story - you will find stories of people moving from Linux to Solaris, really just because they want the features of ZFS. There is demand for ZFS in the Linux kernel, and if it becomes a common filesystem on OSX, I predict the demand will only increase. I don't expect ext4 will satisfy this demand, either.
Wouldn't that pose a problem for mmap?
I think it would pose a problem for secure deletes. Try to obliterate a file by overwriting it with garbage, you end up writing somewhere else instead? Would the next overwrite attempt get the original location or would you have to write enough garbage to cycle over all the free space of the volume? Considering how large these volumes can get, that's a lot of boiled oceans for a multi-pass secure delete.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Is this what you want?
4 870
http://www.apple.com/xserve/management.html
Or this
http://docs.info.apple.com/article.html?artnum=30
There are two types of people in the world: Those who crave closure
I doubt Apple is going to require ZFS for Time Machine anytime soon. I fully expect HFS+ to continue to be the default file system for Mac OS X for a while (years). I believe ZFS support in Mac OS X is solely for the high-end / server space of the customer spectrum.
I wish that was the case. I really don't want to set up my linux root partition on fuse.i d=95110224#Platforms)
"Porting ZFS to Linux is complicated by incompatibilities between CDDL, the license its source is released under, and GPL, the license which governs the Linux kernel. To work around this problem the Google Summer of Code program is sponsoring a port of ZFS to Linux's FUSE system[10] so the filesystem will run in userspace instead. However, running a file system outside the kernel on Linux has significant perfomance impact." (from http://en.wikipedia.org/w/index.php?title=ZFS&old
"That's no moon -- that's a file server!"
i'd hit it so hard, if you pulled me out you'd be the king of britain [bash.org]
Indeed, we may see Mac OS X Server supporting default ZFS before Mac OS X proper. It would make sense to deploy it first in a limited market with technical expert users as your target market.
The Rise and Fall of Online Community
Hard drives silently losing data is a problem solved by RAID.
_ data
That is profoundly wrong. Vanilla RAID will not discover and cannot automatically correct silent data loss. The reason is that RAID has no way of knowing which data is correct. For example, if two mirrored copies disagree on the contents of a block, the data is unrecoverable without manual intervention or external knowledge. Furthermore, in normal operation your RAID subsystem will simply read data from whichever drive is idle at the time the read request comes in; it does not ordinarily compare the two mirrors. The data will remain corrupted until the user notices a problem, at which point they have no practical recourse. Essentially the same problem occurs with parity RAID.
There is no dedicated hardware in your system that provides the end to end data integrity that ZFS does. I honestly suggest you learn more about it before airing your opinions. Here is a start:
http://blogs.sun.com/bonwick/entry/zfs_end_to_end
If two mirrored copies disagree on the contents of a block, the data is unrecoverable without manual intervention or external knowledge.
Or, you know, a checksum. Or more than one level of redundancy.
I agree that RAID-1 cannot, by itself, correctly recover from error-free reads of mis-matched data. But RAID 5 and 6 are both capable of verifying the primary data source against the parity data and transparently correcting errors that occur on less than the critical number of disks. In the common configuration this is only done when a hardware-level error is detected to keep things fast, but it's quite possible to verify every read if so your system is so configured. Mutli-layer RAID also provides this same sort of protection.
[ZFS] will be implemented for Linux pretty quickly.
*sigh*. I wish. ZFS is being implemented on FUSE. This automatically creates limitations in performance and function (no root ZFS). IMO ZFS on FUSE will be a no starter in production.
I don't think we'll see ZFS in the kernel proper either, given the history of incorporating XFS and ReiserFS 4. Along the same lines, DTRACE will probably never make it in. It's being cloned in the form of Systemtap.
Meanwhile, FreeBSD has been porting ZFS and DTRACE. MacOSX is (partly) based on FreeBSD and DTRACE has shown up in MacOSX.
I agree that ZFS is a good reason to convert a file server to Solaris from Linux. FreeBSD may become a good candidate also. I'm a Solaris admin and haven't done much with FreeBSD so I'll lean that way. I'd love to see ZFS in the Linux kernel, but I'm not waiting for it.
Perhaps the way to go is Solaris x86 with ZFS file server then a BrandZ zone running Linux to provide other functions?
But RAID 5 and 6 are both capable of verifying the primary data source against the parity data and transparently correcting errors that occur on less than the critical number of disks.
:)
With RAID-5, as with RAID-1, the critical number of disks is one. RAID-5 cannot transparently correct errors that occur on even a single stripe, unless you know a priori which stripe is affected.
With RAID-6 you can automatically correct errors that occur on a single stripe, but it still does not automatically detect such errors on read the way ZFS and RAID-Z do.
Or, you know, a checksum.
That's a great idea. Too bad those ZFS guys didn't think of it. Oh, wait.
Over past months, I've read a lot of people commenting on ZFS who have no idea what it is. What it is, is the next generation of filesystems, not a "tweak" of current fs technology. It just happens to "look like" an ordinary POSIX fs, from a distance (if you ignore the administration/pool stuff...) But inside, it's something new under the Sun, folks.
RAID experts don't grok it, because it does things RAID can't do (end-to-end).
Devotees of ext2fs, reiserfs (yay!), NTFS (LOL!), or HFS+ don't grok it, because none of those filesystems do what ZFS does.
Read about it before you write it off as old wine in a new bottle. To ask the question, "Does OS X need a new filesystem?" is a perfect example of missing the point. Once you've looked at what ZFS really brings to the table, you'll see why it's an inevitable future, sooner or later, and you'll stop looking foolish.
Some links I posted this week:
- http://www.osnews.com/story.php/16739/Screenshot-Z FS-in-Leopard
- http://mac4ever.com/news/27485/zettabyte_sur_leopa rd/
(older rumour http://www.osnews.com/story.php?news_id=14473)
For OS X people wondering why the fuss about ZFS - summaries include: - http://www.sun.com/2004-0914/feature/ - http://www.sun.com/bigadmin/features/articles/zfs_ part1.scalable.html
"Why ZFS for home": - http://uadmin.blogspot.com/2006/05/why-zfs-for-hom e.html
"Here are ten reasons why you'll want to reformat all of your systems and use ZFS.": http://www.tech-recipes.com/rx/1446/zfs_ten_reason s_to_reformat_your_...
And some more technical explanations from Chief Engineer: - http://blogs.sun.com/bonwick/entry/zfs_end_to_end_ data
- http://blogs.sun.com/bonwick/entry/smokin_mirrors
you had me at #!
Have you read the link I posted? It is getting old to keep repeating Jeff Bonwick's arguments when you can just read what I already pointed you to.
The underlying hardware will not necessarily notice errors. Hard drives are only designed to error protect the magnetic domains on the disk. There are all sorts of other places in the increasingly long datapath to disk where data can be lost, and, in fact, routinely does get lost.
The choice to verify every read is purely an implementation decision
RAID-6 does not verify every read because it is a stupid way to achieve data integrity. It wastes two thirds of your aggregate IO read bandwidth when you can just use checksums virtually for free. CPU cycles for checksumming is dirt cheap whereas IO bandwidth is extremely expensive.
I was just arguing the novelity you seem to think ZFS has -- using checksums to verify data integrity is not exactly cutting-edge computer science.
Yet strangely there aren't any other widely available storage solutions that provide transparent data integrity from the filesystem down.
I just don't think this particular feature is unique or superior to other available solutions for the same problem.
Then name another one. I think we've already shown that vanilla RAID does not qualify.
....raid-{1,5,6,10}.... + reiserfs == survive crash at anytime including a drive physically dying.It won't survive a marriage to Hans Reiser!
Do you even lift?
These aren't the 'roids you're looking for.
Don't any of you guys use Macs?
.Mac? ZFS enables all kinds of coolness and I, for one, can't wait to get it on my laptop.
Here are five good reasons for Apple to go to ZFS:
-No more Disk Warrior. The entire data store is self-validating. No bit rot.
-No RAID controllers needed: ZFS gives you fast RAID for free. Just add drives. Why would anyone care? See #5.
-No more volumes and, therefore, no more volume management. ZFS eliminates the whole volume concept. Add a disk to your system and it joins your storage pool. More capacity. Not more management. What home user would want that?
-Continuous Data Protection out of the box. Time Machine could give you a view of your data every time you update a file.
-ITV, or whatever it is going to be called. Multi-GB files that each cost $10-20, that can't be backed up - thanks DRM! - and therefore need a cheap and highly reliable RAID. ITV, two firewire drives, ZFS and you are in business.
-Not to mention the existential pleasure of having great technology that Vista doesn't have. In fact, since consumer technology is driving the enterprise, expect ZFS on Mac to raise the bar for every OS and file system.
I suspect that Time Machine is simply the first of several beautifully designed storage utilities that we'll see on Leopard. How about automatic synchronization when you plug in an external drive? Snapshots automatically exported to
Read more at ZFS On Leopard: How Cool Is That? Means, Motive & Opportunity: Apple Kills the Media Center PC and the latest ZFS On Mac: Now All-But-Official.
And you heard about the native iSCSI support in Leopard, right?