Slashdot Mirror
ZFS, the Last Word in File Systems?
guigouz writes "Sun is carrying a feature story about its new ZFS File System - ZFS, the dynamic new file system in Sun's Solaris 10 Operating System (Solaris OS), will make you forget everything you thought you knew about file systems. ZFS will be available on all Solaris 10 OS-supported platforms, and all existing applications will run with it. Moreover, ZFS complements Sun's storage management portfolio, including the Sun StorEdge QFS software, which is ideal for sharing business data."
1) Even Sun has succumbed to recursive acronyms, now.
2) Is it just me, or is the post surprisingly bereft of unique details? I mean, integration with all existing applications is rather assumed, given that it's a file system and all...
It's only an insult if it's not true.
Logically, the next question is if ZFS' 128 bits is enough. According to Bonwick, it has to be. "Populating 128-bit file systems would exceed the quantum limits of earth-based storage. You couldn't fill a 128-bit storage pool without boiling the oceans."
So, what was the point of creating a 128-bit filesystem?
-1, Marketing Hype.
*Yawn*
Billion billion is a perfectly valid number. Or would you rather they say 6.0 × 10^18? Most people can't imagine that. But people can (kind of) visualize a billion, and then multiply that by a billion, and see it's really, really big.
I'm in the hole of the broadband donut.
and
Compared to AIX or HP-UX, 28 steps is shockingly bad, both have had much simpler logical volume management for several versions now (AIX for 5 years or more? certainly as long as I have used it). The existing Solaris 9 logical volume infrastructure is years behind the competition, this is bringing it up to date, but not putting it far ahead.
Ewan
Actually what's the big deal of supporting such massive amounts of data?0
.. how are you going to ORGANIZE a 128 bit file system? Oh I see folders? Umm so if you're going to use folders .. umm why not have multiple drives or partitioning?
.. well what about it? Look if each data word size is so massive that the only way to address it is with 128 bits .. how the hell do you process such a huge amount of data in one pass anyway? Show me a CPU (not parallel system) .. that do operations on billions of trillions of gigabytes of data simultaneously.
.. Gillete decided to have 3 blades ..and so Schick put 4 and claimed to be superior .. Now why not add 5 .. what about 6?
.. not that they made the breakthrough of ... "hmm we used 16 bits .. 32 bits .. 64 bits .. hmm why not 128 bits!" When they have a system capable of actually processing such data ..I'll be the first to cheer.
OK I am saying it now and it wont be back to curse me.
64 bits should be enough for everybody.
Now, here's the deal
Ah yes I can hear people saying "what about large file data sets?"
Reminds me of the Gillete Mach 3 versus Schick Quattro lawsuit
This 128 bit file system only serves marketing purposes. I want to see more clear advantages
Who else instantly thought of, "640 K ought to be enough for anybody", uttered by the chief architect of twenty years of chaos?
I suspect that whatever open source license Sun release Solaris under, they'll be careful to make sure it's incompatible with the GPL.
I don't know about the "last" word in file systems, but they won't be anything but klugey simulations of antiquated paper cabinets until their first word is "SELECT". Will someone finally replace the hierarchical inode database with relational tables, and a SQL API? Throw in a traditional file/directory API mapped to SQL statements, and the world will beat a path/filespec to your door.
--
make install -not war
Such a feature would rock, because it would be possible to make things like installers completely atomic: interrupt the installer process and the whole thing rolls back.
Use 'slashdot stuff' in the subject line in any email you send me if you want to get past the spam filter.
You don't do much video editing, do you? ;)
Looks to me like nothing more than an excuse to put up a patent tollboth for anyone who wants to implement ZFS.
As the world's first 128-bit file system, ZFS offers 16 billion billion times the capacity of 32- or 64-bit systems.
A 64-bit (unsigned) binary number can already store values up to 16 billion billion (actually, closer to 18, but who's counting). That's roughly 2.5 billion individually addressable locations for every man, woman, and child living on Earth.
Shouldn't that be enough to hold us for a few generations at least?
Never need more than a 128 bit filesystem? My arse... and I'll never need more than 640k of system memory. Just because 128 bit filesystems allow an utter crapload of data doesn't negate the fact that 256 bit filesystems would allow a super utter crapload of data...
You organize a 128bit file system with a database.
Why bother with folders as a root? You can create a folder hierarchy *with* a database too.
GPL Deconstructed
64 bits should be enough for everybody.
Well 128 Bit is more of an issue of coming up with something without a limit or a limit that anyone any time soon will use up. The difference between 64bit and 128 bit is the diffence of a number that we can handle and comprehend to a number that is much to big for our minds to properly comprehend.
How can someone fill a 64bit file system, Well a large company or government organization that stores all their persons files onto one file system. Or say a program that gives its logs in seporate files. Or say storing uncompiled movies frame by frame. Or having an archive of data spanning hundreds of years. Yes there are ways around it now. But sometimes have a file system that doesn't have those limits. Comes in handy, nor nessarly for not but to expend into the future.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
People can visualize it. Billion is much more common than you think. 1 billion in hertz is 1 GHz. 1 Billion bytes in RAM is 1 Gigabyte.
It would take over 500 years to fill a 64 bit filesystem written at 1GB/sec (and of course 500 years to read it back again).
t ml
One product already can transfer a Terrabyte per second, so that would cut the transfer down to half a year. And I imagine that transfer rate would continue to increase.
I don't see how one would necessarily argue against such a thing for products that will go for cluster and supercomputer use. I say might as well get the bugs out so when you can so that once the 65th bit is needed, the supercomputer suppliers are ready.
http://www.sc-conference.org/sc2004/storcloud.h
No, precisely because we can't do it now, and for the very predictable future, we shouldn't be wasting all that disk space, access and CPU time for a boundary that no production system is likely to ever reach before they get upgraded. That's just practicality.
Seagate apparently sold 18.3 million desktop drives last year. Assuming they're all about 120GB (which is generous of me), that would be about 17.6*10^18 bits. Guess what, that's 2^64 bits. Yes, you would have to buy every single desktop hard drive Seagate shipped in the last year to have the capacity to fill a 64 bit filesystem. And find space for 18 million drives. And a power station to deliver the several hundred megawatts you'd need.
Even at 2 times drive capacity growth per year that's still a ridiculously unattainable figure. In 14 years time you'd only need to buy 1000 drives (which are now 2000TB each). But 14 years is a geological time scale when it comes to computers. You'd have wasted 14 years of CPU time and disk space devoted to those extra 64 bits.
If you still think 64 bits isn't enough, how about 96 bits? It would take 46 years before hard disks were big and cheap enough so you could fill the filesystem by buying 1000 of them. But no, they chose 128 bits because it sounded good.
To those who don't know, no amount of explanation can make a joke funny. In fact, if you have to explain the joke, it's pretty much guaranteed not to be funny. I found it kind of amusing - I didn't know that [ was the next character but I was able to guess that it was simply by what was said. Consequently, I found it amusing. The response from someone who doesn't think about that stuff is going to be similar to "Ah. That's funny." Followed by a shaking of the head as they walk off toward the water cooler to tell everyone what an insufferable nerd you are.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
the funny thing about reading the article is that you get the details. you should try it sometime.
here are some more details, but nowhere near as long a list as you'll get from reading the article (since the full list would mean quoting the article, which i suggest reading).
- data checksums eliminate the need for fsck
- easy to add disks to the pool
- seems to support raid 0 and at least one real raid
- data rollbacks (sound like netapp snapshots)
- can mount the same filesystem on sparc or x86
while not necessarily amazing, when you start adding all of it together it makes for a large improvement over ufs or vxvm. it's interesting, to say the least. i consider this a big announcement for the solaris platform (and, as more than one person pointed out, possibly linux and bsd since the code for it will ultimately be open source).
as far as greater technical details, how are people even going to know it exists in order to, say, make independent performance benchmarks if there's no announcement. should everyone just discover the feature accidentally?
And how many have a clue of how much that is?
Beware: In C++, your friends can see your privates!
IMO a necessary feature of a modern file system is that it doesn't need to be defragged.
This sig under construction. Please check back later.
1) Adding more address space bits doesn't significantly slow down performance.
2) Migrating from one address space to another is painful. Why make it more frequent by aiming low? Do you think migration would be any less painful in 14 years?
3) New applications: Broadband didn't just result in really fast web-page downloads - the entire online music industry stems from that. The original creators of TCP/IP had no idea that they were developing media on-demand, they were making it so that you could transfer bits from one archaic machine to another.
Building flexible, capable systems creates an environment where development isn't as constrained by limitations - resulting in new, unpredictable developments.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
There's a big difference between visualizing the space containing a billion elements and visualizing the elements themselves. Try imagining all the little plastic millimeter chips that fill that half mile.
Then, since it's actually a billion billion at stake, try to imagine that half by half mile square full of tiny plastic chips.
Finally, put them in an oversized bathtub, surround the tub with video games, a bad pizza parlor and tired parents, and wham! You're Chuck E Cheese. Therefore, we can state firmly:
1) Visualize Billion Billion.
2) ??? [Which adequately describes setting up a chuck e cheese]
3) Profit.
In soviet slashdot, billion billion profits you.
Pardon me; I have to find a way to convince myself that my hot grits cluster joke isn't outdated.
StoneCypher is Full of BS
once you're going to expand past a 64-bit filesystem, there's not much point in going smaller than a 128-bit fileystem.
Why expand past a 64 bit filesystem. 64 bits with 1k blocks as your smallest addressable unit (which is more than reasonable for a filesystem this size) gives you 2^74 bytes to play with. For reference, that's 16 * 2^70 bytes = 16 * 2^30 terabytes, or "one hell of a lot of data".
You've pointed out just why we need this. The problem is, you're still thinking in terms of individual hard drives in individual computers that can only be accessed by the local machine.
What are you going to do when you access all of your data through a network, and the whole world has their storage on the internet, using a global filesystem? You said yourself that one manufacturer makes 2^64 bits of HD space every year, so 64-bit is obviously not enough. We need 128 bits if we want to be able to make use of all the HD space that is going to waste on networked computers today.
Hell, we could do that today, if we had - wait for it - the right filesystem.
The fact that it's Sun that came up with this suggests they're thinking along the same lines. They would benefit greatly if people started using a massively networked filesystem, especially if they own the code to it.
Actually, it's only 18.3 million desktop drives if you address every single byte of the filesystem. Most don't do this; they allocate space in blocks. 1k is a reasonable block size if you're talking many terabyte systems.
With a 1k block size, you'd be addressing 16 billion terabytes of storage. Let us know as soon as every single person on earth has more than 2 terabytes to donate to your distributed
filesystem project.
I've come for the woman, and your head.
"3) New applications: Broadband didn't just result in really fast web-page downloads - the entire online music industry stems from that. The original creators of TCP/IP had no idea that they were developing media on-demand, they were making it so that you could transfer bits from one archaic machine to another."
How could they predict iTunes? Why would you think it reasonable to predict the usage of such a filesystem?
I have no problem with your religion until you decide it's reason to deprive others of the truth.