The Hairy State of Linux Filesystems
RazvanM writes "Do the OSes really shrink? Perhaps the user space (MySQL, CUPS) is getting slimmer, but how about the internals? Using as a metric the number of external calls between the filesystem modules and the rest of the Linux kernel I argue that this is not the case. The evidence is a graph that shows the evolution of 15 filesystems from 2.6.11 to 2.6.28 along with the current state (2.6.28) for 24 filesystems. Some filesystems that stand out are: nfs for leading in both number of calls and speed of growth; ext4 and fuse for their above-average speed of growth and 9p for its roller coaster path."
In the case of NFS for instance, hasn't there been a performance improvement? Isn't that the thing that matters?
The briefness of the article and lack of actual functional analysis make me think this should have been a comment on the original /. article rather than a whole new article of its own.
Slow news day?
got to make one call...
Everybody Wang-Chung tonight!
While OSes may be "sliming down" as the article says, what does the removal of standard db packages from Ubuntu have to do with filesystem-related kernel calls?
The article doesn't seem to mention the possiblity that more functionality may be pushed into the kernel from userspace, which might make sense in other situations, but I don't think that argument would hold up here.
I am struggling to make the connection between the summary and the so-called article. The fact that they are not stripping/locking fs functionality means that OSes aren't shrinking? That's the hypothesis?
You are kidding arent you ?
Are you saying that this linux can run on a computer without windows underneath it, at all ? As in, without a boot disk, without any drivers, and without any services ?
That sounds preposterous to me.
If it were true (and I doubt it), then companies would be selling computers without a windows. This clearly is not happening, so there must be some error in your calculations. I hope you realise that windows is more than just Office ? Its a whole system that runs the computer from start to finish, and that is a very difficult thing to acheive. A lot of people dont realise this.
Microsoft just spent $9 billion and many years to create Vista, so it does not sound reasonable that some new alternative could just snap into existence overnight like that. It would take billions of dollars and a massive effort to achieve. IBM tried, and spent a huge amount of money developing OS/2 but could never keep up with Windows. Apple tried to create their own system for years, but finally gave up recently and moved to Intel and Microsoft.
Its just not possible that a freeware like the Linux could be extended to the point where it runs the entire computer fron start to finish, without using some of the more critical parts of windows. Not possible.
I think you need to re-examine your assumptions.
Thoughts:
- This is measuring, I believe, calls to different functions; a call to one function from multiple places is only counted once. So it's really a measure of the diversity of external calls.
- Size and complexity aren't necessarily the same thing. It's actually possible that as common functionality is abstracted out of filesystems, they get smaller but make more external calls. There was a point a few years ago when this was happening at quite a rapid pace in the fs code, I don't know if it is still true.
- Journalled filesystems and networked filesystems are pretty complex creatures by their nature, the quoted numbers don't seem unreasonable. NFS in particular implements (IIRC) protocol versions 2, 3 and 4, and 4 had a lot of new stuff.
I've never seen a slim neckbeard
There is little calling overhead from using multiple calls. Of course these interface changes are all done for a good reason: performance, stability, security.
Engineering is the art of compromise.
What's your argument here? That filesystem code in the kernel shouldn't be growing more sophisticated over time?
This rings of the armchair-pundit argument that the kernel is getting more and more "bloated" and a breath later crying out that there still aren't Linux hardware drivers for every computing device ever made.
The state of Linux filesystems may be in disarray, but it's nothing to kill your wife over...
*rimshot*
Not being a filesystem/db geek, I honestly can't tell if "speed of growth" refers to "how frequently it's updated" or to "how rapidly it allocates space to store things". And I don't understand what the number of external calls means at ALL. Is that a bad thing? A good thing? Why? Can someone please provide some context? This doesn't have any at all!
as so often with linux stuff can be configured, when the kernel is compiled you can disable lots of stuff the filesystems use (acl, security hooks, etc etc). Bet with that stuff disabled the chart looks way better
Microsoft has been known to use FAT file systems, FAT16, VFAT, FAT32, with NTFS I was able to allow my Windows to go on a diet. :)
No really we don't need extra features for an operating system, make them optional and don't load them up at default. Maybe make most things a kernel module that only loads when it is used.
Bloating the Operating System with uneeded features is just plain stupid. Windows 2000/XP is most anyone needs for Windows, everything else is just Fat on the operating system and slows it down with bloated features most people don't need.
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
While Tux3 is not yet ready to run on your desktop, and won't be for a good many months, it is relatively trim at around 6K lines, and is expected to be somewhere around 10K complete with versioning, recovery and proper code comments. Of course, that will still be significant growth in a few months, and nothing says it won't just keep growing. But Tux3 is starting much smaller than its peers, and already has a pretty good range of "big filesystem" features. One of our guiding principles is to keep it tight, therefore leaving fewer places for bugs to hide.
Have you got your LWN subscription yet?
What you described sounds like upgrading to an alpha version of Ubuntu, with 3rd party packages.
I have not seen conflicts like that when using official distros (of any type) though.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
I'd even rather hear one more variation on our insensitive clod overlords from Soviet Russia.
In Soviet Russia, insensitive overlords run Linux without windows on YOU!!
The author of the article takes the position that filesystem external calls == "operating system size", and then proceeds to start measuring his new definition.
What he never mentioned or even tries to justify was his metric. Why does more external calls (or as someone more accurately pointed out, diversity of external calls) equate to "operating system size"? Why does this equate to an even more abstract concept of "simple"?
I don't see any reason to equate these measurements to such conclusions. "size" and "simple" are abstract concepts that involve a lot more than a simple count of external references.
AccountKiller
Or some such arbitrary statistic?
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
The trouble is packages are advertised as Linux packages. Now, Joe Public knows he has Linux in Ubuntu (and he's right), so he goes ahead to install the package. Why? Because Ubuntu does not have the package he's trying to install. He then runs into all sorts of trouble. Right?
never happened to me.
BTW, what's the point ? and how is it related to the growth of operating systems ?
I didn't see anything in the article where the author made a value statement, that it is bad (or good) that system calls are increasing. He was just pointing out that the trend is not towards simplicity in this area.
I would also point out, that ext4 is very new and ntfs may not be new but never has been quite completed so active feature development could explain away the upward curves in their call counts, though not the absolute values.
I think that you can compile only the filesystem you want in the kernel..
So the only complexity which matter to an user is the one of the filesystem they select to compile in the kernel!
Sorry to disrupt the trolling copy-pasta, but :
Is this post on ZDNet's forum the original form of this troll ?
Or is this troll older, and jerryleecooper was already copying it from somewhere else ?
I'm just curious to know where this fine piece of humorous trolling was originally born.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Perhaps you can provide a real world example of this since the rest of us
seem to be baffled by the idea, having never actually experienced it ourselves.
A Pirate and a Puritan look the same on a balance sheet.
Imma firin' mah Reiser!
LOLOLOLOL good one, had me going for a minute
We need that Reiser fella on this pronto.
So what if he killed his wife, he still makes a damn fine filesystem. Reiserfs 3 was a good bit of kit.
I'd imagine he has time on his hands at the moment, does anybody know if he's expressed an interest in/allowed to continue development of Reiserfs 4?
You feel sleepy. Close your eyes. The opinions stated above are yours. You cannot imagine why you ever felt otherwise.
""Size" and "simple" is not so easily measured."
Says the retard with the tiny dick.
Unless I've misread it, TFA's definition of "size" for a filesystem is "how many distinct external/kernel subroutines does it call?"
That seems to be a very strange metric. Seems to me that a well-designed filesystem will have little code and make extensive (re)use of well-defined services elsewhere (rather than reinventing those wheels). This "slims" the filesystem's own unique code down to its own core functionality.
Now maybe, if the functions are hooks in the OS for the filesystem, a larger number of them implies a poorer abstraction boundary. But it's not clear to me, absent more explanation, that this is what TFA is claiming.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
How does the Linux file systems compare to Mac and Windows? I mean it might help get some perspective if we knew what the numbers for the 3 major OSes are. How about Mac and Win server? While these numbers might mean a lot to hardcore Linux gurus to a jack of all trades guy like me it doesn't really tell me a lot without a couple of the competitors to compare it to. But maybe it is just me.
And it might also be just me but the formatting on the chart made it a little hard for me to read. pretty much all gray except for the three that he wanted to highlight in red. I mean for all i know there could be tests for Mac and Windows in there but his text was so tiny on my 1024x768 laptop screen that I frankly couldn't tell exactly what line was for which.
ACs don't waste your time replying, your posts are never seen by me.
Because when all those millions of Joe Public's upgraded from XP to Vista or Win7, everything worked swimmingly out of the box, right?
In my life I have installed the following Linux "Packages" that were not part of the distro, but I wanted (the great thing with Linux is that it is rare, and it becomes more-so with time).
The results were:
Simcity 2000(3000maybe?) it needed a download from the companies website to install, it was old and my CD was not compatible with new Linux.
Myth2,same
Nvidia drivers, no effort at all, just run the installer, this was across many distros.
Return to Castle Wolfenstein: Enemy Territory, Run the install, no problem. It ran into issues with newer distros and pulseaudio though.
Enemy Territory: Quake Wars, No trouble that I recall.
As I said, the only issue I have ever had with circular dependencies is when using a distro in alpha stage, and not in anything official. In those cases I usually only need to wait a couple of days for it to be resolved. Of course I can't imagine I could ever get SimCity to work today.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
climbing higher than the Moore's law, that's fine!
I don't mean the developer could write ugly code because we have faster CPU. I mean, if implementation of more functionality is necessary, more code and call is needed and that's fine to me.
I am happy to trade my CPU time for harddrive access, which the access speed didn't follow the Moore's law.
This is where you *nix people do yourselves in for winning over the rest of the tech population, one word "homogeneity". As a "Windows Boy" I don't have to worry about bollocks like this. I have enough trouble working with the variances involved with Windows machines (their hardware and software) when I deploy an application... nevermind having to also worry about miriad filesystems and kernels compiled a L33T HAXOR. ** END RANT ** And for anyone about to say Apple/Mac... I'll stop myself right there.
I have no idea of the real code behind those file systems but I find "external calls" a bit of an weird metric. The more "common code" used by the file systems, the more "external calls" would be seen, while this would actually be a Good Thing.
24 Filesystems!? I think we've just metricised a clusterfsck
If you don't risk failure you don't risk success.
Why would you expect the kernel to shrink?
The Linux kernel is a gigantic C program, with lots of coupling, global variables, no module system, and lots of external dependencies. Those kinds of systems are exceedingly hard to shrink. And C encourages people to re-implement data structures and software components multiple times (hash tables, linked lists, etc.) instead of providing mechanisms for generic algorithms and reuse; that contributes further to the bloat.
In addition, the Linux kernel is adding a lot of functionality and generalizing functionality. For example, instead of simple hard-coded partitions, you now have several layers of abstractions and indirection allowing you to deal with partitions much more flexibly.
I have my own opinion as to whether the Linux kernel is well-engineered and whether I'd want to hire Linus. But as long as bloat and software engineering in the Linux kernel doesn't get in the way, I don't really care what it's written in or how it's written.
As long as it gets the job done, popularity and support are more important than elegance and code quality in a kernel.
Why are fancy charts and graphs needed to prove that the Linux kernel is not "sliming down"?
sliming being coated with a soft, moist, adhesive mucilaginous substance down in the direction of gravityI mean, it's obvious, isn't it? Just look at the thing. No slime at all that I can see.
The Web is like Usenet, but
the elephants are untrained.
ZOMG... did Teh Steve Bulmeer pay deez guyz off to beat up Teh Rieser?!?!?
Yoo jes know Teh Bulmeer wuz gunna trow a char at him, but teh prizun gots in teh way. So he hire teh thuggz to beats up Teh Rieser, cuz he herd he was, liek, workin on Teh Lunix an stuff.
Dunt Wurry Teh Rieser! We FOSSies still luvz yooz, yoouz is sekund unly to Teh Lunis hisself. Weez noes Teh Balmer set yoo up!!
Free Han Rieser!!!
Are Han Longs to be Free!!! Free liek in beers!!!!
The inline function specifier is a preference for the compiler. There is no guarantee. I'd suggest you read section 7.1.2 of the ISO/IEC 14882-2003 document detailing the C++ standard. kthxbye.
5 minutes vs "seconds" just because of a function pointer? I hope you're kidding.
Here is what you can do. Take your favorite C++ compiler generate the assembly listing for both versions of the source code. Download the Intel manual for whichever CPU you ran it on "many years" ago. Look up the instruction timings for the opcodes and you'll get a good ballpark estimate. Its easy to prove to your own mind how stupid your claim was.
But if you were high when you made that statement, My Apologies. I'd suggest you go to hospital immediately and check yourself in for some drug detoxification sessions.