Slashdot Mirror


A Visual Expedition Inside the Linux File Systems

RazvanM writes "This is an attempt to visualize the relationships among the Linux File Systems through the lens of the external symbols their kernel modules use. We took an initial look a few months back but this time the scope is much broader. This analysis was done on 1377 kernel modules from 2.6.0 to 2.6.29, but there is also a small dip into the BSD world. The most thorough analysis was done on Daniel Phillips's tree, which contains the latest two disk-based file systems for Linux: tux3 and btrfs. The main techniques used to establish relationships among file systems are hierarchical clustering and phylogenetic trees. Also presented are a set of rankings based on various properties related to the evolution of the external symbols from one release to another, and complete timelines of the kernel releases for Linux, FreeBSD, NetBSD, and OpenBSD. In all there are 78 figures and 10 animations."

85 comments

  1. Hmm by Anonymous Coward · · Score: 5, Funny

    I could use a visual expedition among the various links on this summary.

    1. Re:Hmm by arth1 · · Score: 1, Insightful

      No kidding. Sometimes I like to (gasp!) RTFA, but in this case I couldn't find out which of the F links went to the F article.

    2. Re:Hmm by suso · · Score: 1

      <AOL>Me too!</AOL>

    3. Re:Hmm by PitaBred · · Score: 2, Insightful

      You can't be from AOL... you used correct capitalization and the proper homonym of "too". I call shenanigans.

    4. Re:Hmm by suso · · Score: 1

      ur right. i'm from youtube show me how to make a linux filesystem

    5. Re:Hmm by Anonymous Coward · · Score: 0

      You're retarded, linux filesystems qre a scam, I bet you can't find a single person whos really seen one, dumbass!

    6. Re:Hmm by badc0ffee · · Score: 1

      RTFA? I had fun finding the pictures that looked like paper tape and hollerith punch cards... then I ran across the one that looked like a program board for an IBM 557 and gave up.

      --
      1011 1010 1101 1100 0000 1111 1111 1110 1110
  2. You left the best 'til last! by Kawahee · · Score: 1

    In all there are 78 figures and 10 animations.

    --
    I'll subscribe to Slashdot when I see a month without a dupe, a typo, or an article the "editors" didn't read.
  3. Let me correct that for you! by Azeroth48 · · Score: 4, Funny

    This analysis was done on 1337 kernel modules

    --
    This is where we are, our rock we stand, among the world, looking forward, eternally.
  4. BSD? by Rosco+P.+Coltrane · · Score: 2, Interesting

    So they finally managed to recover DNA evidence on BSD's corpse after all. Hopefully we'll find out who killed it.

    Seriously, what a useless "analysis". It's all a bunch of unreadable tables, graphs and other things, such as the history of the number of exported symbols in the BSD kernel (yeah right...) nobody cares about.

    --
    "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    1. Re:BSD? by Anonymous Coward · · Score: 0

      No kidding. The world was really waiting for the history of the number of exported symbols in the Linux kernel, right? Geez. If they ever legalize gay marriage where you live why don't you propose to Linus and get it over with. Until then, clean your room, your mom's pissed.

  5. Verry Pretty ...but by Chrisq · · Score: 4, Insightful

    What does it all mean?
    Is having a large number of external symbols good because it has more integration points or bad because of bloat? I don't think I have ever RTFA and come away with so little understanding before.

    1. Re:Verry Pretty ...but by RuBLed · · Score: 2, Insightful

      As the child in me would say.. Just look at the pictures.

    2. Re:Verry Pretty ...but by frodo+from+middle+ea · · Score: 1

      what does it all mean ?
      ZFS rules ?

      ducks.

      --
      for the last time people, I am "frodo from middle eaRTH", not "middle eaST".
    3. Re:Verry Pretty ...but by Tenebrousedge · · Score: 0

      ZFS rules ducks?

      It's cases like this, and that damn gun-toting panda that remind us of the need for proper use of punctuation. Though it's always puzzled me how people can write code all day long and then fail to use analogous typographical rules correctly in communication. Or maybe there's more "Hello ,Wrold!" code than I'm aware of.

      Oh well. I, for one, welcome our new anatidaen overlords!

      --
      Those who advocate genocide deserve every protection afforded by law, and none afforded by common human decency.
    4. Re:Verry Pretty ...but by Anonymous Coward · · Score: 1, Informative

      You can draw several conclusions from the images. One of them is how similar some filesystems are.

      Have a look to the hierarchical clustering and hamming distances graphs.

    5. Re:Verry Pretty ...but by smallfries · · Score: 1, Offtopic

      Is that an American style of punctuation? In British English the first (more logical) example is correct.

      --
      Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
    6. Re:Verry Pretty ...but by rant64 · · Score: 0, Offtopic

      If the entire sentence were a quote, then the second example would be correct, at least around here (NLD). But the "dd" is not a quote, it's a literal string that should not include the period.

      Both of them make sense.

    7. Re:Verry Pretty ...but by GaryOlson · · Score: 1

      This means ZFS will continue to suffer from an unknown formative malaise because it cannot be properly analyzed. Without the full analysis, a proper treatment cannot be attempted without further damaging the file system. Isolation is the best recourse at this time.

      --
      Every mans' island needs an ocean; choose your ocean carefully.
    8. Re:Verry Pretty ...but by Sinbios · · Score: 0, Offtopic

      Your incorrect conclusion that quotation marks are logically wrong arose from the fact that you don't understand how quotation marks work.

      --
      Anyone can "stand up for what they believe", but it takes a very brave individual to change what they believe. - Loundry
    9. Re:Verry Pretty ...but by Anonymous Coward · · Score: 0
      said the one with the wrong quotation stile in the signature:

      Anyone can "stand up for what they believe", but it takes a very brave individual to change what they believe. - Loundry

    10. Re:Verry Pretty ...but by Anonymous Coward · · Score: 0

      Then please educate us, there are more of us who do not understand the second usage of quotation marks (as it would be incorrect in many languages I've come across).

  6. Typo in summary. by SharpFang · · Score: 4, Funny

    This analysis was done on 1377 kernel modules from 2.6.0 to 2.6.29

    I think they meant 1337 kernel modules?

    --
    45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
    1. Re:Typo in summary. by M8e · · Score: 0, Offtopic

      Very funny, moding that Informative.

    2. Re:Typo in summary. by Anonymous Coward · · Score: 0, Offtopic

      Umm, I don't think this was meant to be informative. I think SharpFang was trying to be funny.

    3. Re:Typo in summary. by rant64 · · Score: 0, Offtopic

      Umm, I don't think this was meant to be informative. I think SharpFang was trying to be funny.

      You're trying to be informative, Wise Guy?

    4. Re:Typo in summary. by wezeldog · · Score: 1

      Devito: "You mean, let me understand this cause, ya know maybe it's me, I'm a little fucked up maybe, but I'm informative how, I mean informative like I'm a philosopher, I enlighten you? I make you think, I'm here to enlighten you? What do you mean informative, informative how? How am I informative?"

  7. So what? by Anonymous Coward · · Score: 0

    Ok pretty visualisations, but what do we learn from them?

    1. Re:So what? by Rosco+P.+Coltrane · · Score: 4, Insightful

      Nothing. The whole point was do create said visualizations. From the "expedition" homepage here:

      This is an exercise in visualization and kernel exploration. I'm not an expert in either of them but I like file systems and I also find great pleasure in creating visual representations of the things around me. --RazvanME

      He likes file systems and he likes to create visual representation of things. There's your explanation. I suspect the guy is a student with too much free time and a desire to be featured on Slashdot.

      --
      "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    2. Re:So what? by gbjbaanb · · Score: 4, Insightful

      Which is fair enough (unless you happen to have had a quick look at the summary, an even quicker look at the diagrams and thought "d'uh WTF?" to yourself). I think if he'd created a colourful fractal image, or moving dots swirling around then everyone would be saying how great it was. As it is, it looks like dull statistics.

      I found the interesting bits to be how closely tux3 kept coming up next to fat or ntfs, whilst btrfs was close to xfs, and ext4 with ext3 and ext2. Maybe there's something in the analysis after all!

    3. Re:So what? by machine321 · · Score: 1

      I found the release timeline graphs for OpenBSD fascinating. Visualization really gave insight into the meaning of the data.

    4. Re:So what? by networkconsultant · · Score: 0, Flamebait

      I wonder if he's ever had a date?

  8. Useless exercise by Anonymous Coward · · Score: 0

    I agree that it's a useless analysis, since if you're building a kernel you only build the parts that you're going to use, on your arch. Even a distributor doesn't necessarily compile "all" of the filesystems, for one example. A silly distro kernel uses initramfs and stuff so everything can be modules anyways. Not everyone loads them all. So whoopee doo if the whole mess is several megabytes.

    There's a lot of functionality available, and you can choose what you want. Representing it in a meaningless graph does not at all show alarming, or even interesting trends.

    1. Re:Useless exercise by hedwards · · Score: 1

      That's sort of what I want to know. For FreeBSD, as an example, there's no reason to compile more than a small fraction of those at a given time. I think most users could exist quite well without anything beyond just NTFS, UFS, NFS, ISO9660, ZFS and MSDOSFS. And you can throw out NTFS, NFS and MSDOSFS as often as not.

      I'd assume that Linux is similar except subbing one of the EXTFSes in for UFS.

  9. My chart by iamdrscience · · Score: 4, Funny

    Usefulness of a hammer:
    XXXXXXXXXXXXXXXXXXXXXXXXX

    Usefulness of an average Slashdot article:
    XXXXXX

    Usefulness of a screen door on a submarine:
    XXXX

    Usefulness of this Slashdot article:
    X

    1. Re:My chart by Anonymous Coward · · Score: 1, Funny

      Usefulness of a hammer:
      XXXXXXXXXXXXXXXXXXXXXXXXX

      Pass me that hammer, there's a fly on my mother-in-law's forehead

    2. Re:My chart by Yosho · · Score: 1

      Pass me that hammer, there's a fly on my mother-in-law's forehead

      Now that's what I call killing two birds with one stone!

      --
      Karma: Terrifying (mostly affected by atrocities you've committed)
    3. Re:My chart by Shikaku · · Score: 1

      .

      Filter error: You can type more than that for your comment.

    4. Re:My chart by Sulphur · · Score: 1

      When a hammer is wrong, then the result is not funny.

    5. Re:My chart by bertoelcon · · Score: 1

      I would have thought a screen door on a submarine was on par with an average article, or rather the average of all articles.

      --
      Anything can be found funny, from a certain point of view.
    6. Re:My chart by Hurricane78 · · Score: 1

      Talk about an open door with just a screen door left between you and 500 feet or polar ice water.

      A hammer. Bah. Crybaby. ;)

      --
      Any sufficiently advanced intelligence is indistinguishable from stupidity.
    7. Re:My chart by Hurricane78 · · Score: 1

      Reminds me of the best way to die, ever: Take a parachute drop, but without the parachute. Instead put a large thorn/needle/picket on your head. Then land exactly on your favorite bad guy.

      When I'm old and gray, this is what I am going to do. If it were today, I'd take the boss of Monsanto or some Rothschild boss with me.

      --
      Any sufficiently advanced intelligence is indistinguishable from stupidity.
    8. Re:My chart by shutdown+-p+now · · Score: 0, Troll

      You forgot

      CowboyNeal:
      (-INF)

    9. Re:My chart by Anonymous Coward · · Score: 0

      Where does pr0n fit in?

  10. Re:Warning by Chrisq · · Score: 2, Funny

    Slashdot user BadAnalogyGuy is a scientologist. If he offers you a free personality test, firmly refuse him.

    Thanks, but you don't have to worry. I have no personality.

  11. Eye-candy knowledge by koolfy · · Score: 4, Insightful

    Am I the only one not understanding a word nor the point of those graphics, but still going to print them as posters and put them everywhere in my bedroom ?

    'cuz dude, it's so beautiful !

    --
    Segmentation Fault in "Life, Universe and Everything" at line 42. Don't Panic.
  12. Comment removed by account_deleted · · Score: 2, Interesting

    Comment removed based on user account deletion

  13. External symbols for 2.6.29 + tux3 by bagsta · · Score: 1

    As far as I can tell, from this table the external symbol mcount is used by all file systems... :)

    And Obama is president of US...

    --
    Until the skies turn blue...
    Until the air of freedom strikes us...
  14. After perusing the article, and ... by OneSmartFellow · · Score: 2, Insightful

    ...slowly loosing the will to live as my eyes became tired and I confused by the lack of any conclusions which could possibly be drawn from such an overwhelming variety of apparently useless graphical analysis of kernel ABI calls made by mostly arcane file systems, I couldn't help wondering .....
    .
    .
    .
    .
    .
    .
    .
    .
    . Who gives a shit ?

    1. Re:After perusing the article, and ... by shish · · Score: 1

      Who gives a shit ?

      Nerds, which is what this website is for. If you're looking for some sort of pop culture website where things are only posted if they appeal to the majority, you're doing it wrong...

      --
      I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
  15. One other observation though... by OneSmartFellow · · Score: 1

    How is it possible to create a file system driver which does not call "register_filesystem" ?

    1. Re:One other observation though... by jeremyp · · Score: 3, Funny

      It's easy.

      I did it once and the fact that it couldn't be mounted was offset by the fact that no bugs were ever reported, nor did it it ever corrupt or lose any data. In fact, I didn't even have to write any of the implementation.

      --
      All I want is a secure system where it's easy to do anything I want. Is that too much to ask ~~ Randall Munroe
    2. Re:One other observation though... by jiriw · · Score: 2, Informative

      How is it possible to create a file system driver which does not call "register_filesystem" ?

      If you 'rtfa'... it stated:

      Another (again expected) observation is that the lack of (un)register_filesystem identifiers in the modules which only provides services to others: dlm, lockd, fat, and jbd2/jbd2.

      Yes I did read the article and thought it was very interesting. For one thing the fact that in earlier kernel releases file system drivers used to use more or less the same set of external symbols and in recent releases they more often used exclusive external symbols. If that's a good or a bad thing I don't know because I'm not an expert in file systems. It may indicate file system kernel module writers are re-using code less or do no longer compare their work that much anymore or it could mean file systems are now functioning more efficiently because they use more specialized external functions now... or maybe it means nothing or something completely else ... I don't know really ....

  16. Link to the article's Front Page please? by AceJohnny · · Score: 5, Informative

    Dear submitter,

    A /. summary is a bit like a main page on a website. Make the organization clear. Don't pile on shortcuts to different parts of the website: the reader risks being discouraged trying to find out how best to get to the important part of your website. Less is better.
    I actually clicked on one of the links that appeared to go to the "Expedition" website (based on its similarity to other links, as shown in my browser's statusbar!), then changed the address in the address bar to get to the front page.

    You actually didn't include a link to your article's front page, for heaven's sake!

    Hope this helps for the next time you write a summary.

    --
    Misleading titles? Inflammatory blurbs? Keep in mind that Slashdot is a tabloid.
    1. Re:Link to the article's Front Page please? by Hurricane78 · · Score: 1

      Hope this helps for the next time you write a summary.

      You must be new here...

      --
      Any sufficiently advanced intelligence is indistinguishable from stupidity.
  17. Odd visualization, but not so bad by AceJohnny · · Score: 5, Interesting

    My first reaction when browsing through the article was disgust about the measure being used. I couldn't see the value of graphs about the number of symbols used by different file-system modules. So vfat doesn't use kprintf, woo-fucking-hoo!

    But then, the Hamming distance and hierarchical structure struck a chord. Huh, so NFS, uses a very different set of symbols than, ext3. (ok, this may be biased by NFS using a metric fuckton of external symbols where ext3 uses less), which implies that NFS is pretty different, internally, than ext3.

    So now, I agree that such a visualisation, while very abstract from the implementation details, can be pretty useful to have a (very) rough overview of the filesystems and their code structure. And I think we need more abstract overviews like this.

    Reminds me of Code Swarm, which creates a movie of commits to a repository.

    --
    Misleading titles? Inflammatory blurbs? Keep in mind that Slashdot is a tabloid.
    1. Re:Odd visualization, but not so bad by JasterBobaMereel · · Score: 1

      This is a case of a thorough analysis done by someone who knows how to do an analysis but has no idea what they are analysing or what is important?

      So you end up with lots of pretty graphs and trees that actually mean absolutely nothing

          NFS has more external symbols than VFAT and there are less than in the previous kernel version... and this means ..... well not a lot really ....

      --
      Puteulanus fenestra mortis
    2. Re:Odd visualization, but not so bad by OneSmartFellow · · Score: 2, Funny

      He's in training to work for Accenture (or Capgemini, Deloitte) where the chief product is useless graphs, presented in marathon PowerPoint sessions in which management try to impress each other by using the latest buzz-phrases.

      They really do produce client-centric best practice inter departmental core competencies with a high return on investment and extremely low granularity event horizons. Just ask them !

    3. Re:Odd visualization, but not so bad by JasterBobaMereel · · Score: 1

      There usual recommendation is to rename the product and/or company ... which they did themselves Andersen Consulting became Accenture, this means you can advertise that you are the same as always and nothing interesting is going on ....

      --
      Puteulanus fenestra mortis
  18. Satanic messages by (pvb)charon · · Score: 2, Funny

    Man, I just thought I saw some obscured letters in one of the heatmaps but then realized it was just my damn glossy Macbook screen reflecting the writing on my t-shirt... Damn it...

    1. Re:Satanic messages by networkconsultant · · Score: 1

      Did you try reading the other side of your macbook, I hear there's a satanic message from Steve in the apple logo.

  19. Is it just me or ... by Anonymous Coward · · Score: 0

    Is it just me or do those thing remind anyone else of those back inside covers of old Mad magazine?

    I have this strange feeling that if I looked at the monitor for the edge that a bucktooth grin of Alfred E. Neuman would appear saying "What, me worry?"

  20. it's obviously by viralMeme · · Score: 1

    Taking a look at the image, it's obviously a signal from some alien civilization. Now if we could only decode it. Oh, wait that's what all the coders are about .. :)

  21. This is Linux... I know this! by Anonymous Coward · · Score: 0

    This immediately conjured up thoughts of Jurassic Park, where the young girl is surfing through the "city block" of file systems.

  22. Comment removed by account_deleted · · Score: 0, Offtopic

    Comment removed based on user account deletion

  23. Comment removed by account_deleted · · Score: 0, Offtopic

    Comment removed based on user account deletion

  24. Informative timelines by AlpineR · · Score: 3, Insightful

    My only interests in filesystems are how much free space I have and whether mine will recover from a power outage. Thus 95% of these graphs are a total bore to me.

    But I do like the Timelines of kernel releases. Some kernels see an exponential slowdown of release rate as they approach finalization and others are released like clockwork throughout their lifetime.

    I'd love to see these methods applied to other topics I care more about, like games and science

    When I develop maps for Starcraft, I usually go with a "release when it's ready" approach. That leads to a first public release long after my internal rough draft. Then there are a few quick releases as major bugs are found. And later the releases slow to a trickle as the focus move from bug fixes to balance tweaks. The magnitude of the changes also decreases over time, but each one's effect on game play can be disproportionately large.

    But recently I went with a public balancing approach. I released the rough draft to get a feel for how it played. Then released new drafts as often as twice a day as suggestions were made and problems became apparent. I love to see that contrast visually or see other patterns I hadn't considered.

    1. Re:Informative timelines by Eil · · Score: 1

      My only interests in filesystems are how much free space I have and whether mine will recover from a power outage.

      I'm usually more concerned about how much free space I have after a power outage.

  25. Re:Lick my by cheftw · · Score: 2, Interesting

    Who types this?

    But really I would like a visual expedition into the mind of whoever writes slashcode these days. I'm not even going to get started on UTF-8 support though. My problem is "Anonymous Cowardon". Latest Firefox stable shows no space between the two names (there is one, it's just tiny). It looks stupid, please fix.

    I'd attach a screenshot, but I'd know better than to click on a link replying to a troll myself so you'll just have to imagine if your browser can render it correctly.

    --
    Always back up, never back down. ---- Think you're cool 'cos your uid is prime? Take mine, modulo the one digit integers
  26. oh Jurrasic Park, what a prophetic movie by discogravy · · Score: 1

    Lex: It's a UNIX system! I know this!

  27. Devito? huh? by BitterAndDrunk · · Score: 1

    Did he do a parody of Pesci's speech in Good Fellas that I'm unaware of?

    --
    You better watch out, there may be dogs about . . .
  28. Stop merginig useless file systems by edivad · · Score: 1

    Linux should stop merging file systems used by three individuals at most.
    There are so many totally useless file systems inside the main tree, and I honestly fail to see why those should bloat the main tree while the junk could happily live in the hard drive of the three perspective users.
    For God's sake, even POHMELFS was merged! Where does it end? Where the boundary to say NO, and keep this irrelevant code out of the tree?

    1. Re:Stop merginig useless file systems by Slashcrap · · Score: 1

      Linux should stop merging file systems used by three individuals at most.
      There are so many totally useless file systems inside the main tree, and I honestly fail to see why those should bloat the main tree while the junk could happily live in the hard drive of the three perspective users.
      For God's sake, even POHMELFS was merged! Where does it end? Where the boundary to say NO, and keep this irrelevant code out of the tree?

      It's unlikely that anybody would care about your opinions even if they weren't stupid.

  29. What software did he use? by fccoelho · · Score: 1

    What software did he use for the analyses ?

  30. Re:Lick my by badkarmadayaccount · · Score: 1

    Same here. Also, does anybody else have rendering problems with part of the comments, more specifically, the dark gray band under the title which holds the username and timestamp? The band extends as a gray block from the username taking up it's width down to the Reply button. It's driving me nuts.

    --
    I know tobacco is bad for you, so I smoke weed with crack.
  31. Painfully obvious by Xylene2301 · · Score: 1

    Ho hum... Why did this nerd waste his time researching a matter that's obvious to everyone here. They had us run all this stuff down in the first 6 weeks of CS 101. I guess he thinks we're stupid! Harumph!