Slashdot Mirror

← Back to Stories (view on slashdot.org)

Anatomy of Linux Journaling File Systems

Posted by timothy on from the dear-diary dept.

LinucksGirl writes "Journaling file systems used to be an oddity primarily for research purposes, but today it's the default in Linux. Discover the ideas behind journaling file systems, and learn how they provide better integrity in the face of a power failure or system crash. Learn about the various journaling file systems in use today, and peek into the next generation of journaling file systems."

59 comments

  1. Obligatory. by Tackhead · · Score: 4, Funny

    "And then there's ReiserFS, which had integrity issues when it came to retrieving the location of a bunch of widely-scattered bits under an unbalanced tree."

    1. Re:Obligatory. by nawcom · · Score: 4, Insightful

      when it came to retrieving the location of a bunch of widely-scattered bits... go ahead, mod me down, but for a second there i thought there was some inside joke referring to his wife in Tackhead's post.
    2. Re:Obligatory. by c0d3h4x0r · · Score: 1

      Replace "widely-scattered bits under and unbalanced tree" with "dead wives," and then I think you've got it.

      --
      Moderator hint: a comment is neither "Flamebait" nor "Troll" if it is true.
    3. Re:Obligatory. by Anonymous Coward · · Score: 4, Funny

      Replace "widely-scattered bits under and unbalanced tree" with "dead wives," and then I think you've got it.

      If you had done so automatically and silently as intended, then I think you'd have gotten "the joke".

    4. Re:Obligatory. by MSG · · Score: 4, Funny

      Jokes about murder aren't funny. What's wrong with you?

    5. Re:Obligatory. by Fast+Thick+Pants · · Score: 3, Funny

      Nothing that a few swift blows to the back of the head with a five iron can't fix...

    6. Re:Obligatory. by Yfrwlf · · Score: 3, Funny

      Yeah, no, they aren't, see how funny it is to you when it's someone you care about.

      --
      Promote true freedom - support standards and interoperability.
    7. Re:Obligatory. by LingNoi · · Score: 4, Funny

      I saw this post on the main page and instantly knew they'd be a Reiser joke. In fact, that's all I came here for, cya!

    8. Re:Obligatory. by Anonymous Coward · · Score: 0

      BZZZZT, wrong. _Murder_ isn't funny. Jokes about murder, on the other hand, can be utterly _hilarious_.

    9. Re:Obligatory. by Anonymous Coward · · Score: 0

      Someone once said that human beings make the most fun out of the most serious situations.

    10. Re:Obligatory. by Anonymous Coward · · Score: 0

      Your mom's funny.

    11. Re:Obligatory. by drinkypoo · · Score: 5, Insightful

      I've been in car crashes, yet jokes about car crashes are still funny in spite of my personal experience to tell me that the event itself is potentially very serious. I only hope that if someone I care about is murdered, I'll still manage to retain my sense of humor. It would be a shame if it had to die, too.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    12. Re:Obligatory. by Anonymous Coward · · Score: 0

      I already used all my good rape and genocide jokes here with the Firefox story.

    13. Re:Obligatory. by evilviper · · Score: 2, Funny

      Jokes about murder aren't funny.
      ...but jokes about rape are HILARIOUS!
      --
      Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
    14. Re:Obligatory. by neomunk · · Score: 1

      I will refer you to George Carlin's album "Parental Advisory: Explicit Lyrics" for a strong and sensible rebuttal to your position.

    15. Re:Obligatory. by neomunk · · Score: 1, Funny

      Proof (via George Carlin): Picture Porky Pig raping Elmer Fudd...

    16. Re:Obligatory. by consonant · · Score: 1
      Obligatory is right.

      Especially with the highly provocative story title..

      --
      [Slashdot Comments We Liked]
    17. Re:Obligatory. by Profane+MuthaFucka · · Score: 1

      Are you aware that you just proposed that a murder be committed as a form of some kind of science fair experiment?

      --
      Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
    18. Re:Obligatory. by prestomation · · Score: 1

      http://en.wikipedia.org/w/index.php?title=Comparison_of_file_systems&oldid=209063556#Features

      That is all

    19. Re:Obligatory. by Anonymous Coward · · Score: 0

      Q: What was the last thing that O.J. said to Nicole?
      A: Your waiter will be with you in a minute.

    20. Re:Obligatory. by Anonymous Coward · · Score: 0

      That's not funny. My brother died that way.

    21. Re:Obligatory. by Anonymous Coward · · Score: 0

      Wait a minute, is this Digg?!

    22. Re:Obligatory. by Anonymous Coward · · Score: 0

      Q: What was the last thing that O.J. said to Nicole? A: Your waiter will be with you in a minute.

      I don't get it. :-/

    23. Re:Obligatory. by Anonymous Coward · · Score: 0

      Ron Goldman was her waiter.

    24. Re:Obligatory. by Anonymous Coward · · Score: 0

      It makes me glad that I wasn't the only one.

    25. Re:Obligatory. by Anonymous Coward · · Score: 0

      That's why I came here too.

    26. Re:Obligatory. by painehope · · Score: 1

      So just kill the person that made the joke.

      --
      PC moderators can suck my White pierced, tattooed dick. If you think pride == hate, s/dick/Aryan meat mallet/g.
    27. Re:Obligatory. by Anonymous Coward · · Score: 0

      You wouldn't lose your sense of humor, you'd just realize that certain tasteless jokes aren't funny.

      I used to not be bothered by jokes about rape and sexual assault, then my best friend was raped and I witnessed first hand the pain that it causes. I don't think I've lost my sense of humor (I guess I'm not the best judge of that though), I just realize now that certain things shouldn't be trivialized by stupid jokes. Like a man murdering his wife, for instance.

    28. Re:Obligatory. by Kusuriya · · Score: 1

      been in many car crashes. Had several family members die in horribly bloody and violent ways some of them were murders... guess what... the jokes about how they died never get old.

    29. Re:Obligatory. by Yfrwlf · · Score: 0

      Then, you grow up, and realize that car crashes actually aren't funny when someone gets hurt.

      --
      Promote true freedom - support standards and interoperability.
    30. Re:Obligatory. by drinkypoo · · Score: 3, Insightful

      So, I'm to be considered an adult when my sense of humor is compatible with yours? In my world, the one who can't laugh at life is the one whose life is lacking. I reject your world view and all the fearfulness it implies. They can kill you once, but they can't kill you twice.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
    31. Re:Obligatory. by Yfrwlf · · Score: 1

      I'm not implying any fearfulness, and I respect your want to try to move on and make amends emotionally. If laughing is your way of emotionally dealing with a hardship even though you're sad inside, then so be it. All I'm saying is that it's still a tragedy. I know that certain things aren't good, and making fun of them not only boarders on a disrespect for those involved and who were negatively impacted, but I simply see it as childish and insensitive. If that's where your humor level is though, then so be it. I know tragedy though, and I find no reason to laugh at others when they go through it.

      I just don't support negativity, and prefer the real reasons for being happy which don't involve making fun of others, which is apparently something unfelt by internet griefers for example. It's pretty sad when you feel you have to validate your importance by picking on others. Back in my day, it was called bullying.

      --
      Promote true freedom - support standards and interoperability.
  2. File systems should know more about file type by Animats · · Score: 5, Interesting

    File systems should know more about file type. Not "file type" in the extension sense, but file type in the sense of what the data written to the file needs for integrity.

    There are only a few standard use cases:

    • The entire file is the commit unit. For most files, you either want the entire file written correctly or you don't want anything written. When nothing is written, the previous version, if any, should remain intact. Applications with "Save" functions need this model. For many binary file types, from images to executables, a partial file is totally useless. So the file should be committed when closed. IBM put this in some of their early UNIX versions, the ones based on UCLA Locus. Done right, if the program crashes before closing or committing the file, the file reverts to the old version. It's not necessary to update the metadata until file close, so this is the fastest mode, and should be the default.
    • Log files. Files are only extended; old data is not overwritten. Ordered journaling is desirable, so that after a crash, the file is intact out to the point of the crash. This is UNIX "open for append" mode.
    • Database files. The file is being used to support a database with read/write data structures. The database system needs to know when the file system has committed a write and may need to know about ordering of queued writes. This is the most complex case, but database implementors pay attention to file system details and are willing to make special calls if necessary to tell the file system when to commit and to wait for commits to complete.

    If those three cases are properly supported, you should never see a garbled file from an unexpected shutdown. Some of the file systems out there have approximately the right feature set for this, but there's no standardized interface and set of expectations that corresponds to these use cases.

    1. Re:File systems should know more about file type by klapaucjusz · · Score: 4, Interesting

      The entire file is the commit unit. For most files, you either want the entire file written correctly or you don't want anything written. When nothing is written, the previous version, if any, should remain intact.

      You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. Fsync before renaming for extra credit.

      Good text editors have been doing that for as long anyone can remember.

      Log files. Files are only extended [...] This is UNIX "open for append" mode.

      Unfortunately, it doesn't work very well, since POSIX doesn't (AFAIK) specify the largest write that is guaranteed to be atomic. Hence, unless you're careful, you may end up with log entries from two processes being interleaved.

    2. Re:File systems should know more about file type by 7+digits · · Score: 2, Insightful

      > You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. Fsync before renaming for extra credit. And lose all the hard links you may have had on that file...

    3. Re:File systems should know more about file type by Animats · · Score: 1

      You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. Fsync before renaming for extra credit.

      It's not the default. It's hard to do portably. The ritual for doing it on Windows is quite complex and usually implemented wrong. Rename is only atomic on some UNIX/Linux file systems. One can end up with forgotten temp files lying around. File loss is possible on systems lacking an atomic rename function, although the data is still in a temporary file. Somewhere.

    4. Re:File systems should know more about file type by Carnildo · · Score: 5, Informative

      MacOS is probably the most advanced in this regard: it's got a system call for "swap the contents of these two files". This does an atomic swap, and as a side benefit, preserves any links to the files.

      --
      "They redundantly repeated themselves over and over again incessantly without end ad infinitum" -- ibid.
    5. Re:File systems should know more about file type by dargaud · · Score: 1

      There's also the case of changing a few bytes inside an existing file using fseek/fwrite. You don't want the file duplicated, but you have to if the operation needs to be atomic, haven't you ? I guess it's similar to the database case.

      --
      Non-Linux Penguins ?
    6. Re:File systems should know more about file type by vrmlguy · · Score: 1

      There's also the case of changing a few bytes inside an existing file using fseek/fwrite. You don't want the file duplicated, but you have to if the operation needs to be atomic, haven't you ? I guess it's similar to the database case. You need a file system that supports copy-on-write. Whenever you change a block, the old block is first saved elsewhere. When the file is closed, the old blocks are either discarded, or (more commonly) kept around as snapshots. Check out WAFL (http://en.wikipedia.org/wiki/Write_Anywhere_File_Layout) and ZFS (http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model).
      --
      Nothing for 6-digit uids?
    7. Re:File systems should know more about file type by Anonymous Coward · · Score: 0

      File systems should know more about file type. Not "file type" in the extension sense, but file type in the sense of what the data written to the file needs for integrity.

      Or maybe you should put those different types of files on different partitions, formatted or configured to work in those methods?

      Which is one of the central strengths of unix/linux - because you don't need to have a "one size fits all" mentality when it comes to file systems.

    8. Re:File systems should know more about file type by shani · · Score: 1

      Log files. Files are only extended [...] This is UNIX "open for append" mode.

      Unfortunately, it doesn't work very well, since POSIX doesn't (AFAIK) specify the largest write that is guaranteed to be atomic. Hence, unless you're careful, you may end up with log entries from two processes being interleaved.

      You are wrong, any size buffer passed to write() is guaranteed to be written atomically:

      http://www.opengroup.org/onlinepubs/000095399/functions/write.html

      Look for O_APPEND.
    9. Re:File systems should know more about file type by klapaucjusz · · Score: 1

      Unfortunately, it doesn't work very well, since POSIX doesn't (AFAIK) specify the largest write that is guaranteed to be atomic. Hence, unless you're careful, you may end up with log entries from two processes being interleaved.

      You are wrong, any size buffer passed to write() is guaranteed to be written atomically: http://www.opengroup.org/onlinepubs/000095399/functions/write.html Look for O_APPEND.

      Indeed. Thanks for the info.

  3. I am jack's file system by Anonymous Coward · · Score: 5, Funny

    Dear journal,

    Today I was suddenly restarted. It seems as if the large meat machine which regularly uses me was startled by a file which was being written to my logs, "goatse.jpg". Fortunately, thanks to my reliability, The meat machine will be able to view the image upon his return! I hope it is happy with me!

    Yours truly,
    XFS

    1. Re:I am jack's file system by Anonymous Coward · · Score: 0

      that would be 'hello.jpg'

      as in http://goat.cx/hello.jpg

  4. ehhh by larry+bagina · · Score: 1, Troll

    More anatomy of LinucksGirl, less anatomy of Linux file systems.

    --
    Do you even lift?

    These aren't the 'roids you're looking for.

    1. Re:ehhh by Anonymous Coward · · Score: 0

      homo

  5. More interesting story: by bersl2 · · Score: 5, Informative

    Barriers and journaling filesystems

  6. Hey, she's HOT! by rts008 · · Score: 0, Offtopic

    Here you go!

    --
    Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
    1. Re:Hey, she's HOT! by nawcom · · Score: 0

      Slashdot: News for nerds... I mean horny perverts. Who are married.

    2. Re:Hey, she's HOT! by Anonymous Coward · · Score: 0

      You need to look up what 'perversion' means, because being horny is normal. Being attracted to the opposite sex is normal.

      It's being married that's the perversion! :)

  7. Anatomy? Reiser? Scary... by mangu · · Score: 1

    I wish that people submitting stories and writing blurbs chose more carefully the words they use.

  8. Atomic replacing under all file systems/ by tepples · · Score: 2, Interesting

    You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. This is true if your file system supports atomic replacing. Under some file systems, renaming file A over file B will cause file A to replace file B. Under other file systems, renaming file A over file B will return a failure code or throw an exception; an application is expected to delete file B, warn the user, and then retry moving file A. See the documentation for Python os.rename .