Anatomy of Linux Journaling File Systems

LinucksGirl writes "Journaling file systems used to be an oddity primarily for research purposes, but today it's the default in Linux. Discover the ideas behind journaling file systems, and learn how they provide better integrity in the face of a power failure or system crash. Learn about the various journaling file systems in use today, and peek into the next generation of journaling file systems."

Comments

File systems should know more about file type

[Animats]

File systems should know more about file type. Not "file type" in the extension sense, but file type in the sense of what the data written to the file needs for integrity.

There are only a few standard use cases:

• The entire file is the commit unit. For most files, you either want the entire file written correctly or you don't want anything written. When nothing is written, the previous version, if any, should remain intact. Applications with "Save" functions need this model. For many binary file types, from images to executables, a partial file is totally useless. So the file should be committed when closed. IBM put this in some of their early UNIX versions, the ones based on UCLA Locus. Done right, if the program crashes before closing or committing the file, the file reverts to the old version. It's not necessary to update the metadata until file close, so this is the fastest mode, and should be the default.
• Log files. Files are only extended; old data is not overwritten. Ordered journaling is desirable, so that after a crash, the file is intact out to the point of the crash. This is UNIX "open for append" mode.
• Database files. The file is being used to support a database with read/write data structures. The database system needs to know when the file system has committed a write and may need to know about ordering of queued writes. This is the most complex case, but database implementors pay attention to file system details and are willing to make special calls if necessary to tell the file system when to commit and to wait for commits to complete.

If those three cases are properly supported, you should never see a garbled file from an unexpected shutdown. Some of the file systems out there have approximately the right feature set for this, but there's no standardized interface and set of expectations that corresponds to these use cases.

Re:File systems should know more about file type

[klapaucjusz]

The entire file is the commit unit. For most files, you either want the entire file written correctly or you don't want anything written. When nothing is written, the previous version, if any, should remain intact.

You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. Fsync before renaming for extra credit.

Good text editors have been doing that for as long anyone can remember.

Log files. Files are only extended [...] This is UNIX "open for append" mode.

Unfortunately, it doesn't work very well, since POSIX doesn't (AFAIK) specify the largest write that is guaranteed to be atomic. Hence, unless you're careful, you may end up with log entries from two processes being interleaved.

Atomic replacing under all file systems/

[tepples]

You don't need any extra kernel support for that. You just write the new version under a temporary name, and then atomically rename it over the old file. This is true if your file system supports atomic replacing. Under some file systems, renaming file A over file B will cause file A to replace file B. Under other file systems, renaming file A over file B will return a failure code or throw an exception; an application is expected to delete file B, warn the user, and then retry moving file A. See the documentation for Python os.rename .