Slashdot Mirror


Replacing Atime With Relatime in the Kernel

eldavojohn writes "Our friend Jeremy at the Kernal Trap has dug up some interesting criticism of atime from Linus Torvalds. As Linus submitted patches to improve relatime he noted: 'I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.' And later severely beat atime about the head with a pointed stick: 'It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" Well, I guess I can expect my Linux machine to become a little bit faster!"

33 of 416 comments (clear)

  1. Personally by Nikron · · Score: 5, Interesting

    After I mounted my system with nodiratime and noatime, I did not 'feel' any actual speed increase. I didn't did any hard testing of course.

    --
    Disclaimer: Disregard the above post.
    1. Re:Personally by ciroknight · · Score: 4, Informative

      I can't "feel" a difference on my desktop with 2GB of RAM, but on my laptop which only has 1, and a much slower disk, the improvement is pretty noticeable (as in from "nearly unusable for lots of access applications [trackerd]" to "smooth as silk").

      It's funny the kernel devs are just now talking about this, I discovered this almost 6 months ago on the Ubuntu forums while glancing around for a fix for my laptop's ridiculous sound issues.

      --
      "Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
    2. Re:Personally by Burz · · Score: 4, Informative

      I discovered noatime about 5 years ago, right about the same time I learned that DMA was turned off by default. I was horrified to hear that Linus' opinion of DMA at the time was 'no performance gain'.

      I'm surprised Linus is just cluing in now about atime updates: The noatime workaround is listed in all kinds of performance-tuning guides.

    3. Re:Personally by Trepalium · · Score: 4, Informative

      For a laptop, there is a more important reason to use noatime. Without that option the hard drives may never get the chance to spin down and save power because /bin/sync always has something to write to the disk (it's own atime update).

      --
      I used up all my sick days, so I'm calling in dead.
  2. Probably overblown by jimicus · · Score: 4, Insightful

    Seriously. Many have recommended mounting filesystems with the "noatime" parameter if you don't need to know atime for many years now,

    1. Re:Probably overblown by Yokaze · · Score: 4, Interesting
      No, it is not overblown, because, as Alan Cox put it:

      Ext3 currently is a standards compliant file system. Turn off atime and its very non standards
      compliant, turn to relatime and its not standards compliant but nobody will break (which is good)

      It is no option for the kernel to make noatime standard, as it would brake POSIX compliance. But without noatime, the OS suffers a large penalty compared to some other OSs. The magnitude of the penalty has been made clear in the quote from the article.
      So a solution, which is POSIX compliant, but doesn't suffer as much a penalty as the current solution is sought for.
      --
      "Between strong and weak, between rich and poor [...], it is freedom which oppresses and the law which sets free"
    2. Re:Probably overblown by Fred+Ferrigno · · Score: 4, Insightful

      That's a really silly question to ask the user at install. If you had to answer all questions of that level of significance, it would take forever. ("Do you want version 0.8.5 of this library or 0.9.1? 0.9.1 has more features but is less compatible...") Let the distro pick a sensible default and have users who know or care about it change it later for themselves.

      The only real question is whether noatime or relatime make for a more sensible default.

  3. Linus did not say that! by JbirdUAH · · Score: 5, Informative

    if the poster had read the article they would have noticed that Linus did not say those things that are quoted - Ingo Molnar did.

    1. Re:Linus did not say that! by Anonymous Coward · · Score: 5, Funny

      My name is Ingo Molnar and you mis-attributed my quote. Prepare to die!

  4. atime vs ctime by Intron · · Score: 5, Insightful

    Amazingly, standard Unix filesystems keep time of last access (atime), change of status (ctime), and file modification (mtime) but do not remember when the file was first created, which is something I have frequently wished for.

    --
    Intron: the portion of DNA which expresses nothing useful.
    1. Re:atime vs ctime by EvanED · · Score: 5, Interesting

      There is a technical reason for this.

      A lot of the time, modification of a file... isn't a modification of a file. Instead, the program will delete the existing file and create a new one in its place. (There is sometimes other operations in there, like saving to a temp file, deleting the original, then renaming the temp file to the original file name.)

      This means that storing the real creation time of a file means that it won't be what you expect, because the file that you think is the same file actually isn't.

      (MS-DOS/Windows have something called filesystem tunneling to attempt to get around this problem. If a file is deleted and a new one created in its place (see the MSDN article linked to from there for details) within a default 15 seconds, the creation time of the old file is carried over. This technique exchanges purity and absolute correctness (not that metadata times are reliable against tamper anyway) for utility.)

    2. Re:atime vs ctime by EvanED · · Score: 5, Interesting

      I know self-replies are stupid, but I should have mentioned something else. The metadata tunneling that Windows does is much more important than it is on Unix because the filename may need to be tunneled as well. If you open a file called "somefile with a long name.txt" in an old DOS program by opening SOMEFI~1.TXT (or even in a recent program) and it does this delete/create thing, you don't want the OS to say "oh, you're making a new file called SOMEFI~1.TXT. Spiffy"; you need the original "somefile with a long name.txt" name to carry over.

      The linked site explains all this, but I know the propensity of /. readers (myself included) to RTFA. ;-)

  5. Re:Ummm.. by ArcherB · · Score: 5, Informative
    In your /etc/fstab, add noatime to your options portion of you drive line. For example:

    /media/sda1 ext3 defaults,noatime 1 1 atime logs when a file has been written or read. So every time a file is used, it has to write an entry on the HDD, slowing performance, but it can have uses, like in forensics, security or backups (if a file has not been read in three years, it's probably safe to archive and move off the drive). I don't care for it, so I have the noatime line in my fstab.

    --
    There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
  6. Re:Summary please? by mpeg4codec · · Score: 5, Informative

    File access timestamps, by default, are updated each time a file is read, and this occurs a write to the disk. Even if a file resides in cache, a write must be performed every time it is opened [even though the cache prevents a read from the disk]. A few people found that by disabling this, performance increases dramatically. The long and short of it: use noatime when you're mounting a file system [or in the fstab] if you want to try for yourself.

  7. Re:Ummm.. by Compholio · · Score: 5, Informative

    Most professional Unixes (Solaris springs to mind) have the option of disabling the access timestamps. I haven't read the article yet, so I'm not quite sure why Linux hasn't followed suit by now.
    Linux has had the capability to turn off access timestamps for a long time. "man mount" has details, but all you really have to do is edit /etc/fstab:
    /dev/hda3 / ext3 defaults 0 1
    becomes
    /dev/hda3 / ext3 defaults,noatime,nodiratime 0 1
  8. Re:Ummm.. by mikeee · · Score: 4, Insightful

    Deferred, maybe, but not for more than a few seconds, and when it happens it means more disk seeks...

    It's long been standard practice to disable this on, eg, news/mail spools (anything with large numbers of read-mostly small files). The new plan of, if I understand correctly, only updating atime when the file is modified or if the new atime is more than 24 hours after the current atime should provide nearly the same functionality with practically none of the performance hit.

  9. You fell victim to one of the classic blunders! by stefanlasiewski · · Score: 5, Funny

    "You fell victim to one of the classic blunders! The most famous is never get involved in a flame war on Kerneltrap..."

    --
    "Can of worms? The can is open... the worms are everywhere."
    1. Re:You fell victim to one of the classic blunders! by dcapel · · Score: 5, Funny

      A Slashdot summary wrong? Inconceivable!

      --
      DYWYPI?
  10. latest relatime patch by Ingo+Molnar · · Score: 5, Informative

    Hey, Slashdot posted an article about me! [ They also renamed me to Linus - what more can a geek ask for? ;-) ]

    In any case, the latest version of the better-relatime patch can be picked up from:

    http://redhat.com/~mingo/relatime-patches/

    Apply it, build it, reboot into the new kernel and enjoy a faster (and lower latency) desktop. (no fstab twiddling needed)

    1. Re:latest relatime patch by Anonymous Coward · · Score: 5, Funny

      I agree - modifying a text file is a messy complicated business only suitable for the elite super hackers out there. It's much simpler for me to apply the patch and recompile the kernel.

      I kid, I kid.

    2. Re:latest relatime patch by Ingo+Molnar · · Score: 5, Informative

      I agree - modifying a text file is a messy complicated business only suitable for the elite super hackers out there. It's much simpler for me to apply the patch and recompile the kernel.

      I kid, I kid.

      ok, you are kidding, but i'll still bite :-)

      Firstly, the patch is mainly about modifying relatime behavior to make it more compatible and more usable.

      The fact that you dont have to change fstab is no big deal, provided you have the right util-linux package installed, with the relatime user-space patch applied which not even the latest distro devel repositories have included.

      If you dont have that then adding "relatime" to your fstab might leave you with a read-only mounted root filesystem and some commandline (or rescue-image) tinkering to do.

      People prefer all-in-one kernel patches that just turns on the feature they are interested in. You'd be surprised how many people are willing to try almost arbitrary kernel patches but loathe to touch their user-space environment in any way.

      And ... it's also kind of ironic that this relatively small patch often brings more practical benefits to the desktop than all the "big" desktop interactivity/latency features (cfs, swap-prefetch, -rt kernel) combined.

      Ingo^H^H^H^H Linus

  11. Re:Laptop power issue. by GoRK · · Score: 5, Informative

    Just a handy tip for you (if you don't already use it) or others who are looking at disabling atime more for the power savings than the IO -- on laptops you should also be using noflushd (non-journaling filesystems only) or (ideally) Laptop Mode Tools. Also do not forget to configure syslog so that it doesn't constantly sync writes.

    It's also worth mentioning that you *can* have atime enabled with properly configured laptop-mode and laptop-mode-tools and still see almost as much power savings -- The atime writes will still happen, but at least they will be buffered for when the HDD actually needs to spin up and do a lot of other more pressing IO.

  12. Re:Ummm.. by vidarh · · Score: 5, Insightful
    People are using Mutt because it works and has the features lots of people needs. I have to question why people keep upgrading just for the sake of it. As for an event system, Linux has had that too for ages - search for dnotify and inotify.

    However Mutt's use of atime simply is cheap enough that there's simply never been a reason to change it all the time most people have had atime updates on anyway. If it ain't broke, don't fix it.

  13. Is this reason why we cant spin down disks? by grims · · Score: 4, Interesting

    On of my gripes with Linux is that one cannot spin down the disks to lessen their wear and tear.
    Ive been told that the kernel constantly needs to access the disk...

    Is this the reason of something else prevents the disks from spinning down?

    1. Re:Is this reason why we cant spin down disks? by rahvin112 · · Score: 4, Informative

      Stopping and Starting a disk causes the most wear on the disk. When in an idle state the spinning disk has fully lubricated bearings and is using just a tiny amount of energy. Strap in a stop and start and you have bearings on start that don't have their lubrication optimally spread (gravity and lack of motion cause the metal to get closer together and depending on the manufacturer may have some of the bearings without lubrication), and the spinup of the disks themselves is on average a 6W draw (idle is like .3W). Plus, when the disks are fully spinning wobble is going to be constrained a bit by the speed, whereas on startup and stopping the minor imperfections in the disk can let the minor wobbles show which strains the bearings and causes wear and tear. See if the bearings start to fail and are say 20% shot, the motor on the drive can just use a little more energy to drive them, but the startup cost might exceed what the motor is capable of outputing. Using the example above, the .3W draw on idle might climb to .6W, but on startup that could translate in moving from 6W to 12W (and exceeding what the motor can physically provide).

      And if you don't believe any of that you shouldn't have any trouble using google to find Admins who tell horror stories about having to reboot a drive and losing the entire drive because the bearings were shot to the point that once the disks stopped the motor couldn't generate enough force to restart them. But the disk could have lasted years more as long as it wasn't stopped. In fact in companies where a lot of data is stored the disks are put on their own power source at least partially because the disks don't have to be stopped if a server needs to be rebooted because of failure or updates. This is also one of the reasons to be wary of purchasing used storage arrays. Might have worked great when they shut it off, but you might be able to restart the array.

  14. As a flash fs writer... by EmbeddedJanitor · · Score: 5, Informative
    atime really hurts some file systems, particularly flash file systems. Many/most flash file systems don't support atime for that reason. So, even if you're running atime in the kernel you will often not be getting atime in the fs.

    If you're using a desktop system with a hard disk you'll hardly notice any difference unless you hammer the system really hard.

    Remember though that most Linux systems are either embedded (using mainly flash) or servers. In both these cases atime updates can be very damaging to performance and should be avoided unless there's a very good reason to turn it on.

    --
    Engineering is the art of compromise.
  15. atime on EXT3, huh? by Trogre · · Score: 4, Funny

    (mount ext3 filesystem with noatime flag)

    $ time for i in `seq 1 10000`; do touch file1.dat; done

    real 0m15.231s
    user 0m3.075s
    sys 0m11.970s

    $ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done

    real 0m14.326s
    user 0m2.928s
    sys 0m11.172s

    (remount without noatime flag)

    $ time for i in `seq 1 10000`; do touch file1.dat; done

    real 0m12.629s
    user 0m2.687s
    sys 0m9.772s

    $ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done

    real 0m12.401s
    user 0m2.624s
    sys 0m9.624s

    Yes I think I'll stick with atime for now, thanks Linus.

    --
    "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
  16. Re:Ummm.. by Cheesey · · Score: 4, Interesting

    Of course, I have to question why they're still using something as ancient as MUTT.

    The thing I really *really* like about Mutt is that it uses Unix mailbox files. These are not just human-readable, they can also be manipulated using tools like 'cat'. I periodically archive my working mail files into a backup directory by just concatenating the working files onto the archive files of the same name. The resulting archive mail files are still fully usable with Mutt, even though some are 100Mb+ in size with mail going back to 1997. I could also use them with even older mail clients such as Pine, but that would be like using vi when you've got vim.

    When I initially began using Linux, I used the Netscape email software, got fed up with it, and tried a few other mail clients. But none of them came close to the flexibility of Mutt. They all used their own mailbox formats, which could not be archived in the way I just described. I suspect that this is still true. I'm not going to trust Opera, Sylpheed, Thunderbird or Evolution with my mail, because (a) I doubt present-day mail files will still work reliably in 10 years time, (b) I can't easily migrate to another mail reader, and (c) I can't efficiently archive my email because the database files are not plain text.

    That's why I like Mutt, anyway.

    --
    >north
    You're an immobile computer, remember?
  17. Re:Ummm.. by nschubach · · Score: 5, Funny

    $ man mount

    How can anyone write that with a straight face?
    --
    Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
  18. Accounting is a nuisance in general by mi · · Score: 4, Interesting

    But sometimes you need it... Whether it is to project your savings or to figure out, if a particular file was read within the last year.

    My problem with atime is that it is not universal enough. For example, reading a file via mmap() or sending it directly to a socket via sendfile() (both methods widely used by web-servers) will not update its atime. The access-timestamp should be updated every time a file is opened for reading, rather than when a read() is issued on it...

    So, when I wanted to report, when my little piece of software was last downloaded (via HTTP), I could not, unfortunately, rely on the file's atime...

    --
    In Soviet Washington the swamp drains you.
  19. Re:Laptop power issue. by Eideewt · · Score: 4, Informative

    I'm not sure your link is correct.

  20. Re:Ummm.. by PopeRatzo · · Score: 4, Funny

    I have the noatime line in my fstab.
    Ah, these are the kind of stories I first starting coming to Slashdot for.

    I don't think you can fully realize how poetic a phrase such as the one above can be until you read it with absolutely no understanding, as I do.

    Thank you, ArcherB and thank you, Slashdot.
    --
    You are welcome on my lawn.
  21. Re:Ummm.. by misleb · · Score: 5, Funny

    How can anyone write that with a straight face?


    I'm sure it was written with a gay face.

    --
    "THERE IS NO JUSTICE, THERE IS ONLY ME." -Death