Slashdot Mirror
Replacing Atime With Relatime in the Kernel
eldavojohn writes "Our friend Jeremy at the Kernal Trap has dug up some interesting criticism of atime from Linus Torvalds. As Linus submitted patches to improve relatime he noted: 'I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.' And later severely beat atime about the head with a pointed stick: 'It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" Well, I guess I can expect my Linux machine to become a little bit faster!"
After I mounted my system with nodiratime and noatime, I did not 'feel' any actual speed increase. I didn't did any hard testing of course.
Disclaimer: Disregard the above post.
Seriously. Many have recommended mounting filesystems with the "noatime" parameter if you don't need to know atime for many years now,
if the poster had read the article they would have noticed that Linus did not say those things that are quoted - Ingo Molnar did.
Amazingly, standard Unix filesystems keep time of last access (atime), change of status (ctime), and file modification (mtime) but do not remember when the file was first created, which is something I have frequently wished for.
Intron: the portion of DNA which expresses nothing useful.
This is talking about the Last Access Time feature of most Unix systems. The OS will write a timestamp to disk each time the file is opened, closed, changed, or just looked at funny. Most professional Unixes (Solaris springs to mind) have the option of disabling the access timestamps. I haven't read the article yet, so I'm not quite sure why Linux hasn't followed suit by now. It's a security feature that's not all that useful on most systems.
Javascript + Nintendo DSi = DSiCade
/media/sda1 ext3 defaults,noatime 1 1 atime logs when a file has been written or read. So every time a file is used, it has to write an entry on the HDD, slowing performance, but it can have uses, like in forensics, security or backups (if a file has not been read in three years, it's probably safe to archive and move off the drive). I don't care for it, so I have the noatime line in my fstab.There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
atime updates actually help a LOT in some circumstances. I use it to help with security checks (even though you can hack the atime record), to help debug (what input file caused it to hang?), and all sorts of things where it's useful to know when a file was read.
I could cope with not having them, but in many cases I love the extra information atime gives.
File access timestamps, by default, are updated each time a file is read, and this occurs a write to the disk. Even if a file resides in cache, a write must be performed every time it is opened [even though the cache prevents a read from the disk]. A few people found that by disabling this, performance increases dramatically. The long and short of it: use noatime when you're mounting a file system [or in the fstab] if you want to try for yourself.
becomes
Deferred, maybe, but not for more than a few seconds, and when it happens it means more disk seeks...
It's long been standard practice to disable this on, eg, news/mail spools (anything with large numbers of read-mostly small files). The new plan of, if I understand correctly, only updating atime when the file is modified or if the new atime is more than 24 hours after the current atime should provide nearly the same functionality with practically none of the performance hit.
Yeah, I finally read the article. I thought I remembered that Linux could turn off atime, but I wasn't going to commit to it until I was sure.
FWIW, the Relative Access Time (relatime) patch simply doesn't update the access time unless the file has been modified since the last atime write. That allows ancient applications like MUTT to still synchronize on various files. Synchronization that does not work with noatime set.
Of course, I have to question why they're still using something as ancient as MUTT. A nice event system would be 1000x more efficient than trying to synchronize on flat files stored in your home directory on the file system. Of course, that would require designing OSes beyond the standard UNIX/POSIX philosophy and design. So I doubt we'll see that in Linux any time soon.
Javascript + Nintendo DSi = DSiCade
"You fell victim to one of the classic blunders! The most famous is never get involved in a flame war on Kerneltrap..."
"Can of worms? The can is open... the worms are everywhere."
Hey, Slashdot posted an article about me! [ They also renamed me to Linus - what more can a geek ask for? ;-) ]
In any case, the latest version of the better-relatime patch can be picked up from:
http://redhat.com/~mingo/relatime-patches/
Apply it, build it, reboot into the new kernel and enjoy a faster (and lower latency) desktop. (no fstab twiddling needed)
Just a handy tip for you (if you don't already use it) or others who are looking at disabling atime more for the power savings than the IO -- on laptops you should also be using noflushd (non-journaling filesystems only) or (ideally) Laptop Mode Tools. Also do not forget to configure syslog so that it doesn't constantly sync writes.
It's also worth mentioning that you *can* have atime enabled with properly configured laptop-mode and laptop-mode-tools and still see almost as much power savings -- The atime writes will still happen, but at least they will be buffered for when the HDD actually needs to spin up and do a lot of other more pressing IO.
However Mutt's use of atime simply is cheap enough that there's simply never been a reason to change it all the time most people have had atime updates on anyway. If it ain't broke, don't fix it.
On of my gripes with Linux is that one cannot spin down the disks to lessen their wear and tear.
Ive been told that the kernel constantly needs to access the disk...
Is this the reason of something else prevents the disks from spinning down?
If you're using a desktop system with a hard disk you'll hardly notice any difference unless you hammer the system really hard.
Remember though that most Linux systems are either embedded (using mainly flash) or servers. In both these cases atime updates can be very damaging to performance and should be avoided unless there's a very good reason to turn it on.
Engineering is the art of compromise.
You can disable it in a few different ways Once this is done, the Last Access Time attribute for newly created files will simply be their File Creation Time. Disabling Last Access Time may affect the operation of backup programs that use the Remote Storage service. YMMV
Or there's a registry key, which requires a reboot:
Cool! Amazing Toys.
"/dev/hda3 / ext3 defaults,noatime,nodiratime 0 1"
Actually, noatime is a superset of nodiratime, so there is no need to specify both.
(mount ext3 filesystem with noatime flag)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m15.231s
user 0m3.075s
sys 0m11.970s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m14.326s
user 0m2.928s
sys 0m11.172s
(remount without noatime flag)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m12.629s
user 0m2.687s
sys 0m9.772s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m12.401s
user 0m2.624s
sys 0m9.624s
Yes I think I'll stick with atime for now, thanks Linus.
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
Of course, I have to question why they're still using something as ancient as MUTT.
The thing I really *really* like about Mutt is that it uses Unix mailbox files. These are not just human-readable, they can also be manipulated using tools like 'cat'. I periodically archive my working mail files into a backup directory by just concatenating the working files onto the archive files of the same name. The resulting archive mail files are still fully usable with Mutt, even though some are 100Mb+ in size with mail going back to 1997. I could also use them with even older mail clients such as Pine, but that would be like using vi when you've got vim.
When I initially began using Linux, I used the Netscape email software, got fed up with it, and tried a few other mail clients. But none of them came close to the flexibility of Mutt. They all used their own mailbox formats, which could not be archived in the way I just described. I suspect that this is still true. I'm not going to trust Opera, Sylpheed, Thunderbird or Evolution with my mail, because (a) I doubt present-day mail files will still work reliably in 10 years time, (b) I can't easily migrate to another mail reader, and (c) I can't efficiently archive my email because the database files are not plain text.
That's why I like Mutt, anyway.
>north
You're an immobile computer, remember?
How can anyone write that with a straight face?
Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
But sometimes you need it... Whether it is to project your savings or to figure out, if a particular file was read within the last year.
My problem with atime is that it is not universal enough. For example, reading a file via mmap() or sending it directly to a socket via sendfile() (both methods widely used by web-servers) will not update its atime. The access-timestamp should be updated every time a file is opened for reading, rather than when a read() is issued on it...
So, when I wanted to report, when my little piece of software was last downloaded (via HTTP), I could not, unfortunately, rely on the file's atime...
In Soviet Washington the swamp drains you.
It's also commonly used by cleanup scripts to delete rarely accessed temp files and such. I'm sure there are plenty of scripts in production dotted around doing things like find -atime +7 -delete, some probably quite recent. Of course, I have to question why they're still using something as ancient as MUTT Err, because it's a nice mail client? Do you also want to question why we're still using bourne compatible shells, vi derivatives, or Perl? Also, it's spelled "mutt" or "Mutt", not "MUTT". Anyone would think you were trolling...
or backups (if a file has not been read in three years, it's probably safe to archive and move off the drive)
Ummmm... if you are doing regular backups then the atime will be changing every time so you can't tell if the file has not *really* been read in three years.
So atime is only useful in the case that you havn't backed that file up in three years and it has never even been accessed in that time... at which point I have to ask why you are only now deciding to archive it to offline media, if its relevence is really that low?
In the free world the media isn't government run; the government is media run.
I'm not sure your link is correct.
I don't think you can fully realize how poetic a phrase such as the one above can be until you read it with absolutely no understanding, as I do.
Thank you, ArcherB and thank you, Slashdot.
You are welcome on my lawn.
The old Netscape 4.x software also used Unix mailbox (mbox) files; it just also had index files. I'm not sure about current versions. mbox plus index was/is a popular storage format - mbox because it was simple, with added indexes for speed.
Maildir is my preferred format - each message is a separate plain text file, which means no locking issues, and simultaneous access from multiple machines. You can just tar the directory up if you want to move it, or cat all the files together if you want mbox back. You can use maildir with mutt (and lots of other software).
I'm sure it was written with a gay face.
"THERE IS NO JUSTICE, THERE IS ONLY ME." -Death
Is this something that's limited just to Linux and the ext3 filesystem?
/etc/rc, but perhaps it's been moved into that new SystemStarter business since I last checked). It seems like the same things ought to apply to HFS -- it has an attribute that's functionally identical (at least, I think it is -- feel free to correct me on this) to atime, stored in the catalog file -- but I'm not familiar enough with the workings of the filesystems to know if that's actually the case.
I'm particularly curious as to whether it's an issue on Mac OS X with the HFS filesystem also, and whether it would be possible / advisable to force Mac OS X to mount the root HFS partition as noatime/nodiratime.
OS X doesn't use a traditional UNIX-style fstab, so it's not immediately clear to me how you'd change the mount options (last time I checked disk mounting was all just in
If this doesn't occur in other OSes (I picked OS X because it's the other OS that I use frequently, and it uses a default fileystem that's pretty different in design from ext2/3), it seems like it might be worthwhile to look at why that is, and what tradeoffs other OSes have made to avoid the same issue.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."