Slashdot Mirror
Replacing Atime With Relatime in the Kernel
eldavojohn writes "Our friend Jeremy at the Kernal Trap has dug up some interesting criticism of atime from Linus Torvalds. As Linus submitted patches to improve relatime he noted: 'I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.' And later severely beat atime about the head with a pointed stick: 'It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" Well, I guess I can expect my Linux machine to become a little bit faster!"
After I mounted my system with nodiratime and noatime, I did not 'feel' any actual speed increase. I didn't did any hard testing of course.
Disclaimer: Disregard the above post.
Since I have no idea what atime or relatime actually are, could someone just tell me which kernel settings should be changed re: this story for an ideal desktop system?
Thanks.
Seriously. Many have recommended mounting filesystems with the "noatime" parameter if you don't need to know atime for many years now,
And, for every file that is already cached and which we read from the cache ... do a write to the disk!'
Uh, why? Or rather, why not cache the atime update until disk usage is clear, THEN do the write, at least for already cached files?
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
if the poster had read the article they would have noticed that Linus did not say those things that are quoted - Ingo Molnar did.
Amazingly, standard Unix filesystems keep time of last access (atime), change of status (ctime), and file modification (mtime) but do not remember when the file was first created, which is something I have frequently wished for.
Intron: the portion of DNA which expresses nothing useful.
Yes its there...I use noatime on many linux servers. Most definetly use it where news and maildir type mail folders are mounted.
/dev/sda2 /var/spool ext3 noatime 1 2 /dev/sdb1 /home ext3 noatime,usrquota 1 2
For instance on the mail server do this in the fstab (not necessary on root and the like...just use it on the heavy traffic mounts):
atime updates actually help a LOT in some circumstances. I use it to help with security checks (even though you can hack the atime record), to help debug (what input file caused it to hang?), and all sorts of things where it's useful to know when a file was read.
I could cope with not having them, but in many cases I love the extra information atime gives.
Why should atime updates have to be written out to disk immediately? It probably isn't the end of the world if a few get lost if a filesystem doesn't get unmounted cleanly, and it probably updates a *lot* more frequently than anything else in the inode, so why not just have the filesystem keep the atimes separately from the rest of the metadata somewhere? It would only take a little bit of space to hold all the atimes on most filesystems (4 bytes per atime times say 250,000 files plus 5% for indexing overhead (you'd have to map inode numbers to indices into the array of atimes) is just a little over a megabyte), so if you just set that aside somewhere, cached a copy in memory somewhere, and wrote out updates whenever there was some free bandwidth to the disk, you'd be able to merge updates for many different files together instead of having to write out an entire block for every atime update, that you have to write out immediately because it counts as an inode update and it'd be bad to let those fall out of sync.
File access timestamps, by default, are updated each time a file is read, and this occurs a write to the disk. Even if a file resides in cache, a write must be performed every time it is opened [even though the cache prevents a read from the disk]. A few people found that by disabling this, performance increases dramatically. The long and short of it: use noatime when you're mounting a file system [or in the fstab] if you want to try for yourself.
if the file is appended, the ctime will be updated. the BSD tail checks for an inode change or a decrease in filesize (and reopens in that case). Otherwise, the file descriptor is kept open and continually read from. When new data is written to the file, it will be available for reading automatically.
Do you even lift?
These aren't the 'roids you're looking for.
The exactness of atime might suffer, but nobody will notice.
That said, I agree the noatime mount option covers most needs. Almost nothing actually uses atime. The Relatime patch therefore only updates atime if mtime or ctime is newer (to show that the file has been accessed since it has been updated or created) or the current atime is over 1 day old. This increases IO usage only slightly compared to noatime, but is less likely to break those few applications that do use atime.
Stylish sheet to fix many problems in Slashdot's D3: https://gist.github.com/801524
"You fell victim to one of the classic blunders! The most famous is never get involved in a flame war on Kerneltrap..."
"Can of worms? The can is open... the worms are everywhere."
So every time a file is used, it has to write an entry on the HDD, slowing performance,... so I have the noatime line in my fstab.
I have noatime set on my laptop.
Writing atime to the disk every time data is read from the cache keeps the disk spinning, burning battery power. "noatime" means the disk can shut down much of the time, spinning up only when writes must be synced or data not yet in cache must be read.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
I can't believe this is even an issue. I've been mounting everything noatime on my BSD systems for ages. The only thing I can think that it might possibly be used for are scripts that purge /tmp of files that haven't been accessed in a while. Even for that, ctime or mtime isn't a bad compromise.
Bit of trivia -- NTFS does the exact same thing with regards to access times. There's a registry entry to disable it buried somewhere deep that I don't remember at the moment, but if you're stuck on Windows it might be worth looking up to improve I/O performance.
I thought I'd tack this onto the FP, but the OP is severely mistaken. First of all, Ingo was the one who finally submitted the patch, and it was also Ingo who made the 'faster than all the improvements over the last 10 years combined' statement.
If you don't believe me RTFA. I just wish the OP had actually read the article before submitting it.
+5, Truth
No.
You could schedule the update (updatedb) to run at a more convenient time.
Locate is a pretty useful command - locate file, and it shows you instantly where it is. I don't use it a lot but when I do it's VERY handy to have.
Hey, Slashdot posted an article about me! [ They also renamed me to Linus - what more can a geek ask for? ;-) ]
In any case, the latest version of the better-relatime patch can be picked up from:
http://redhat.com/~mingo/relatime-patches/
Apply it, build it, reboot into the new kernel and enjoy a faster (and lower latency) desktop. (no fstab twiddling needed)
Where is Hans Reiser when you need him? This is exactly the type of question he likes to argue about.
http://michaelsmith.id.au
On of my gripes with Linux is that one cannot spin down the disks to lessen their wear and tear.
Ive been told that the kernel constantly needs to access the disk...
Is this the reason of something else prevents the disks from spinning down?
If you're using a desktop system with a hard disk you'll hardly notice any difference unless you hammer the system really hard.
Remember though that most Linux systems are either embedded (using mainly flash) or servers. In both these cases atime updates can be very damaging to performance and should be avoided unless there's a very good reason to turn it on.
Engineering is the art of compromise.
Problem solved. Many older/slower machines [laptops] can be sped up considerably by this step.
You can disable it in a few different ways Once this is done, the Last Access Time attribute for newly created files will simply be their File Creation Time. Disabling Last Access Time may affect the operation of backup programs that use the Remote Storage service. YMMV
Or there's a registry key, which requires a reboot:
Cool! Amazing Toys.
I wish Unix (including Linux) would be more proactive about culling options which are not used by the majority and have such a detrimental effect. If the disk performance leaps by 10% by not using atime, it seems a no-brainer that noatime should be the default. If some app uses atime, it should be fixed or deprecated. There is no real excuse for dragging the whole system's performance for the sake of some crummy mail app.
(mount ext3 filesystem with noatime flag)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m15.231s
user 0m3.075s
sys 0m11.970s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m14.326s
user 0m2.928s
sys 0m11.172s
(remount without noatime flag)
$ time for i in `seq 1 10000`; do touch file1.dat; done
real 0m12.629s
user 0m2.687s
sys 0m9.772s
$ time for i in `seq 1 10000`; do cat file1.dat >>/dev/null; done
real 0m12.401s
user 0m2.624s
sys 0m9.624s
Yes I think I'll stick with atime for now, thanks Linus.
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
But sometimes you need it... Whether it is to project your savings or to figure out, if a particular file was read within the last year.
My problem with atime is that it is not universal enough. For example, reading a file via mmap() or sending it directly to a socket via sendfile() (both methods widely used by web-servers) will not update its atime. The access-timestamp should be updated every time a file is opened for reading, rather than when a read() is issued on it...
So, when I wanted to report, when my little piece of software was last downloaded (via HTTP), I could not, unfortunately, rely on the file's atime...
In Soviet Washington the swamp drains you.
In the email discussion linked in the article kernel devs were wondering the same thing... why doesn't mutt/bash just use inotify on the mailbox/maildir if available at config/build time? (Linux does have a event system in epoll and inotify -- and we almost got that unified under kevent, and it's threatening to happen again pulling in AIO too)
They were also concerned about breakage of other systems that are not open source (HSMs) which use atime for usage tracking. Noting that HSMs and atime-sensitive maintenance programs usually don't exhibit time resolution in policies more fine than a day, they came up with the relatime = update at least once a day compromise.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Nothing about POSIX requires that it be impossible to get higher-resolution time stamps. You just can't get them with POSIX-only code.
For instance, OS X (and possibly other BSD-flavored UN*Xes) either defines the time stamps, in "struct stat", as struct timespec st_[amc]timespec; and #defines st_[amc]time as st_[amc]timespec.tv_sec, or puts a long st_[amc]timensec; field after each time_t st_[amc]time; field, depending on whether _POSIX_C_SOURCE is defined - POSIX-only code will work, and non-POSIX-only code can get at the higher-resolution times.
And the POSIX requirement for st_atime to be updated has nothing to do with the issue you're complaining about in any case.
Name one single use.
There is a file on my drive, and I want to find out when it was created, just out of curiosity?
for tracking which files are no longer used. That is why backup utilities such as tar have a --preserve-atime option to avoid updating atime during a backup. The proposed relatime option preserves the function of finding files no one uses anymore - for that purpose, 1 day resolution is fine. HSM systems depend on atime to know what to archive. Again, 1 day resolution is fine for that purpose.
Is this something that's limited just to Linux and the ext3 filesystem?
/etc/rc, but perhaps it's been moved into that new SystemStarter business since I last checked). It seems like the same things ought to apply to HFS -- it has an attribute that's functionally identical (at least, I think it is -- feel free to correct me on this) to atime, stored in the catalog file -- but I'm not familiar enough with the workings of the filesystems to know if that's actually the case.
I'm particularly curious as to whether it's an issue on Mac OS X with the HFS filesystem also, and whether it would be possible / advisable to force Mac OS X to mount the root HFS partition as noatime/nodiratime.
OS X doesn't use a traditional UNIX-style fstab, so it's not immediately clear to me how you'd change the mount options (last time I checked disk mounting was all just in
If this doesn't occur in other OSes (I picked OS X because it's the other OS that I use frequently, and it uses a default fileystem that's pretty different in design from ext2/3), it seems like it might be worthwhile to look at why that is, and what tradeoffs other OSes have made to avoid the same issue.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
Well, not exactly, but I got a used notebook drive. A year after I started using that notebook as my home web server, heat and centrifugal force and such seems to have spun the grease away from the contact surfaces of the bearings. Or, the disks may have been sleeping and restarting every ten minutes to run the dynamic dns client. (I really need to figure out how to keep that script and the perl it uses in a RAM disk or something.)
Anyway, it started humming from the re-seeks caused by disks that couldn't maintain speed, and then eventually the disk froze.
Thought I had lost the data.
But I thought twice about it. If I just trashed the drive, the data was gone. I couldn't afford to send it in to a professional recovery service, and I did have backups that were sort of recent, anyway. And I wanted to show the insides of the drive to my son.
So I opened the enclosure, showed it to my son but didn't let him touch it, rotated the disks by hand (very carefully avoiding touching or letting dust fall on the disk surfaces), closed it up, plugged it into a USB shirt-pocket enclosure, and pulled off my data.
Turns out I can still use the drive to carry unimportant data around. (Very light use.) I don't trust critical data on it, of course.
joudanzuki