Slashdot Mirror
Replacing Atime With Relatime in the Kernel
eldavojohn writes "Our friend Jeremy at the Kernal Trap has dug up some interesting criticism of atime from Linus Torvalds. As Linus submitted patches to improve relatime he noted: 'I cannot over-emphasize how much of a deal it is in practice. Atime updates are by far the biggest IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday Linux performance than all the pagecache speedups of the past 10 years, _combined_.' And later severely beat atime about the head with a pointed stick: 'It's also perhaps the most stupid Unix design idea of all times. Unix is really nice and well done, but think about this a bit: 'For every file that is read from the disk, lets do a ... write to the disk! And, for every file that is already cached and which we read from the cache ... do a write to the disk!'" Well, I guess I can expect my Linux machine to become a little bit faster!"
if the poster had read the article they would have noticed that Linus did not say those things that are quoted - Ingo Molnar did.
This is talking about the Last Access Time feature of most Unix systems. The OS will write a timestamp to disk each time the file is opened, closed, changed, or just looked at funny. Most professional Unixes (Solaris springs to mind) have the option of disabling the access timestamps. I haven't read the article yet, so I'm not quite sure why Linux hasn't followed suit by now. It's a security feature that's not all that useful on most systems.
Javascript + Nintendo DSi = DSiCade
Yes its there...I use noatime on many linux servers. Most definetly use it where news and maildir type mail folders are mounted.
/dev/sda2 /var/spool ext3 noatime 1 2 /dev/sdb1 /home ext3 noatime,usrquota 1 2
For instance on the mail server do this in the fstab (not necessary on root and the like...just use it on the heavy traffic mounts):
In your /etc/fstab, append noatime to the options of your main file systems. This will prevent the access timestamps of files and directories from being updated on each access. Not so sure what the relatime stuff is, but I imagine the use of noatime will also prevent the bottlenecks seen from relatime.
/media/sda1 ext3 defaults,noatime 1 1 atime logs when a file has been written or read. So every time a file is used, it has to write an entry on the HDD, slowing performance, but it can have uses, like in forensics, security or backups (if a file has not been read in three years, it's probably safe to archive and move off the drive). I don't care for it, so I have the noatime line in my fstab.There is no "I disagree" mod for a reason. Flamebait, Troll, and Overrated are not substitutes.
atime updates actually help a LOT in some circumstances. I use it to help with security checks (even though you can hack the atime record), to help debug (what input file caused it to hang?), and all sorts of things where it's useful to know when a file was read.
I could cope with not having them, but in many cases I love the extra information atime gives.
File access timestamps, by default, are updated each time a file is read, and this occurs a write to the disk. Even if a file resides in cache, a write must be performed every time it is opened [even though the cache prevents a read from the disk]. A few people found that by disabling this, performance increases dramatically. The long and short of it: use noatime when you're mounting a file system [or in the fstab] if you want to try for yourself.
becomes
if the file is appended, the ctime will be updated. the BSD tail checks for an inode change or a decrease in filesize (and reopens in that case). Otherwise, the file descriptor is kept open and continually read from. When new data is written to the file, it will be available for reading automatically.
Do you even lift?
These aren't the 'roids you're looking for.
I think lazy-atime already keeps a dirty cache of atimes.
I think relative-lazy-atime optimizes this further by only scheduling updates for atime if the old atime is older than ctime or mtime. If the old atime is newer than ctime or mtime then it silently forgets if you've accessed the file. If you are updating ctime or mtime, then I think atime is updated anyhow.
This optimization still allows certain programs to tell if files have been read once since they were last modified (probably the only non-security feature of atime that is in common use), but of course ignores when they are read multiple times.
So every time a file is used, it has to write an entry on the HDD, slowing performance,... so I have the noatime line in my fstab.
I have noatime set on my laptop.
Writing atime to the disk every time data is read from the cache keeps the disk spinning, burning battery power. "noatime" means the disk can shut down much of the time, spinning up only when writes must be synced or data not yet in cache must be read.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
No.
You could schedule the update (updatedb) to run at a more convenient time.
Locate is a pretty useful command - locate file, and it shows you instantly where it is. I don't use it a lot but when I do it's VERY handy to have.
Hey, Slashdot posted an article about me! [ They also renamed me to Linus - what more can a geek ask for? ;-) ]
In any case, the latest version of the better-relatime patch can be picked up from:
http://redhat.com/~mingo/relatime-patches/
Apply it, build it, reboot into the new kernel and enjoy a faster (and lower latency) desktop. (no fstab twiddling needed)
I can't "feel" a difference on my desktop with 2GB of RAM, but on my laptop which only has 1, and a much slower disk, the improvement is pretty noticeable (as in from "nearly unusable for lots of access applications [trackerd]" to "smooth as silk").
It's funny the kernel devs are just now talking about this, I discovered this almost 6 months ago on the Ubuntu forums while glancing around for a fix for my laptop's ridiculous sound issues.
"Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
If you're using a desktop system with a hard disk you'll hardly notice any difference unless you hammer the system really hard.
Remember though that most Linux systems are either embedded (using mainly flash) or servers. In both these cases atime updates can be very damaging to performance and should be avoided unless there's a very good reason to turn it on.
Engineering is the art of compromise.
"/dev/hda3 / ext3 defaults,noatime,nodiratime 0 1"
Actually, noatime is a superset of nodiratime, so there is no need to specify both.
Stopping and Starting a disk causes the most wear on the disk. When in an idle state the spinning disk has fully lubricated bearings and is using just a tiny amount of energy. Strap in a stop and start and you have bearings on start that don't have their lubrication optimally spread (gravity and lack of motion cause the metal to get closer together and depending on the manufacturer may have some of the bearings without lubrication), and the spinup of the disks themselves is on average a 6W draw (idle is like .3W). Plus, when the disks are fully spinning wobble is going to be constrained a bit by the speed, whereas on startup and stopping the minor imperfections in the disk can let the minor wobbles show which strains the bearings and causes wear and tear. See if the bearings start to fail and are say 20% shot, the motor on the drive can just use a little more energy to drive them, but the startup cost might exceed what the motor is capable of outputing. Using the example above, the .3W draw on idle might climb to .6W, but on startup that could translate in moving from 6W to 12W (and exceeding what the motor can physically provide).
And if you don't believe any of that you shouldn't have any trouble using google to find Admins who tell horror stories about having to reboot a drive and losing the entire drive because the bearings were shot to the point that once the disks stopped the motor couldn't generate enough force to restart them. But the disk could have lasted years more as long as it wasn't stopped. In fact in companies where a lot of data is stored the disks are put on their own power source at least partially because the disks don't have to be stopped if a server needs to be rebooted because of failure or updates. This is also one of the reasons to be wary of purchasing used storage arrays. Might have worked great when they shut it off, but you might be able to restart the array.
Your test is invalid. Try again with 10000 different files, and make sure you unmount/mount before each test.
I discovered noatime about 5 years ago, right about the same time I learned that DMA was turned off by default. I was horrified to hear that Linus' opinion of DMA at the time was 'no performance gain'.
I'm surprised Linus is just cluing in now about atime updates: The noatime workaround is listed in all kinds of performance-tuning guides.
Syslog is often the culprit so if you can get it to stop writing log files all the time your disk can sleep longer. There is a syslog out there somewhere that limits it's writes to disk to when it fills a cache - the quick hack otherwise with an unchanged system is to get it to dump logs to a ram disk (kernel parameter on boot to enable ram disks) and sync every day or whenever.
Another thing is cron - work out what it is doing and if it really needs to run that thing every hour, day or whenever and wake up the disk.
That's a start - I think there's actually a HOWTO on running linux with a minimum of power usage that is a few years old that goes into this.
This setting appears to be disabled by default on Vista. (ie atime is not set). Which makes sense with the amount of thumbnail viewing that goes on in that OS.
I.O.U One Sig.
For a laptop, there is a more important reason to use noatime. Without that option the hard drives may never get the chance to spin down and save power because /bin/sync always has something to write to the disk (it's own atime update).
I used up all my sick days, so I'm calling in dead.
In the email discussion linked in the article kernel devs were wondering the same thing... why doesn't mutt/bash just use inotify on the mailbox/maildir if available at config/build time? (Linux does have a event system in epoll and inotify -- and we almost got that unified under kevent, and it's threatening to happen again pulling in AIO too)
They were also concerned about breakage of other systems that are not open source (HSMs) which use atime for usage tracking. Noting that HSMs and atime-sensitive maintenance programs usually don't exhibit time resolution in policies more fine than a day, they came up with the relatime = update at least once a day compromise.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
I just enabled it on my little file server at home.
Specs:
AMD 3000+, 512MB ram, 1x 80GB IDE, 2x 500GB SATA (Raid)
MASSIVE difference in performance, even just doing an ls on a directory with a lot of files/directories in it.
Try it on a multi terabyte high speed raid0 for scratch data on a big box like a 64 cpu altix. noatime can give massive, and by massive I mean MASSIVE speed increases. atime is retarded, I totally agree with Linus.
The old Netscape 4.x software also used Unix mailbox (mbox) files; it just also had index files. I'm not sure about current versions. mbox plus index was/is a popular storage format - mbox because it was simple, with added indexes for speed.
Maildir is my preferred format - each message is a separate plain text file, which means no locking issues, and simultaneous access from multiple machines. You can just tar the directory up if you want to move it, or cat all the files together if you want mbox back. You can use maildir with mutt (and lots of other software).
Turning it off (or using fastfs) can substantially improve the speed of a filesytem restore.
Bit of trivia -- NTFS does the exact same thing with regards to access times. There's a registry entry to disable it buried somewhere deep that I don't remember at the moment, but if you're stuck on Windows it might be worth looking up to improve I/O performance.
Bit of additional trivia, Vista already disables it by default, and the whole access timestamp mechanism has changed, removing this performance penalty all together.
Been doing this on NTFS (Windows NT-based OS (NT 3.51, NT 4.0, 2000/XP/Server 2003)) for more than a decade now, via this registry parameter (alongside others I do not need, like DOS backwards compatibility, as all my programs are Win32 ones nowadays, for example):
r ol\FileSystem]
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Cont
"NtfsDisable8dot3NameCreation"=dword:00000001
"Win31FileSystem"=dword:00000000
"Win95TruncatedExtensions"=dword:00000001
"AsyncFileCommit"=hex:01
"ContigFileAllocSize"=dword:00001000
"DisableLastAccessUpdate"=dword:00000001
"DisablePagingExecutive"=dword:00000001
"NtfsDisableLastAccessUpdate"=dword:00000001
"Disable8dot3NameCreation"=dword:00000001
Some of those today, might be legacy support OR invalid for NT-based OS (like ones for Win9x would be), which is no biggie, because if they are invalid parameters? They only get read @ system startup, & if invalid, they're ignored (penalty = SUPER-MINOR registry bloat, 1-N lines @ most)...
(Specifically, ones like like Win31 filesystem & DOS 8.3 naming being kept up to date, especially since normal "DOS" can't even READ NTFS, without addons, for example, will help since modern OS & filesystems can use, what? 254 characters in their names? Why keep the DOS entry then in the MFT$? An unnecessary write))
STILL, those entries above for NTFS filesystem performance control?
They, for the MOST part if not all, all ought to still be totally function & USEFUL for performance' sake, nowadays even.
If I am "off" on ANY of those above? Correct me if needed - things DO go "out of date" in this field, very quickly, but those SHOULD still be ok, & of SOME performance-enhancing gains as well.
(... & every LITTLE BIT, helps, when you optimize for performance STRICTLY, more on the trade off below! Much like a mustache (each hair doesn't count for much, but, put them all together, you have VISIBLE results you CAN SEE/NOTICE, easily)).
The 'downside' would be for access tracking: When each file may have been used/touched etc. for security purposes.
(Trade off = speed, vs. POSSIBLE security uses)
APK
P.S.=> I've been doing this for more than a DECADE now on NTFS & Windows NT-based OS - I am personally surprised that folks from the *NIX world (especially the penguins) have NOT caught onto & applied this decades ago as well... note the trade offs though (speed, vs. security potentials really)! apk