Slashdot Mirror
Preload Drastically Boosts Linux Performance
Nemilar writes "Preload is a Linux daemon that stores commonly-used libraries and binaries in memory to speed up access times, similar to the Windows Vista SuperFetch function. This article examines Preload and gives some insight into how much performance is gained for its total resource cost, and discusses basic installation and configuration to get you started."
This is exactly why live CDs like Damn Small Linux (and Knoppix, if you have a ton of ram) run so fast if you load the CD image to ram. Ram is fast!
Shiny. Let's be bad guys.
I read a guide on the Gentoo forums a while ago about copying different directories into ram to "preload" them.
http://forums.gentoo.org/viewtopic-t-296892.html
I never actually tried it, but I might now that I have 4gb ram! A daemon to help automate this process would be welcome, though.
I don't know what rock you were under, but preload has been available for a while:
preload 0.2 release: 2005-09-01
And it was there before as it was packaged in Gentoo (back when it was still popular) and Suse 9.3
Custom electronics and digital signage for your business: www.evcircuits.com
Just let it go. This pissing match over innovation serves no useful purpose.
it doesn't make GNU/Linux as *fast* Microsoft Vista.
Is this functionality available in Apple's OS X?
I never had any luck with preload the times I tried it (a year or two ago?). Nowadays I use alltray for preloading often used apps that are a bit chunky such as Firefox or Openoffice. Openoffice also has a built in preload feature...but you can use alltray anyway for the same effect.
Once you start despising the jerks, you become one.
You have the option not to run this sort of program. If it sucks, turn it off.
Also, Windows' VM system (IMNSHO) has always sucked and will continue to suck; predictive loading of entire bits of software has nothing to do with it.
You mean like losing the data after a few hours of no power?
now we need to go OSS in diesel cars
innovation = first time to do something from your point of view
invention = first time to do something ever
Note how MS is always careful to point out they innovate.
*flush*
Tsunami -- You can't bring a good wave down!
Vista's implementation is marketed as being useful for older, slower machines with less RAM, where it actually may be unwanted, and could cause performance issues (unless it's disabled below a certain threshold - it might be). It's only really useful if you have lots of RAM (around 2GB or so). Yes, SuperFetch has an extra mode where it uses a USB-2 stick as a secondary disk cache, but that's not what we're talking about here. That mode is generally perceived as a gimmick.
Linux handles having lots of RAM a lot better than Windows (XP) does, because of differences in the way the caching system was designed. Linux (and OS X) was intended to run entirely from RAM and use little swap. I've run, say, OpenOffice once, not used it for several weeks, and the next time I start it it loads almost instantly, because it was still sitting in the cache. My machines have 2GB of RAM, with much less than 500MB actually in use - the remaining 1.5GB is being used as disk cache. Swap usage is either zero, or very close. Of course, performance goes to hell if you do something that flushes the disk cache, or if you try using such a system on a machine with 256MB of RAM.
Windows, on the other hand, was designed to run almost entirely from swap, and tends to drop stuff from the disk cache when it's not been used in a while, as well as moving stuff out to swap rather aggressively. That works great if you barely have enough RAM to run the OS, but it's terribly wasteful if you have more than enough RAM. In this case, SuperFetch is actually useful, allowing it to catch up to and actually surpass Linux, by monitoring which files are actually used and making sure they're already in the disk cache.
That's great, although nothing new. Other OSes have had this for years (this Linux implementation dates back to 2005, Mac OS X has had it for ages, and neither implementation was original) - Microsoft were just the first to brand it.
TFA said nothing about Vista's implementation.
I think the primary problem people have with Microsoft's implementations is that they're typically very complicated, and have a tendency to degrade over time. XP is the typical whipping boy for this - none of the self-maintaining performance stuff (prefetching, or the prelinker) actually works for longer than about six months, meaning that an XP installation starts off fast, gradually gets faster, and then rapidly slows down as the system tries to speed itself up.
The submitter is the author of the blog, and is merely paraphrasing the whitepaper written by the author of the software -- and that is two years old. Nothing new or interesting here, just someone trying to draw eyeballs to his blog.
It's actually a little different than the preload that's been in Gentoo for years. The core functionality is of course the same, but now a daemon runs that caches libraries and updates the linkage periodically. So, it can possibly give much more performance, since everything is always up-to-date. It will be standard in Hardy Heron when it comes out.
SuperFetch was one of the first things that I had to disable in Vista. I had downloaded a linux distro (a large .iso file) using Firefox, and for the next two weeks, everytime I rebooted my computer I would have to listen to my hard drive chug away for the next 10 minutes while it loaded the file into memory. (The new resource monitor in Visa is nice -- that is what helped me track down the problem).
My computer is MUCH faster now that SuperFetch is disabled. Like night and day.
Vista users respond positively toward the speed boost everytime we "Reload" their Vista. The downtime and data lost as a result of "Reload" might irritate some disgruntled users, but most of them enjoy the free break at the expense of the company.
Nothing in those Linus thingy could beat that user satisfaction. I might be bias though.
The Amiga one of the same era was very good. You had a recoverable RAM disk, which functioned the same as a standard RAM disk, but would maintain its contents on restart. That meant reboots were lightning quick, and any data you stored in the RRAM disk was still there.
Shame we haven't got back to that level of functionality.
"I've got more toys than Teruhisa Kitahara."
Only if you run out of cold spray.
I'm envisioning a sensible sort of preload program in gentoo right now:
... *snip* ...
*Preloading commonly used data, libraries, and binaries...
gcc OK
make OK
libc-dev OK
emerge OK
kernel-src OK
*Preload done, 3827K of USE Flags, 2TB of source code, and one compiler, and firefox to surf forums.gentoo.org for better use flags while you compile loaded into main memory
Photos.
In Linux, most things never reach the x.0 stage, no matter how mature they are.
This reminds me of a geek girlfriend I had... she told me she was 29.9.9.12.1 years old. But when I met her in real life, i was suprised she had a daughter... 17.1.25 years old!
When information is power, privacy is freedom.
I have a pretty good amount of memory on my current machine - 2Gb - and I mostly just never close any applications, especially with the big ones like Gimp just reusing the already open instance when you open a new file. I suspect that preload would not actually be all that useful for me in practice; I'm still goign to enable it to see if I'm wrong, though.
Trust the Computer. The Computer is your friend.
You mean Linux adapted something from Windows instead of the other way around? What's next, a sane proactor i/o api?
Not really. Caching policies like this have been around for longer than Windows has even existed. Most of the things that Linux "adopts" from Windows or Macintosh originally came from UNIX or mainframes. Even in 2008, there is hardly an original idea in any of those operating systems. And preload itself is, of course, older than Vista.
You can be mad at Vista for a number of reasons, but SuperFetch is not one of them - I have noticed a decent speed improvement because of it, and look forward to having something similar in Linux.
It's not clear to me why this should be a separate user process; what it's doing is simple enough that whatever is doing can be done directly by the kernel. In fact, I wouldn't be surprised if you could get the same speedups by simply tuning a couple of kernel parameters.
Does anyone knows the difference between the two projects? Does preload have a better algorithm for selecting the files to read? Does it also use this special syscall?
Except when you were coding something that ran away in memory, corrupted the RAD (the name of the recoverable RAM disk), destroying everything you hadn't saved for an hour or so. :)
(The Amiga didn't have an MMU originally, and even when they got it, the OS didn't support memory protection due to the shared message passing)
c++;
tried it out on my little eeepc and it definitely made a difference, on average its sped up all loading times by about 30 percent. This is especially good because i upgraded to a 2gig stick of ram but most programs hardly need that much ram and on average im left with about 1.2 gigs just sitting there doing nothing, now the ram is more productive and the loading time is noticably faster eg. firefox on a cold start without preload took 10 seconds to load before, now on a cold start it loads in 6 :). Also since the cpu is relatively slow it means fetching data and the overhead of moving it around it cut down alot. I'd love to shake the creators hand for this plucky little piece of software :) thanks!
The series of comments to which you replied is about Linux LiveCDs, which don't require/touch the hard disk unless you explicitly tell them to. Using "toram" or "dochache" or similar kernel switches allows the entire contents of the CD to be loaded to the ramdisk, dramatically speeding up loading and allow one to remove the CD.
Even if your particular LiveCD is set to watch for and automatically use swap partitions, a HD is still significantly faster than an optical drive. If you install Linux permanently to your HD those particular kernel switches no longer do anything.
You could do it in the kernel, but you shouldn't. The kernel keeps track of files using inodes and device numbers, not paths, which may be volatile between reboots (udev+kernel can dynamically assign device numbers to kernel devices, filesystems are identified and mounted by scanning for UUIDs or labels in superblocks, etc). A tracking daemon can monitor system calls and keep a small database with logical paths, access patterns, and so forth; the user-space view of activity tracks intent better so the statistics can be more meaningful.
Moreover the act of caching the file is easily accomplished by a low-priority user-space task which speculatively reads the files which may be referenced soon. In this fashion the kernel memory manager does not need to be changed in any way; we are not creating a new kernel memory pool which would need new logic under memory pressure. In the case that RAM is suddenly needed for storing application pages or an unexpected demand-loaded program binary or library, we can flush these buffered files just like any other cached file; it's not treated any differently. It's just a daemon touching files (with the hope of a benefit in startup times of applications that require them).
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
"Sounds like 7 years ago you had no idea how to build a decent computer..."
Sounds like you're quite an arrogant asshole...
If it don't GO... chrome it. ~ Frank Banks
Note the 379M number. That is the amount of data read from disk and kept in ram. When an application needs to malloc and no completely free memory, yes it will free up those pages (it ideally picks the cache least likely to be needed again). But absolutely, disk contents are kept in disk cache, but only after load. And no, memory leaks aren't hopelessly pervasive.
What preload does normally happens implicitly during boot. It's hard to demonstrate on init scripts effectively, but log into gnome right after boot, and the disk will thrash like crazy. Log out, kill every last process of that user, log in again. It will be quite dramatic. preload aims for that subsequent experience without the pain of the first.
So what preload brings is simple, and all that has to happen is simple, know which files are relevant to typical usage ahead of time, and be aggressive about 'cat file >
Linux implicitly aids this, but the user interface side still subjectively 'feels' bogged down because it won't proactively load things it doesn't know you'll need, despite the ability to derive this historical data in user space. If preload takes idle time (let's say, for example, while services with arbitrary sleeps and while waiting for username and password) and proactively gets cache populated, it is more IO work in the aggregate (disk will be hit up for things that will never be needed), but it will feel smoother out of the gate.
XML is like violence. If it doesn't solve the problem, use more.