Slashdot Mirror
Swap Performance in Linux
GizmoDuck writes "I'm working in a computational chemistry lab, and we find ourselves using memory and CPU hogs like Amber and Gaussian. The CPU hogging isn't a problem, thanks to Condor, but when submitting one of the jobs that request (and pretty much require) all the physical RAM in the machines, Linux promptly starts swapping so hard that the mouse pointer in X stops moving, NFS and NIS halt, and things don't get back to normal for five minutes. I've tried toying a bit with the settings in /proc/sys/vm/kswapd to no avail. I've done some poking around on the 'net looking for answers. Faster disks and swap partitions at the beginning of the drive aren't really an option at this point. I haven't found a good solution yet. I was wondering if the /. community has any input on how to keep the system from locking during periods of necessarily high swap activity?"
Maybe try out the preemptible kernel patch?
My personal experience is that it has helped my workstation's interactive performance noticeably for big ass c++ compiles and periods of lots of disk activity (big apt-get dist-upgrades). Thankfully, I'm no longer doing the big ass c++ compiles, so it's not as big of an issue as it used to be :)
It should improve interactive performance (i.e., your mouse will start moving again :) ) when load is high. Also, running your background process nice'ed will be helpful.
You might also consider a crazy idea of having swap file on NFS -- you'll get (if your network is decent) almost the same bandwidth as you get when accessing (older) disk, but much higher latency (this will put your background process in disadvantage compared to your interactive processes).
Hope this helps.
Paul B.
If your program(s) push Linux to the point where it actually runs out of available RAM faster than it can free it up, then "all hell breaks loose". It has to swap something out, and just about every program is eligible to be swapped out. That includes GPM (if you are on a virtual console) or X (if you are in X Windows). You need to account for all of these things to determine your RAM needs. Add up the memory usage of all your active programs, plus the buffer demands they have doing disk I/O, plus the kernel, and you need that much RAM. If the program is doing a LOT if disk/file writes, you can expect the buffer demands to be the majority of this, too (because the kernel believes what you just wrote you might soon want to read back, so it tries to keep lots of it in RAM even if that means swapping out GPM and X).
now we need to go OSS in diesel cars
The best way to handle this(or at least the best way I handled a similar situation) is to combine Robert Love's Preempt patch and Ingo's Scheduler.
They will significant increase high load user performance, keeps the system from running away with itself. If your feeling really, adventuresome you could also throw in Rik's Rmap VM...I have done very little testing with it, but I hear alot of reports that it helps.
there are all available in the authors respective directories on Kernel.org riel,rml,mingo
Power Corrupts,Absolute Power Corrupts Absolutely, leaving one person(group)in charge is absolutely corrupt.
Unfortunately, you're out of luck. The current linux VM (in later 2.4 series) is fine for low to medium load systems but falls apart on high load systems. The previous VM (early 2.4 series) is a good design but isn't really ready for production.
I would suggest buying more RAM (it's cheap) if you aren't already maxed at 4 gigs (x86). Alternatively switch to FreeBSD which has a very stable efficient VM. Any source should recompile without too much trouble and it can run linux binaries at almost full speed!
Best slashdot comment
Do you really have to be using the machine while it's calculating? If not, what about shutting down X and any other memory-hogging system components? Unlike on Windows you do have the option of turning off that expensive GUI.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Is switching to FreeBSD an option? The virtual memory management there is much better than in Linux under stress.
I'd have to agree. The author should look into using FreeBSD. A GIS project I'm currently working on allocates 3GB of RAM at startup. Until we get the rest of the funding for our SunFire solution, we're using what we have available, which is (was, actually: we've replaced the OS with FreeBSD) a P4 Linux box with 2GB of RAM, a 9GB SCSI drive for swap partition and a 36GB SCSI drive for everything else.
I'm not a Linux expert, but the techs in the department are. After a few weeks of their tinkering, it did pretty much the same thing as you're experiencing. I have a small development system at home (P3, 1GB RAM, 4GB SCSI swap, 40GB IDE for all else) running FreeBSD. Installed the software, and it runs like a charm. X works beautifully, Apache still serves up pages (of course, it doesn't get much traffic at home) and the program never chokes the system. Granted, with only a gig of real memory, it spends a fair amount of time accessing the disk (about 30 seconds every 2 minutes), and it steals almost all the cycles from dnetc!
Give them an inch and they'll take a foot. Much more than that, you won't have a leg to stand on.
With that as a given, if your app needs all available memory, run top and lsmod to see what's using your memory and remove everything you don't need (usually by deleting the links to those processes in the /etc/???/rc5.d directory).
If you can't remove it, scale it down. For example /etc/inittab lists off the different virtual terminals that appear when you press ctrl-alt and a function key. If you never use this feature, try reducing this down to 1 or 2 terminals. Leave some behind just in case you need them later. To do this, just comment the higher numbered lines that look like this;
6:2345:respawn:/sbin/mingetty tty6
(NOTE: Removing these lines might not make any difference -- it all depends on the distribution.)
As for X (assuming you need it and are using XFree), try removing any Load lines in the modules section that you don't need and scaling down the display size, background images, and color depth. Another big area of savings is changing the window manager. FVWM usually is installed, and while it is ugly it is also fairly light weight when compaired to KDE, Gnome, and other popular full-featured WMS.
While these steps alone won't eliminate the speed problems -- the other comments might solve that -- the time you spend waiting might be cut way down.
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.