Swap Performance in Linux

GizmoDuck writes "I'm working in a computational chemistry lab, and we find ourselves using memory and CPU hogs like Amber and Gaussian. The CPU hogging isn't a problem, thanks to Condor, but when submitting one of the jobs that request (and pretty much require) all the physical RAM in the machines, Linux promptly starts swapping so hard that the mouse pointer in X stops moving, NFS and NIS halt, and things don't get back to normal for five minutes. I've tried toying a bit with the settings in /proc/sys/vm/kswapd to no avail. I've done some poking around on the 'net looking for answers. Faster disks and swap partitions at the beginning of the drive aren't really an option at this point. I haven't found a good solution yet. I was wondering if the /. community has any input on how to keep the system from locking during periods of necessarily high swap activity?"

Comments

Linux 2.4.x VM

[Trevelyan]

Did you miss all the 2.4 Linux VM Stories?

I suggest build/installing the latest kernel with the aa VM (the default VM, since 2.4.10). If you still have VM (Swap) problems then go get the latest rmap VM patch and try that.

The kernel VM (Virtual Machine) is what manages memory and sawp, btw.

And if u did miss all the VM stories, a summery:
at the start of 2.4 a new fancy mv was put in to action, using something known as reverse mapping. this was very clever but it wasn't quite ready and there were teathing troubles then suddenly (2.4.10) Linus switched VM to one similar to that of 2.3 (with some updates and a few features from the previous 2.4 VM) This started a big fight, which caused concerns (such that it may split the linux comunity)

which is better i dont know some swer by one other swer the other. but unless ur using RH 2.4.9 kernel i would not recommend a pre 2.4.10 kernel.

however you may need to experiment which is best the VM now in 2.4 (to stay) or rmap, u should try both and see

steps
Install 2.4.[17,18,19]
try it
if it fails u try the rmap patch

Re:FreeBSD

[Strog]

How many CPU's?

FreeBSD on a dual processor box will match a dual linux box any day. I didn't say beat but they can go back and forth depending on exact applications. If you go more cpus then performance will start dropping off in a hurry. It shouldn't do as well on paper but real world applications show that it performs very well with 2 cpus. It's kind of like micro-kernel *should* be better than monolithic but in the real world it isn't.

I'd go with a dual FreeBSD box any day especially if it is going to be under high loads. I have more linux boxes than anything else right now but their performance under load has been an issue. If you would rather stick with linux then look at some of the alternative VMs out there. I would stick with linux if you have more than 2 cpus unless you really want to go with a commercial Unix (Solaris x86 maybe??).

Not mentioned yet -- go lean...

[Spoing]

I'll second the other comments already made. In addition, sometimes the simplest ideas are the most valuable, though I'll assume you can't just drop in more RAM.

With that as a given, if your app needs all available memory, run top and lsmod to see what's using your memory and remove everything you don't need (usually by deleting the links to those processes in the /etc/???/rc5.d directory).

If you can't remove it, scale it down. For example /etc/inittab lists off the different virtual terminals that appear when you press ctrl-alt and a function key. If you never use this feature, try reducing this down to 1 or 2 terminals. Leave some behind just in case you need them later. To do this, just comment the higher numbered lines that look like this;

1. 6:2345:respawn:/sbin/mingetty tty6

(NOTE: Removing these lines might not make any difference -- it all depends on the distribution.)

As for X (assuming you need it and are using XFree), try removing any Load lines in the modules section that you don't need and scaling down the display size, background images, and color depth. Another big area of savings is changing the window manager. FVWM usually is installed, and while it is ugly it is also fairly light weight when compaired to KDE, Gnome, and other popular full-featured WMS.

While these steps alone won't eliminate the speed problems -- the other comments might solve that -- the time you spend waiting might be cut way down.