Linux 2.2 and 2.4 VM Systems Compared

Derek Glidden writes "I got sick of trying to figure out from other people's reports whether or not the 2.4 kernel VM system was broken or not, so I decided to run my own tests, write them up and post them online. The short conclusion is that the 2.4 VM rocks when compared with 2.2, but there's more to it than just that."

Comments

Somebody help me out here

[rho]

Quite often I get the feeling that Linux and BSD are doing quite a bit of "me-too"-isms in an attempt to catch up with the mainstream OSes--including MS, Apple and commercial Unixen.

I read this story and wonder if I should still be getting the same feeling -- isn't a VM subsystem mostly a solved problem? Or am I reading this wrong, and this is merely tweaking and specialization?

Since I'm no Alan Cox (I'm closer to Alan Thicke), I can't see the truth of the matter, but I get the feeling that we're doing a lot of walking in a tight circle on the path, while others have already left the forest.

Re:Somebody help me out here

[DataPath]

It's hard to say if the VM subsystem has been completely reworked in the MS operating systems. It's all closed source. But I think it's a fair guess that NT and 9x had completely different VM subsystems, and in addition, that the Win2k VM subsystem is likely a complete rework of the NT 4 VM. I guess it's something that happens from time to time... someone thinks of a different way of doing things that changes how everything works together, and it makes something faster, something slower.

Re:Somebody help me out here

[Ami Ganguli]

Another factor is probably the tremendous range of hardware and workloads that Linux tries to handle. I don't think any other OS attempts to work well on watches and mainframes (and everything inbetween) while using the same code base.

Re:Somebody help me out here

[iabervon]

There's really no way to tell if the Windows VM is bad and is just being left as is, or is being changed, or is good, because you obviously can't run some other VM under the same load (even if you switch between Windows versions, it's a different load because you've got different system implementations and such).

The design of a VM system also depends a lot on the rest of the system: the best VM is one that always has a page swapped in when you need it, and always has it in an acceptable part of memory. But which page is going to be needed next depends on what sorts of programs you're running, and tons of other factors. The VM system is trying to guess these, and there are some known heuristics for guessing, but there's no right solution for VMs in general.

Apple has had a very different scheduling algorithm, which makes the problem totally different, and much easier: the applications not in front can be swapped out.

I believe that, for a commercial UNIX, if you need swap, then you didn't put in enough RAM. If you could buy the system in the first place, you can afford more RAM. If the OS doesn't support enough RAM, get a version that does.

Re:Somebody help me out here

[rho]

Also, the VM underlies a host of other bits of the OS, and as they change, so the VM has to change to accomodate them - for example, Linux's zero-copy unix domain sockets, or Linux's VFS layer. In short, no, VM design is not 100% solved.

That's interesting. I'm operating on my simplistic, naive notion that a VM is "the hard drive, where you dump pages when you're short on RAM or they get really stale". Thus, in my simple little world, the VM subsystem is affected the most by tweaks to the scheduler that swaps out pages. Is that where the major differences between the two VM schemes lie?

If so, wouldn't it be a worthwhile effort to modularize that part out? (suddenly I see kernel hackers turning white as a sheet, gripping their chair arms in a fit of white-knuckled fear and loathing)

I'm just a twink asking dumb questions...

This is all well and good

[duffbeer703]

But the fact remains is that this VM holy war should have been resolved in the 2.3 series of kernels.

The number of major problems and architectural changes that are being made to the supposedly 'stable' branch of Linux kernel is really run amok.

I'm sure there's plenty of outrages to come as bad bugs are found in the volume manager and other new elements of 2.4

Re:Did someone think otherwise?

[MentlFlos]

I have two dual boxes with 1g each in them and they seem to take whatever I throw at them. I have yet to have any panics or what-not. (So you know what kind of loads I'm talking about, one box runs 3 CS servers with 3 HLTV proxies and the other runs mysql, apache (slash), hlstats for the servers and is my desktop box. Both run DNS and RC5 and have 2.4.10 or greater). Nothing too bad, but not idle either). I have no complaints about the kernels yet.

Now getting ext3 in the tree is something I would like :)

Re:Did someone think otherwise?

[Anonymous Coward]

hmm..i have hard lockups like the guy described with 2.2.19 on a single CPU 733MHz P-III system with 256MB of RAM and 2 Gigs of swap. im running vmware with 5 OSes running simultaneously under high loads and it typically locks up after 3-4 weeks.

The Unreal Tournament test

[Miles]

Just a possibly interesting data point. I played Unreal Tournament with 2.4.12-ac5 and 2.4.10 (both from Debian). 2.4.10 always seems to work fine for extended periods of Lan play (as both a client and server), whereas the 2.4.12-ac5 choked after a few games--the swap ended up being nearly all used up.
Of course, this was hardly a scientific test, but I think I'll stick to something proven for now.

YMMVGV

[Tailhook]

Your observations run counter to the continuous stream of reports of high latency in 2.4 found in the linux kernel mailing list. Specifically, skipping mp3 playing is the canonical report. 2.2 is often cited as being the less latent of the two.

I don't claim you're wrong. I point this out only to illustrate the subjectivity and lack of real data involved in these anecdotal reports. At the very least, the author has attempted to produce hard data on the matter.

Linus obviously thought poorly enough to of original 2.4 VM to space the mess.

Check out Linux Weekly News and Kernel Traffic

[cpeterso]

(Yes, I spend an hour a day reading the kernel mailing list.)

I'm too lazy to read LKML, but I am interested in the happenings of the Linux kernel development. I highly recommend Linux Weekly News' kernel news (updated every Thursday) and Kernel Traffic , an in depth summary of the week's LKML happenings (usually updated every Sunday or Monday).

BIOS settings

[Anonymous Coward]

One thing that is almost impossible to account for in these sorts of "benchmarks" is whether or not the BIOS chipset parameters are properly set. In particular, such settings as RAS, CAS, cache timing, and so on can be critical to stability. I have seen self-appointed know-it-alls post web sites on how to "tweak" the BIOS for ulitmate performance. Usually this involves setting system timing on the very edge with no margin for changes in load and environmental conditions. And usually these so-called "gurus" are not trying to achieve a stable system; they are trying to get the best FPS running Quake or whatnot.

I remember having problems with the 2.2 Linux kernel on occasion. Eventually after much trial and error I went into the BIOS and adjusted the system timing to be a bit slower, allowing system control lines more time to stabalize. My problems went away. Adjusting the BIOS is a black art, and it is often beyond even the best engineers, given that the sparse documentation is usually written in some unfathonable Asian "dialect" of English.

While this comparison of VM systems may be somewhat useful, I have yet to see a study done on a statistically significant sample. I would like to see several hundred different machines compared and have those results analyized. Only then do I believe that truly meaningful conclusions can be reached.

Context for why Linux switched VM Code?

[bteeter]

I thought it was interesting how the author discussed the mid-release swap out of VM code in the 2.4 series kernel. He mentioned that Linus had felt that the AA version of the code was better than the existing version and wholesale swapped it out.

Does anyone know why Linus did this? Are there some empirical results somewhere that dictate a reason to do this? Certainly, it would seem that there would be, but the author didn't point it out.

Also the author pointed out that 2.2.x kernel's break at very high load levels. Is there documentation somewhere discussing what that might be?

Take care,

Brian
--
100% Linux Web Hosting
--

Linux needs professionalism in release management

[Sara Chan]

I have some concern about how all these changes appear to businesses. Linux is supposed to be a high-quality alternative to other operating systems. Yet we've recently had a production kernel that failed to even compile, and there have been major upheavals to the "stable series" VMM, which sometime degrade reliability or performance. This isn't going to impress, and it gives competitors valid ammunition.

Almost all successful enterprises have to overcome a hurdle: how to transition from the "just for fun" start-up stage to the "managed with professionalism" stage. Ideally, fun is kept along with the professionalism. Note that even at Microsoft, B. Gates now lets S. Balmar (CEO) and R. Belluzzo (President and COO) manage things. Gates is the "chief software architect" and Microsoft is still his, but others do the managing.

Linux needs more professionalism in the management of releases, I believe.