Thoughts on the 2.4.10+ VM
by
Shane
·
· Score: 5, Interesting
My personal feelings on the new VM is that it was the right decision. The VM problems have been going on for months. When people would report a problem, Rik would pretty much say: I don't have time to work on so and so.. feel free to pay me or convince my employeer to fund the work. Which is fine, that is his choice... But if I was Linus this would make me more open to looking at alternitive approches even if the short term risks were moderate.
It is also interesting to note that Rik's vm core has had say 15 kernel releases (unstable + stable) to become stable and meet up to the expectations that Rik sold the kernel hackers on in the first place and judging from the reports on LK it is just now becoming stable enough for most work loads.
The new 2.2.10+ VM had a couple minor to moderate problems for _SOME_ work loads but over all has received very good reports as far as I can tell for being so new. 2.4.11 is bound to be even better.
Some people are complaining about the inclusion of major VM modifcations in the stable tree. I believe the truth is that most people that use Linux in production do not roll their own kernels. They use the vendor supplied kernels.
Redhat for example will be releasing a 2.2.7-11-AC kernel which uses Rik's VM, it is what they have been testing for months and thus is what they will end up shipping. So the fact that Linus made this change in the "Stable" tree makes very little difference to me from a stability stand point, and I think it will prove to be a very good call in the short/medium/long run.
Thats my 2 cents anyways.
-- -- You can be a geeklord too:)
Re:VM Changes
by
CraigParticle
·
· Score: 5, Interesting
It shouldn't surprise anyone that 2.4.10 VM performs better than 2.4.9. Even in terms of the "traditional" 2.4 VM from Rik, the Linus and Alan trees deviated starting around kernel 2.4.7. There were numerous complaints about the Linus tree missing important patches, and having contradicting patches applied. It ended up quite a mess, and VM performance reflects this. Alan's tree was much more conservative in this regard.
If you compare 2.4.11 to anything, please compare it to the latest -ac kernels from Alan, where the traditional 2.4 VM is actually working very well. There's NO sense in comparing 2.4.11 to 2.4.9; the VM in 2.4.9 and its kin -- it was just plain broken.
Side note: In Rik's VM, please remember to not just look at swap used as a gauge of whether you're swapping or not. All anonymous pages are mapped to swap, so the space is simply allocated. You can create a huge image in GIMP and lots of swap will be allocated, but without a drop of disk I/O! Use vmstat and look at the 'si' and 'so' columns to see if you're actually writing pages to swap. Or look in/proc/meminfo and subtract "SwapCached" from the amount of swap you think you're using. That's the amount of *written* swap you're using (a better comparison to 2.4.10).
This needs to be made sensible in 2.5, if this VM is to be resurrected.
Andrea's work has cleaned up the handling of inactive pages (which could have been done under the old system), and the new "classzone" approach and VM balancing isn't documented anywhere outside the code itself. In addition, there are very normal loads where it performs badly compared to the -ac tree. Here is a test suite that tests different aspects of aging and swapping, and the results as provided to linux-kernel. 2.4.10 (patched with Andrea's VM tweaks) swapped more pages, took longer, and had to swap more pages back in when the tests completed (i.e. it could have chosen better pages to swap out). It also caused XMMS to skip mp3 playback throughout the tests, whereas -ac didn't.
Nothing's perfect of course; a process that randomly walks through pages performs better in 2.4.10 since it's more streamlined and not trying to be as "intelligent" about page handling. Rik's code could no doubt be improved here.
That's the great thing about open source: let the best idea win! No doubt in 2.5 we'll see these two VM schemes hash it out in much more complete form (i.e. lose the remaining kernel 2.2-isms, maybe add physical page mapping, almost certainly swapfs -- mostly for Rik's scheme; I'm not sure what the next steps for Andrea's VM should be).
Slashdot Mirror
It is also interesting to note that Rik's vm core has had say 15 kernel releases (unstable + stable) to become stable and meet up to the expectations that Rik sold the kernel hackers on in the first place and judging from the reports on LK it is just now becoming stable enough for most work loads.
The new 2.2.10+ VM had a couple minor to moderate problems for _SOME_ work loads but over all has received very good reports as far as I can tell for being so new. 2.4.11 is bound to be even better.
Some people are complaining about the inclusion of major VM modifcations in the stable tree. I believe the truth is that most people that use Linux in production do not roll their own kernels. They use the vendor supplied kernels. Redhat for example will be releasing a 2.2.7-11-AC kernel which uses Rik's VM, it is what they have been testing for months and thus is what they will end up shipping. So the fact that Linus made this change in the "Stable" tree makes very little difference to me from a stability stand point, and I think it will prove to be a very good call in the short/medium/long run.
Thats my 2 cents anyways.
-- You can be a geeklord too
If you compare 2.4.11 to anything, please compare it to the latest -ac kernels from Alan, where the traditional 2.4 VM is actually working very well. There's NO sense in comparing 2.4.11 to 2.4.9; the VM in 2.4.9 and its kin -- it was just plain broken.
Side note: In Rik's VM, please remember to not just look at swap used as a gauge of whether you're swapping or not. All anonymous pages are mapped to swap, so the space is simply allocated. You can create a huge image in GIMP and lots of swap will be allocated, but without a drop of disk I/O! Use vmstat and look at the 'si' and 'so' columns to see if you're actually writing pages to swap. Or look in /proc/meminfo and subtract "SwapCached" from the amount of swap you think you're using. That's the amount of *written* swap you're using (a better comparison to 2.4.10).
This needs to be made sensible in 2.5, if this VM is to be resurrected.
Andrea's work has cleaned up the handling of inactive pages (which could have been done under the old system), and the new "classzone" approach and VM balancing isn't documented anywhere outside the code itself. In addition, there are very normal loads where it performs badly compared to the -ac tree. Here is a test suite that tests different aspects of aging and swapping, and the results as provided to linux-kernel. 2.4.10 (patched with Andrea's VM tweaks) swapped more pages, took longer, and had to swap more pages back in when the tests completed (i.e. it could have chosen better pages to swap out). It also caused XMMS to skip mp3 playback throughout the tests, whereas -ac didn't.
Nothing's perfect of course; a process that randomly walks through pages performs better in 2.4.10 since it's more streamlined and not trying to be as "intelligent" about page handling. Rik's code could no doubt be improved here.
That's the great thing about open source: let the best idea win! No doubt in 2.5 we'll see these two VM schemes hash it out in much more complete form (i.e. lose the remaining kernel 2.2-isms, maybe add physical page mapping, almost certainly swapfs -- mostly for Rik's scheme; I'm not sure what the next steps for Andrea's VM should be).