Slashdot Mirror
Rik van Riel on Kernels, VMs, and Linux
Andrea Scrimieri writes "
An Interesting
interview with Rik van Riel, the kernel developer, in which he talks
about the Linux's VM, particurarly about his own implementation (which was
recently adopted in Alan Cox's tree). With some controversy towards Linus
Torvalds.
"
I'm not so sure I agree with him -- if you want to make a dent in the market shares of Solaris and NT/2000/XP you have to keep up with their innovations (Async-I/O, better SMP, etc.). As a user of Linux as our OS of choice for our database and web servers I am feeling a lot of pressure to switch to Solaris because of their better handling of higher-load environments (OLTP databases, web servers, etc.). If Solaris wasn't so damn expensive we'd probably be using SunFire 280's. So I'm pleading to keep up with the big dogs so that I can be reassured that Linux has what it takes (it's handling things fine now but as he said in the article, everyone needs more RAM, CPU, etc.).
Thanks,
--
Matt
I think it was an excellent decision of Linus to remove Rik's VM from the mainline kernel. If not for technical reasons then for political reasons.
Rik's VM obviously needed to be fixed and/or tuned, but apparently lacked the necessary attention from Rik. If Linus had not removed the VM, it would probably have been the situation for a while. Instead we now have TWO VM's which are rather stable and Rik working full speed to make his VM the best.
Competition is good! Which VM will be the best for the future will be determined by Survival Of The Fittest(tm)
It can be argued though, that it was not the right time during 2.4, but Andreas VM seemed to stabilise rather quick with the high level of attention to the problem. Sometimes it takes drastic measures to get results...
I think this kind of infighting is great, as long as it doesn't get out of control.
We get to see arguments and competing subsections of the kernel - this is SO one of the most underrated benefits of open source. Users of some other OS's don't have this benefit. I am not a programmer, so to me, I don't really understand/care the benefits of different VM systems, but I know that some other, smarter people do, and they're all trying to figure out the best way to do it, and that's good enough for me.
I say let them go at it, let the best code win - it can only help us. And who cares how MS construes this, I'd like to see them open up their development model and see what kind of conflicts they are having.
OT - but kerneltrap.org has a good interview with Alan Cox today....
this kind of bashing between the high priests of Linux is not good
/. Linus is not an all knowing genius. He makes mistakes. Perhaps this is one of those mistakes. The real question is whether the benefits of the stewardship he provides compensates for the hindrances his authoritarianism creates.
Sure it is. How else are we going to find out where our disagreements are and work through them? Or, at the very least, learn not to make the same mistakes in future projects. The problem of the Linus bottleneck has been known for a long time. This "bashing" is not new, it's just current.
Having Linus Torvalds around helps insure that, for the average user, there is no splintering of development effort -- just use the Linus kernel. But it also severely hinders improvement because you are limited to what Linus likes or dislikes.
And despite what may be the common conception on
I have a lot of respect for Rik van Riel, but I think that Linus made a good decision to "cut bait" on his VM implementation for 2.4.
It was not that Rik's ideas were bad, it was just that their complexity and implementation were going to take too long - they should have been hashed out in 2.3 instead of 2.4.10.
I'm looking forward to having Rik prove his reverse mapping technology implementation in 2.5.
May the best ideas ultimately win, and may the giants of the kernel not take offense at each other. It would be a real shame if something stupid like Linus' lossy source code control system put off Rik so much the Linux community at large lost his wonderful contributions.
Here's to hoping that Linus gets more sensitive in some cases, and that Rik gets less sensitive in some cases.
"Provided by the management for your protection."
I strongly feel that honesty wins in the end, because people aren't stupid. No one believes that IBM or Microsoft is one happy camp singing "we are the world."
It's great there is a lot of attention on the VM and intense effort to make it better. I have no doubt linux and Rik are professionals and have no problems putting politics aside to get the job done. That is after all part of being a professional. Rik makes some good argument and given enough time and money he'll build the VM of his design. Will it matter 10 years from now? Most likely not. Development will continue and linux will get better. Butting heads is part of the fun, because without conflict people tend to stagnate.
I've been wondering that myself. It's just work to do but it shouldn't be rocket science. Of course a number of people are concerned about the code size as is, you can't just add a new branch of code every time there is a conflict and pack them all together. This does seem like a somewhat good case for it.
BTW, this kind of bashing between the high priests of Linux is not good. You can bet your bottom dollar that MS is going to use this conflict to fuel their propaganda machine, saying Linux is a fractious OS run by a bunch of young upstarts who can't agree on anything.
No this is a good thing. Most of the linux kernel hackers have egos the size of small countries, and that's a good thing because they take pride in their work. Most of them also work as professionally and egolessly as I have ever seen. They can get in fights and then deal with it and keep working. This is far better than the closed world where people get in fights and take it personally and then try to react in some way. I've seen project where people were trying to fail the project to get revenge on someone on the team or managment for something stupid they did in the past. In the linux world people get in fights and everyone can see and they react accordingly, sometimes being told that you're being an ass is a good thing when you're being an ass, sometimes having people stop talking to you for a few days is a good thing, and sometime people appologize and that's a good thing. It's not behind closed doors though and it's hard to undermine things when it's all in the open. There is also some insanely good discussion on certain things some times. There is also something to be said for defending ideas and the strength they have when you can defend them in public.
An honest environment -- such as fostered by "free" software -- is both good and bad. On one hand, I (as a programmer) am comforted to read the kernel mailing list and other resources that let me know exactly what is happening with my tools. I don't need to wonder what's happening with "free" software -- and this is more comforting to an engineer like myself than is the closed-door, silence-is-golden, hide-the-bugs policy of a Microsoft.
On the other hand: Show this interview to an MIS manager who need 24/7/365 reliability, and she is going to be very nervous about deploying a Linux-based solution. You can talk until you're blue in the face about reliable distros and the open road to sofwtare quality -- what the MIS/corporate person sees is chaos and feels a lack of COMFORT .
"Out of sight, out of mind" is a philosophy many people adhere to, especially when dealing with complex issues they can not or do not want to grasp. From waste storage in Nevada to the the war in Afghanistan, most people lack the time and initiative to understand what is really happening; they go on appearances and marketing, and ignore complex and disturbing facts.
Technology is no different. The MIS manager doesn't want to hear about VM conflicts or file system bugs or different kernels -- such issues are beyond their capability and desire to understand. Buying Microsoft is (or was, until recently) comforting, because no one ever saw the internal debates and code battles and what-not that any development team expresses. Even recent security disclosures about WinXP are unlikely to shake the faithful -- but those same people will run in fear from the blunt honesty of Linux.
Ignorance may be bliss, but it can also get you killed. I know people whose lives depend on cars, but they have no knowledge of how to check the oil. Most MIS managers simply want to drive software; if it looks good (like a Jeep Liberty), they don't pay attention to whether it is safe (the Liberty performs poorly on crash tests).
I doubt, however, we're going to change human nature -- and I'd rather have spirited debate and even some nasty contention if it means that people are striving to make Linux the "best" it can be.
All about me
It started out with Rik's VM in the kernel, since it was a promising new development. However, once it was in Linus's kernel, the fact that Rik's development style was not compatible with Linus's source control style because an issue, because the VM wasn't getting updated in Linus's tree.
So Linus switches to the other VM, which is based more on the original. This means that Rik can do his development without dealing with Linus and the Linus tree can have an up-to-date VM. When Rik's is to the point where he's really happy with it and he doesn't think he'll have to make a lot of patches (and it does all the things he wants), it will probably go back in.
Since then, Rik and Linus have figured out (hopefully) how their interaction failed to work, and what Rik has to say along with his patches to make Linus know they're worth looking at. It turns out that it is possible to automate this process, such that a script will send the patches when appropriate, with the right assurances of freshness (having actually tested them, of course).
Linus wants to be able to ignore any patch that isn't for the part he's thinking about at the time (e.g., non-block-i/o patches around the beginning of 2.5). When it becomes interesting again, however, the original patch may not be right any more. Having not looked at the patch at the time when it was sent, Linus can't determine whether it is still good, since the author may have found bugs, and he doesn't know exactly what the patch was supposed to do. He wants the author to make any updates needed and resend it. It may be, of course, that the patch doesn't need to be changed, and the author doesn't have a new and better patch, but Linus can't tell unless the author sends it again with a note that it's still good.
So Rik's patchbot will test whether the patch still applies and still works, and has not been replaced by a new version, and then will send it again until Linus actually looks at it. This seems to me like a good plan, since it doesn't require Linus to test everyone's old patches and have a complicated mail system. And Linus won't accidentally apply the wrong version of a patch or be unable to find a patch.
First off, you need to study your own subject line: OOM Killer. OOM means out of memory. It does not mean low memory; it does not mean "maybe the admin can link in some more swap;" and it does not mean "move pages into a protected buffer." It means out of memory: there is no memory left. Anywhere.
You say:
then you say:
Make up your mind dude -- which side are you on? Hint: Your second statement is correct, sarcastic as it was. We can't know the true memory behavior of a process ahead of time, so we can't block processes that are going to become too large.
Next you say:
Those pages contain data, they cannot be taken away. The must be moved somewhere
Ah, your fingers are moving, but you don't understand the words. This is still swap space; the label is meaningless. And if we have swap available, then we're not out of memory, are we? This is a low memory condition, and is irrelevant to this discussion.
Okay, so how about:
Once again, you use the term without understanding it. This is called memory (as you said,) and if there's some left, then we're not out of it, are we? Once again, this is a low memory condition, and is irrelevant to this discussion.
This next statment needs to be pulled apart, as you try to make two points with it:
Agreed, but we're not talking about "extreme VM load", are we? We're talking about out of memory conditions. No memory left. Anywhere.Sure, but these systems have been hand-tuned to avoid running out of memory in the first place. Is this a good thing? Let's see:
Oddly, you first said "must die," but now say "no need," but I'm willing to ignore that. Ideally, a system should never run out of memory. But how can you know your machine is safe? After all, you can never know the memory requirements of a program ahead of time. So you can either throw excessive amounts of resources at your machine, or you can add support for low-memory and out-of-memory conditions. Your ideas for low-memory problems may be good, but they're not relevant here.
So the only question is this: Is an OOM killer worth the effort? OOM killer performs a partial system shutdown, allowing reletively quick recovery. This might be valuable for servers, but for single-user computers, it's usually easier to just reboot, especially since the machine will be desperately thrashing by that point.
Is an OOM killer worth writing? We're not paying Rik, so that's up to him. If you want low-memory conditions handled better, pay up, or write it yourself.
Look in the mirror, dude -- that's you!
-B
Not that this wasn't entirely predictable.