Debate on Linux Virtual Memory Handling

xturnip sent us a good piece running over at Byte about Linux's VM. Somewhat more technical then the stuff we usually see online, this one talks about different VM systems, and the egos in the kernel. Its worth a read.

Comments

His favorite?

[LinuxGeek8]

He seems to think a lot in favor of the Andrea VM.
That's ok to me, but he might want to take notice of the fact that linus didn't accept Rik's patches a lot and that 2.4.9 still had actually the VM of 2.4.5. The -ac tree was more up to date.
So for a good comparison you'll need to compare the linus and the ac tree.

Re:His favorite?

[Alrocket]

He seemed to make quite a deal of the fact that the guy wrote it one session (and I am in no way suggestion that's not a major feat), and also having sat down with him for a couple of beers to discuss it also helps colour the tone of the article.

I'd like to see what the author (Moshe)'s opinion was after talking to Rik for the same period.

Al.

Re:His favorite?

[xphase]

It seems Moshe has spoken with Rik about the VM many times:

Rik van Riel and I have variously discussed another approach, called "reverse mapping," which implements a reverse-lookup between the page and process table. Once you have reverse-mapped pages, the VM can simply scan the pages for the ones to be freed. Naturally, some extra fields need to be added to the appropriate control tables to allow this reverse mapping. My own implementation has an overhead of 14 bytes and is therefore certainly a lesser solution than Rik's - his overhead is just 8 bytes.

I'm aware that this doesn't mean they've met in person, but it shows that Moshe has discussed things with Rik before AA's VM was written. So I think he holds nothing agains Rik, he just likes aa's VM better

--xPhase

Re:His favorite?

[mr3038]

I'm aware that this doesn't mean they've met in person, but it shows that Moshe has discussed things with Rik before AA's VM was written. So I think he holds nothing agains Rik, he just likes aa's VM better.

In addition to this it seems that he has implemented VM with reverse mapping also. Therefore it should be clear that he previously thought this was the best method. I've understood that the issue between Rik's and AA's VMs is that Rik's is optimized for normal swapping and AA's for OOM case. Because VM performance really matters only when OOM happens I think AA's should be superior. The real difference depends on benchmark, of course.

Both systems seem to be somewhat equal. AA's needs less swap but Rik's is claimed to be better performer. If AA's system is simpler then that's what should be used. Select maintainability over questionable performance increase. This is like quicksort - there's a point when you usually get better performance bubble sorting the little pieces quicksort generates during the whole sort. The smart version isn't always the best. Nowadays CPUs can easily do a bunch of dumb operations faster than one smart operation.

Re:His favorite?

[Srin Tuar]

Har-

For a quicksort of large objects I use an insertion sort for the compares, and a form of selection to do the moving of the large objects. (best advantages of both)

Even an optimized bubblesort (the combsort) is useless compared to a proper template quicksort.

Re:His favorite?

[be-fan]

Were not talking userspace applications here, we're talking about the kernel. Both the Linux and FreeBSD projects go to extreme lengths to optimize the performance of their kernels. The FreeBSD VM, for example, is very complex, but performs well due to that complexity.

Re:His favorite?

[Anonymous Coward]

After reading the byte article, I downloaded the ext3 patch for 2.4.13 and tried out the new linus kernel. It works very well for me.

I have been using the -ac series since 2.4.4. Most recently 2.4.12-ac6. Performance on my 128M PII 266Mhz thinkpad has been very poor. My day-to-day tasks of running KDE and Mozilla were sluggish. Add a background compile and/or xine and the system was worse than sluggish. Yes even with 2.4.12-ac6.

Since switching to 2.4.13(+ext3) I have been very happy. The machine performs noticeably better on my day-to-day tasks. I am not talking "a little" better... but "a lot" better....

Re:His favorite?

[Anonymous Coward]

No, use heapsort on tests:

Heapsort(list L):
H = an empty heap
for each x in L:
insert x in H
M = an empty list
for i = 1 to |L|:
x = remove from H
append x to M
return M

It should all be configurable.

[Anton Anatopopov]

The VM policy of the OS should be tweakable in the same way that you can dynamically load and unload modules. I see nothing controversial about this. Some people prefer random page ejection to lease recently used. Others think that demand paging can be improved upon.

I say, include it all in the kernel and make it configurable by the user. After all, most Linux users are pretty tech-savvy, they are unlikely to wreck their machines (the way windoze lusers do every time they tweak their registry).

What do others think ?

Re:It should all be configurable.

[micromoog]

After all, most Linux users are pretty tech-savvy, they are unlikely to wreck their machines (the way windoze lusers do every time they tweak their registry).

More generalized Linux cock-swinging, as has become the trademark of Slashdot . . . this is the attitude that dooms Linux on the desktop.

Re:It should all be configurable.

[pwagland]

Sadly, no.

While it is nice to be ultra configurable that leads to two seperate problems:
1) Code maintainability
2) User maintainability

1) Is a serious problem. If you have to test the impacts on two different VM systems, and fully understand the impact that any change will have is a mammoth task.

2) Users are not all technically literate anymore. Look at the recent slashdot story on microsoft losing there grip in Asia...

Re:It should all be configurable.

[Anonymous Coward]

This has come up in the kernel mailing list (see the summary at href="http://kt.zork.net/">kernel traffic. The conclusion was that it was simply too hard with the version of kbuild/config included with the 2.4 series.

Re:It should all be configurable.

[Flower]

While the idea is interesting I don't think it is practical. From what I've read on KT and in this article changing the VM forces design considerations on userland programs. It's additional complexity that most developers (and especially companies like Oracle) wouldn't appreciate. I also think it would raise support costs. At the very least I'd want some variable in /etc that would clearly state which VM was being used. For me at least, the issue is simplicity in favor of flexibility

I think the biggest bone of contention in the community is Linus replaced the VM in the current stable version instead of pushing it into 2.5. Again, not being a kernel hacker and only going from everything I've read, this was a radical change. I'd almost be willing to say the latest kernels should be labeled 2.6 but that's just me.

Oh, and finally, to paraphrase an old saying, give any tech-savvy user enough rope and they will hang themselves.

At least that's what I think. :)

Re:It should all be configurable.

[Nothinman]

This was brought up on the LKML already and doesn't seem like a bad idea but would take another major overhaul in the kernel to make it work, something like this may happen in 2.5 but I wouldn't bet on it.

Some decisions lately have been questionable, but Rik and Andrea are both just trying to get a VM in 2.4 that works so 2.5 can be opened, which would make a lot of people happy.

Re:It should all be configurable.

[jacobito]

That's not going to happen in the 2.4 series. The kernel hackers think that making the VM policy configurable would be a nightmare:

Michael T. Babcock asked how ugly it would be to make Rik van Riel's and Andrea Arcangeli's Virtual Memory subsystem code into a compile-time option, so folks could try each one out as they pleased. Alan Cox replied simply, "Too ugly for words." Mike Fedyk suggested that it might be feasible in 2.5, and asked if there were a way to make it non-ugly. Marcelo Tosatti replied, "Even if its non-ugly, its non-easy. Way too much overhead. For 2.5 we'll probably be able to get people working together."

This is from Kernel Traffic #139.

Re:It should all be configurable.

[volsung]

What makes you think anyone doing the cock-swinging actually has any impact on the development of Linux? :)

Re:It should all be configurable.

[Anton Anatopopov]

The design of a userspace program should not matter on what virtual memory system is in use.

Sure locality of reference matters, but any decent VM design will take this into account.

What I would like to see would be per-process VM algorithms. Like, you give an extra argument to the fork system call, and your new process has its virtual memory managed in a way which is optimal for your application.

Stack based languages exhibit different behaviour to certain other types of languages, and most VM systems seem to be optimised for this general case.

Re:It should all be configurable.

[DarkMan]

Um, I think all the replies here I've read have missed the point. I don't think the poster was asking to be able to switch between the two VM's at complie time, but rather having one VM that was configurable.

That would allow the system to be tuned at compile time for the large servers, and for the small desktops, without haveing to have a 'one size fit's all' solution.

I've always felt that that would be the best answear. The reality is, however, that Andrea's VM would not allow for such a range of configurability, being a very simple, and thus easy to balance, system. That's not to put it down, often the simplest solution is best.

However, Rick's VM is more complex, and can, in principle be made more configurable at compile (or even run) time. It would be a lot of work, but I think that that's the best way to get good performance across the wide range of platforms.

For example, If I knew that my system would have to work with millions of very small files, and only read them once, then I would configure the VM to forget about caching the files, and keep anything that is used more than once in RAM. Or, of dealing with a computation, have large pages RAM to be swapped in or out that match with the arrays the computation uses, so that everything is pre-fetched. Yes, there are other ways of accomplishing these goals, but I think that that would be a good way to go.

If nothing else, it acknowledges that a system with 32 Meg of ram and one processor has a very different VM needs from an Octuple processor system with 32 Gig of ram.

Re:It should all be configurable.

[tonywong]

Mod me down for nitpicking, but when you mention the word literate in your comment (albeit in a technical sense), your following sentence mistakes 'there' for 'their'.

I just found it ironically amusing.

Re:It should all be configurable.

[borgboy]

I think that I do have some measure of control over memory management in my machine, even though it doesn't crash when I tweak the registry. After all, most Windows 2000 power users are pretty tech-savvy.

Re:It should all be configurable.

[Anonymous Coward]

I'm rather sure the people doing the development are too busy shoving their cocks in each others' holes to do any "cock-swinging" .....

On a similar note, I heard that Rik loves it up the ass from Alan. No swinging there, just bend and thrust.

Re:It should all be configurable.

[Anonymous Coward]

Nahhh. They're suffer from delusions of being tech savvy.

Re:...warm bawls

[Anonymous Coward]

this one goes out to all the djs

TROLL TUESDAY - TROLLIN FOR JESUS!

(last post)

To fork, or not to fork

[imrdkl]

From the article:

Nobody has yet dared to speak of a Linux source fork, but this is dangerously close to one.

Is this truly dangerous? If so, why? Why not let the 2 VM's compete and the users will decide?

Better to split than stagnate.

Re:To fork, or not to fork

[scooby-doo]

Essentially that is already going on. The Andrea VM is in Linus's tree now and Rik's VM is still in Alan Cox's tree. So by choosing the official Linus kernel or the -ac kernel you can choose which VM subsystem you would rather use.

Re:To fork, or not to fork

[Zocalo]

It's not too clearly written, but I read that as "no one has brought the subject up, but it may be just around the corner", rather than "it may be hazardous to Linux's health". Source forks do seem to have this "whoo... bad stuff happening" stigma attached to them for many people though.

It's all part of giving people the choice and letting the masses decide, as you advocate, although even for the tech-savvy Linux crowd I think this is going to a small mass. It's not like it's going to cause most users sleepless nights, is it?

Re:To fork, or not to fork

[ZerothAngel]

Is it truly better? I would think forking would lead to stagnation.

As it is, the Linus/AC trees are effectively forks because of the different VM codebases. Any new drivers, any new code would have to be ported from one tree to the other (assuming you want the same features on both trees). Seems like a lot more unnecessary work to me.

Re:To fork, or not to fork

[mwalker]

Better to split than stagnate.


True, look at the success of the "Gnome vs. KDE" split.

Re:To fork, or not to fork

[sshore]

Why not let the 2 VM's compete and the users will decide?

The problem is the duplication of effort and decreased manpower for each VM. Not only that, but any project that works closely with the VM has to test under twice as many conditions, and may require different code for each. Talk about a maintenance problem.

It's certainly good to have competition to bring out the best in each system, but it would be horribly inefficient to keep it going in the long run.

Regarding the users choosing - the users don't have the opportunity to choose only on the basis of the VM. It's not like they can apply the "VM patch" to the stock kernel to try out the other one, rather, they have to apply a fairly large -ac patch that changes a lot of unrelated things.

Re:To fork, or not to fork

[ethereal]

Well, drivers eventually do get from the -ac tree into the Linus tree, you know - the whole point is that AC tries them out until they are stable enough for Linus. Not to mention that Mr. Cox does have some responsibility to provide RedHat with the best kernel he can, no matter what Linus thinks of it. The only weird thing here is that as far as the VM goes, Linus has picked up the more experimental code first. So people who always recompile the Linus kernel when they install a new distro may find that their kernel operates very differently after that.

My naive thought is that the best way to do it would be to somehow modularize the two VMs so that it can be a compile-time or boot-time option, and let users try both on the same box to see which is better. However, I imagine this would be a ton of work to set up.

Re:To fork, or not to fork

[Josh Mast]

Split? They were never even the same project to begin with, IIRC.

Re:To fork, or not to fork

[Flower]

A better analogy would be look at the success of emacs v. xemacs. iirc, that was a true fork.

Re:To fork, or not to fork

[battjt]

Look at the success of EGCS and GCC. That was a successful split and merge. It led to a better GCC in the end while supporting both stable and advance versions of gcc in the interim.

Joe

Re:To fork, or not to fork

[Salamander]

Why not let the 2 VM's compete and the users will decide?

There was an interesting thread on this a while back, rooted at this comment. Unfortunately, the article's old enough that it's only available as a static page, and the oh-so-wonderful Slashdot code that generated the page seems to've done so with the comments in basically random order, so it's almost impossible to follow the thread. Maybe I'll try to recreate its original structure and put the result on my website.

In brief, I think it's a great idea. Competition is good; let individuals and teams compete on the basis of the quality of their work, and bless the "winner" as part of the official tree. The "loser" is always free to try again in the next round. The only problem is that this all should have occurred in the 2.3.xx and/or 2.5.xx series; 2.4.xx should not be changing horses in midstream.

Re:To fork, or not to fork

[einstein]

My naive thought is that the best way to do it would be to somehow modularize the two VMs so that it can be a compile-time or boot-time option, and let users try both on the same box to see which is better. However, I imagine this would be a ton of work to set up.

this was discussed on the kernel mailing, (check out http://kt.zork.net). The general conclusion was that this would be really had to do with the current build/module system, but kbuild 2.5 has the ability to apply patches before building as long as the patches don't overlap.
---

Re:To fork, or not to fork

[Speare]

The problem is the duplication of effort and decreased manpower for each VM. Not only that, but any project that works closely with the VM has to test under twice as many conditions, and may require different code for each. Talk about a maintenance problem.

And this would somehow not be the problem with a fork? Considering Linux vs *BSD is already a division of the pool of possibly alignable geeks, and considering both Linux and *BSD families continue to grow, innovate and expand, I think the problem is overrated.

Organizations align on common goals and pursuits, by definition. If there were two or more unalignable goals in the VM, then either a fork or an unforked competition would be in order, and would have the same issues of reduced effort and increased maintenance chores.

Personally, as a non-kernel developer, I think the different VM issues are probably overblown in the moment, and that the best approaches will forge ahead with some significant consensus in the mid-term. Until then, it's worth the experimentation it takes to decide what are the best approaches.

Re:To fork, or not to fork

[Anonymous Coward]

Too bad gcc still sucks ass. I went from using Apple's MrC on Mac OS 9 to using gcc on Mac OS X, and I'm stunned at the difference. Compile times are longer, binaries are bigger, memory usage is up, and my programs run slower. It's worse on all fronts! Now I know I'll just get modded down for this, because how dare anyone criticize an open-source project; such things are beyond criticism (at least around here). But the facts speak for themselves. I've gotten much poorer results with gcc than with just about any other compiler I've used.

Re:To fork, or not to fork

[jonathan_ingram]

True, but initally Miguel was working on KDE, and only agitated to create GNOME once he realised that they were going to stick with QT.

Re:To fork, or not to fork

[Lars T.]

It says so in the article. It also says:

Quite often, latest patches to drivers and new features are only in Alan Cox's tree. Those who want to go with the official Linux source code may find themselves unable to apply the patches due to the different VM code all over.

So we have a fork with two branches of the Linux kernel, with different VM systems, but also with other features different. In the end the decission between the two may not be made on the VM system at all, but on the other features like a missing driver.

Re:To fork, or not to fork

[Mr. Fred Smoothie]

MrC was heavily optimized for PPC by people who spent years working on it and not having to worry about any other architecture (except maybe m68k).

GCC is a large compiler projects w/ frontends for many languages and backends for an obscene # of platforms.

Over the years, both the frontends and backends of GCC have steadily improved. Just as Intel has contributed code or expertise for the x86 backend, certainly anyone who's worked on Apple's compilers is welcome to send patches to the GCC maintainers to improve PPC performance (though whether or not Apple lawyers would have something to say about that might complicate the issue).

And hey; if you feel so passionate about it, why don't you code up some test programs, compare the assembler output, and mail the code & generated results to the GCC team so that they know where improvement is needed: oh, but wait; it *is* much easier just to bitch on slashdot, isn't it?

On second thought, damn why can't all these volunteer programmers do a better job of making my life easier without any help from me...

Re:To fork, or not to fork

[jovlinger]

'cause I have no intuition at all about these things:

Roughly how much of the kernel/drivers/whatever that constitute the source tree we call the kernel has to know about virtual memory implementation? I would have thought that even the most low-level driver would interact with the VM system at the API level, so it wouldn't matter which VM implementation manages the pages as all the driver cares about is that this page is locked in physical RAM, while those pages can be moved as the VM sees fit.

eh?

Re:To fork, or not to fork

[Nater]

Not only that, but GCC has to dodge patents left and right. There are lots of optimizations that are verboten from the project for that reason, and will probably be included when the patents expire. In the meantime, the GCC crew is left with the incredibly consuming process of developing non-infringing optimizations that simply aren't allowed to be as good as they could be.

Re:To fork, or not to fork

[Jerry]

Well, drivers eventually do get from the -ac tree into the Linus tree, you know - the whole point is that AC tries them out until they are stable enough for Linus. Not to mention that Mr. Cox does have some responsibility to provide RedHat with the best kernel he can, no matter what Linus thinks of it. The only weird thing here is that as far as the VM goes, Linus has picked up the more experimental code first. So people who always recompile the Linus kernel when they install a new distro may find that their kernel operates very differently after that.

So Alan is generating test bed kernels, with even numbers, which RH is putting into their distros?

Not a good practice.

Re:To fork, or not to fork

[Ed Avis]

Suppose Linux did fork...

If Linus's version is called linux, what is Alan's version called?

Re:To fork, or not to fork

[(startx)]

alux? unial? unicox?

Re:To fork, or not to fork

[Tumbleweed]

Erm, 'Redhat', probably.

Re:To fork, or not to fork

[ethereal]

I don't know exactly where the line is; perhaps RH customers prefer enhanced driver support at the expense of some testing? I wouldn't call them "test bed" kernels - -ac kernels are tested by many people worldwide, including RH internal testing.

Now if you were to complain about the compilers that RH puts in their distros :)

Re:To fork, or not to fork

[Skapare]

The sad part of this is that now when manufacturers make drivers, they will work with Redhat (because thats the distribution the suits tend to choose ... Redhat is sort of the "Windows" of the open source OS world given its market aims and its somewhat lower-tech orientation) but NOT with Linux.

Re:To fork, or not to fork

[Anonymous Coward]

> True, but initally Miguel was working on KDE, and only agitated to create GNOME once he realised that they were going to stick with QT.

Unless you're arguing that Miguel was forked, the premise is still invalid - KDE and Gnome projects were not derived from each other, nor from a common root.

Re:To fork, or not to fork

[Cro Magnon]

coxux!

Re:To fork, or not to fork

[(startx)]

That's GNU/coxux to you!

Re:The failure of Open Source

[Anonymous Coward]

Oh please. Have you ever worked on a commercial software project? I've seen just as much if not more ego in moronic engineering team meetings at my enterprise software company. Without a single strong technical leader OR a group of smart people who all equally respect each other's opinions, the SAME THING happens on a commercial project. I've watched a Director of Engineering call meetings almost every day for 3 weeks in a row because he didn't know how to solve exactly this sort of problem. In the end he just decided to go with what the person with the most years of experience said and to get the CEO to give him blanket license to make that technical decision, though none of the other engineers agreed with it - they were all too conflict averse to speak up and too afraid about losing there jobs just as the economy was tanking (he made a bad decision indeed and the project suffered greatly for it, getting delayed by 3-4 months and even then never delivering a large portion of the promised features because this architectural decision made them impossible). That company (mine, unfortunately) is most likely going out of business soon. So don't give me this crap that ego only adversely affects Open Source projects.

OSS Power

[MikeD83]

I think this shows the power of open source software. Everyone, thought Rik's system would be great. Unfortunately it was not, so Linus used Arcangeli's new VM code. Problem solved. Stable as ever. This major OS change happened over a couple months I bet Redmond couldn't make that happen if a VM bug was found now in XP.

Re:OSS Power

[BenHmm]

Sure they could - provided all of the users of XP were the sort of people who don't mind downloading and recompiling a new kernel every two weeks.

They're not. So Microsoft put these changes in point releases instead.

Re:OSS Power

[skyhawker]

Sure they could - provided all of the users of XP were the sort of people who don't mind downloading and recompiling a new kernel every two weeks.

Are you saying that the users of Microsoft Windows XP can download and recompile the OS? I'm not familiar with this new licensing arrangement that Microsoft must now provide. I will say this much for Microsoft. They do respond quickly to critical bugs in their software -- often within months, sometimes within weeks.

Re:OSS Power

[Xzzy]

> so Linus used Arcangeli's new VM code. Problem solved. Stable as ever.

This is actually a wad of baloney. In normal applications (ie, running xmms, reading slashdot and maybe running gimp, with your glitzy desktop of choice), sure the VM works fine.

In any SERIOUS situation though, 2.4 simply falls apart crying because the kernal handles memory so badly. One would like to think that in a low memory situation the kernel would start hacking off whatever was causing the problem so that it could survive. Well, it doesn't. It just freezes. This has been a situation I've been forced to deal with over the past month.. so while I'm not a guru on the subject, I have pieced together some bits of the story.

Basically at my job we have a programming group that has mountains and mountains of source that they have to compile. Lazy as programmers tend to be, they also try to compile it over nfs on the machines with the biggest specs. To give a sense of scope, the resulting executable clocks in at around 500 megs. So basically, their build really stresses out the machine they're compiling on.

The machine freezes EVERY time because of memory shortages. The kernel can't allocate pages for incoming network traffic, causing a backlog, causing processes to hang, causing further backlog.. then powie an unresponsive machine. The obvious solution would be to slim down the build but if anyone's ever worked with a developer suggesting that would be as useful as suggesting Hitler was a saint.

From what I've gathered of the story, the 2.4 kernel was supposed to have this new grand VM that made dorking with the freepages file obsolete.. to the point where you can't even tweak the kernel with the freepages file anymore. The kernel was supposed to have this feature that would let it detect what processes were stealing all the memory and kill them off.

NEWS FLASH they took this feature out because it was buggy.

So what happens? The kernel just paints itself into a corner until the machine freezes. Only way to recover is to power cycle. This is why damn near every patch in the 2.4 line has the line "VM tweaks" in the changelog. Quite frankly the 2.4 VM is garbage, and only functions suitably well in non-intensive applications.

It's been getting better with each dot release but it's still nothing you'd want to bet money on.

Re:OSS Power

[ch-chuck]

think this shows the power of open source software.

While there's still big egos involved, there's probably less of the corporate mindset politics, like chosing a persons code because you owe them a favor, play golf together, wear the same ties, or their mom is on the same United Way committe with a potential customer's chairman, other sucking up and brown-nosing issues.

Re:OSS Power

[Anonymous Coward]

Have you even bothered to read the article this thread is about? The VM of Linux up to 2.4.9 was basically a dog in anybodies opinion. No dispute here, because either developemnt fraction thought so because of different reasons. Then came glorious 2.4.10 to change all that. You don't specify if your machine is running a stock 2.4.x (x <= 9), 2.4.*-ac* or 2.4.x (x >= 10). The basic difference between these is:
2.4.x (x <= 9) contains an unfixed (and in some peoples opinion unfixable) VM (Riel-VM).
2.4.*-ac* contains a modified (tweaked) Riel-VM that's supposedly better at handling big loads (please could anyone with such a load elaborate on this, for me this is hearsay).
2.4.x (x >= 10) contains an (in later revisions slightly tweaked) AA-VM that is looking quite good juding by the reviews in the articles and opinions of many people (most notably NOT including Alan Cox, who still believes the Riel-VM is salvagable and that a VM change of this magnitude does not belong into a stable kernel series).

I think there is someone here that needs to jump over his shadow. Both Linus (I too think that changing a VM is better done in an experimental series, and if it just were a short diversionary series to fix that blundering VM) and Alan (I think he should take a look at how many changes the Riel-VM got in his series and if that isn't likely to be considered a completely different VM when compared to a stock 2.4.9) need to settle this to get a 2.5 started and then we can move on to include the newest and latest stuff everyone likes to try...

regards

Re:OSS Power

[Space cowboy]

If you have an executable that clocks in at ~500MB, you have problems indeed, only one of which will be the VM system!

What on earth does it do? Or perhaps the question should be "what doesn't it do?"

Simon.

Re:OSS Power

[Anonymous Coward]

Uh-oh. You have dared to criticize a piece of open source software. You will now be modded into oblivion (by the editors, not the users). You might even be IP-banned.

Re:OSS Power

[Laplace]

~500MB data sets are easy. I'm working on a real time project. Data streams in, I have to do some very specific transformations on it, analyze the transforms, and spit out the results. The amount of data is huge, and it comes in fast. Think real time scientific computing, where every point of data could contain valuable information (well, my software does try to throw away the unimportant stuff, so that part isn't so true).

Re:OSS Power

[Anonymous Coward]

If you have an executable that clocks in at ~500MB, you have problems indeed... What on earth does it do?

Reply: ~500MB data sets are easy.

Ummm....Yeah. I think you need to re-compile your comment; you have a dangling reference.

Re:OSS Power

[Anonymous Coward]

> ~500MB data sets are easy. ...

..and how does this relate to the above question on a 500MB executable? That's a little different from a dataset, yes?

Re:OSS Power

[Anonymous Coward]

This probably sounds like a flame but I have to ask: DID YOU READ THE FUCKING ARTICLE?

What the guy was saying was that, yes, until 2.4.9 VM had unanticipated problems, and then he talked about the fix. And that's what the poster you were replying to said too.

... and some clueless moderators modded you up too...

Re:OSS Power

[Faceprint]

just curious: is it possible the problem you're having is caused by NFS, not by the VM? NFS has had all sorts of trouble on different occasions. You might try and talk your programmers into compiling a local copy, and see if that makes your problem go away.

Re:OSS Power

[Anonymous Coward]

> I will say this much for Microsoft. They do respond quickly to critical bugs in their software -- often within months, sometimes within weeks.

Or in a day: they launched XP on Thursday and had 20MB of those ready for download on Friday :)

Re:OSS Power

[SurfsUp]

So what happens? The kernel just paints itself into a corner until the machine freezes. Only way to recover is to power cycle.

No, there's a task killing option available under the SysRq key now. And you're wrong, the oom_kill is still in there, it hasn't been removed.

FreeBSD 4.4-STABLE vs 2.4 comparison?

[swb]

He alludes to some FreeBSD vs. Linux benchmarks at the end of the peice. Anyone got any links?

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[Alrocket]

I think he meant that they would soon be posted on the Byte website:

The results were very interesting indeed. Since this benchmark is too much to be handled in this article, Byte.com will post it here soon for you to read.

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[ZerothAngel]

The old benchmark is here, but as the poster above noted, the new benchmark is forthcoming.

Although it will be comparing a moving target (Linux 2.4.x) to a moving target (FreeBSD 4.x), the results will be interesting. AFAIK, there weren't any major changes (I mean like VM changes :) in FreeBSD, so comparing the old and new benchmarks would give a good indication on how much Arcangeli's VM improves things.

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[eparusel]

Hmm, recently OpenBSD and FreeBSD (I'm not sure about NetBSD though) have added improved dirpref code (created by an OpenBSD developer(s)).

When data is written with the new algorithm, subsequent reads and writes are on average faster (being conservative). People are seeing 6x improvements for certain tasks as well!

So while there weren't any major changes to the VM in FreeBSD AFAIK as well, if the benchmark involves using any files on the disk, then it'll most likely be sped up...!

Here's a link to the discussion on the FreeBSD-stable mailing list...

and another link...

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[brer_rabbit]

comparing the old and new benchmarks would give a good indication on how much Arcangeli's VM improves things.

it actually won't give a good indication, see my post elsewhere in this article. Basically, in addition to the newer kernel, the author is also throwing in newer versions of mysql, sendmail, apache etc etc and recompiling everything with a newer version of gcc.

and what he going to say after he finds an improvement of X%? The new linux VM rocks? Exactly how much of this was due to the new VM? After taking gcc, mysql and everything else into account...who knows. So if the author is doing the test he described, it's not a benchmark of the old VM versus the new.

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[jonbelson]

I can't see anything in the original article that says that softupdates were turned on for the FreeBSD machine (AIR it's the default for 4.2 onwards).

This should give a significant speedup for tests which perform a lot of file operations.

--Jon

Re:FreeBSD 4.4-STABLE vs 2.4 comparison?

[Anonymous Coward]

dirpref doesn't help read performance, just directory traversal.

It will help on tests that involve manipulating lots of smaller files, but it won't have any measurable effect on tests that are purely block-read bound (ie, database tests).

Make it a build option

[hattig]

Why not make it a build option in the kernel (along with the preemption patches and the like). A single kernel is not going to satisfy every use of an OS - Server, Multimedia, Desktop, Games, etc. VM is one thing that can change OS functionality with regards for this.

So in a Linux Distro install it can ask you "Are you using this computer as a (a) server (b) Multimedia (c) Desktop (d) Games (e) Mixture, and then it can install an appropriate kernel for you with the appropriate VM and preemption patches. A multimedia computer would like the low latency preemption, for example...

The linux kernel has 5 main parts. There is no reason that each part cannot be modularised away in order to get alternative functionality depending on build options and the target environment.

Do you think that Windows 2000 DataCenter has the same VM system as Windows 2000 Professional? I severely doubt it. ANd I bet that MS' in-house kernel build tool will have VM type as a selectable option, as well as many other subsystems.

Re:Make it a build option

[iamsure]

This was actually answered on the list, and summarized in a Kernel Traffic. As Alan Cox put it "It would be horribly difficult".

While it sounds simple enough, as they said in the KT, the "replacement" of the VM was no small feat. It took 170 patches, which touched a very large percentage of the kernel.

Imagine doing so TWICE (or more) and trying to code 'around' the issues for each.

No.

This way madness lies. While it is a nice idea, the simple truth is that it doesnt belong in 2.4.

2.5 should have branched the second that the patches were considered. Linus didnt want to deal with bitching about 2.4 not being "good enough" and was impatient.

So be it. The differences between Linus' and Alan's kernel trees (other than the VM) is growing VERY small this week, and will probably be 'close-enough' for a handoff within the next two weeks.

The only question is which VM will end up in the 2.4 series. (NOT when Linus hands it over, but when Alan begins his releases of it).

I would not be shocked to see Alan disagree with Linus, and stay with the 2.4.x (x10) VM, and I also wouldnt be shocked to see him agree with Linus and use the new VM.

As to the patch on install idea, it is actually also discussed for kbuild in the 2.5 series.

2.5 will be very excited, if we can only get Linus to get working on it, instead of muddying the stable-series water!!

Re:Make it a build option

[sql*kitten]

Do you think that Windows 2000 DataCenter has the same VM system as Windows 2000 Professional? I severely doubt it

It's actually probably the same algorithm, with different parameters. That's how NT4 did it, in Workstation and Server versions. The kernel would note which version it was supposed to be on startup, then initialize the VM system differently.

Re:Make it a build option

[felicity]

I agree with most folks that this should have waited for 2.5, 2.4 should only be bug fixes at this point. That's why it's the *stable* kernel tree. Big huge changes (and replacing the VM system is defintately in this category) are not appropriate here.

I wonder what Rik has to say about the new "blessed" VM? If he thinks it's a better all-around VM, then the debate can stop pretty quickly I would think.

I think it'll be interesting when the handoff occurs. Will people have to deal with different VMs constantly during official releases? 2.4.0-2.4.9, a change in 2.4.10-2.4.13, and a change back for 2.4.14 and beyond?

I also wonder which way major distros will go (since most people don't deviate from those kernels.) RedHat, for instance, usually bases themselves on the AC kernel tree (surprise) and then additionally patch it a whole lot more. While others take the most recent blessed kernel and go with it straight. Should be interesting.

My overall view of this is simple though: Linus is God, in relation to the kernel, until he says otherwise. (to paraphrase Eric Raymond) If someone wants to maintain a patch against the now-blessed VM to revert to the previous behavior, fine. The decision has been made for the new VM though, let's continue on with things shall we?

Re:Make it a build option

[hattig]

Whilst I appreciate that it wouldn't be possible for the 2.4 series kernels, this sounds more like an issue arriving from heavy integration into the kernel. If the VM was truly a modular piece of code, communicating with the rest of the kernel (FS manager, memory manager, etc) via a documented API, then choosing whatever VM to use would basically just be a choice of pointing at the relevant VM code that implements that API in the source code and using that.

I don't know what the performance issues of having it so modularised would be however. I think that as it is a compile time modularisation, not a run time modularisation that it shouldn't be that bad.

It is just getting the kernel into a state where this can be done. The same goes for SCSI, etc, which apparently need a huge rewrite. Maybe for kernel 3.0?

Bear in mind that I don't know much of what I am talking about except what seems to be reasonably sensible to me to do at a high level.

Re:Make it a build option

[Anonymous Coward]

> I agree with most folks that this should have waited for 2.5, 2.4 should only be bug fixes at this point. That's why it's the > *stable* kernel tree If you have ever tried to use a 2.4.x kernel on a server, you would know that you simply can't call it *stable*. The new VM has changed that.

Re:Make it a build option

[rtscts]

Except Datacenter is not Workstation or Server, which run on more-or-less ordinary PC style hardware.

Datacenter is for Teh Uber Boxen.

But who knows, it's MS after all.

Re:Make it a build option

[Anonymous Coward]

muwahahaha

When talking about FreeBSD SMPng, someone raised the notion of "why not make it like options SMPNG in the kernel".

This only generated a single response. "That's not feasible".

Re:Make it a build option

[Anonymous Coward]

Well, since 2000 pro uses a different system than 2000 server/advanced server, I'm probably going to guess otherwise. At the least, datacenter would use the server version instead of pro. The difference between nt4 workstation and server was registry keys while the difference between the 2000s is larger because I think they learned how to use #ifdef when compiling.

Re:Make it a build option

[Rik van Riel]

I wonder what Rik has to say about the new "blessed" VM? If he thinks it's a better all-around VM, then the debate can stop pretty quickly I would think.

Well, since you wanted to know ;)

First let me explain that most of the time in the beginning of 2.4 was spent making the VM stable, stopping it from chrashing on highmem machines, etc... Speed improvements were a secondary thing, to do later on. Secondly, Linus is a very busy man and didn't seem to have the time even to apply critical bugfixes at times, so his kernel has had a big disadvantage over Alan's kernel.

Around the time where the VM in Alan's kernel got stable, I was finally getting the time to work on speed improvements and Linus still lagged a few patches, suddenly Andrea surprised us all by posting the first version of his new VM online. An even bigger surprise was that Linus integrated this into the kernel within 24 hours, without even asking Andrea!

As to why Andrea's VM is faster for desktop use ... it was optimised for speed on low to medium loads in exactly the same way the 2.2 kernel was. Note that this also means the server falls over quicker under high load and it is basically impossible to tune the system to run decently under all loads ... just like 2.2.

My VM was slower for desktop loads, but since the thing stabilised I put in some time to make things faster and I seem to have mostly caught up with Andrea on the speed front now. The benchmark results posted on the linux-kernel mailing list seem to indicate that Andrea's VM is faster for some things, while my VM is faster for some other things.

Personally, I think it is easier to make a solid VM fast than it would be to make a fast VM solid. This opinion was formed because of the living hell of the Linux 2.2 VM, which was undocumented and horribly subtle.

In the future, I know I'll always be optimising for (1) maintainability, (2) correctness/stability and (3) performance, in that order...

Re:Make it a build option

[jovlinger]

ah!

so if I read you correctly, the post-not-really-a-fork-but-almost -ac kernels (2.4.5?) have gotten progressively better at dealing with high memory commitments, and are now back into the regions of usability? (where usability is somewhat arbitrarily defined as being able to have working sets that are very close to physical RAM without having to wait several minutes for top to fire up to tell me which process is the memory hog (pan, if you must know)).

Re:Make it a build option

[xah]

Rik, you rock. Why don't you release your own kernel series, like the "aa" series? It would allow more people to test your code. I bet you could find ftp space for it.

Thanks for helping make Linux better.

Re:Make it a build option

[pepermil]

Rik,

Thanks for sharing your thoughts on this article here on Slashdot. I always prefer getting thoughts & opinions from their source. I was wondering if you could comment on the following quote from the Byte column:

Another very important change is that the swap is now additional to the RAM, just like in 2.2 times. All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap, similar to some versions of Solaris or other UNIXes.

It seems like using swap like this would be an incredible waste of space, as the article points out & I was wondering if this is in fact the case with your VM & if so, could you provide a technical explanation for why you feel it is necessary?

Major thanks for all your work from a measly college student in awe of kernel hackers like you. :-)

Re:Make it a build option

[mandolin]

In the future, I know I'll always be optimising for (1) maintainability, (2) correctness/stability and (3) performance, in that order...

Thx for the informative post.. tho I would personally give stability precedence :)

(no disrepect intended.. yer VM runs fine on my box)

Brief synopsis:

[grmoc]

This article by Moshe Mar talks about the linux kernel, and especially a recent split in the community in which/whose VM to use.

The "old" vm, the article says, had some relative impressive problems- swapoff with full memory and you could sit in front of a swap-crazy machine for 15 minutes, machines with small memory (40 MB or less) could experience sudden swap storms up to kernel 2.4.7, and the amount of swap available in the system depended on the amount of memory (as opposed to the system in 2.2 where the total memory was RAM+swap).

The VM used in 2.4 'till 2.4.10 was written (it seems from the article) in large part by Rik Van Riel. Later Andrea Arcangeli (in Sept) wrote a whole new VM, which was accepted into the kernel. Viola, new Vm in 2.4.10.

The article discusses that Alan Cox doesn't like the new VM as much, and has stuck with the old one. The author of the article seems to be in support of Andrea's new VM.

The article then goes on to discuss kernel preemption- pros (low latency), cons (lower throughput/power), and on which machines such things are important (servers, vs PCs).

AC kernels are not a fork

[rakarnik]

Moshe Bar seems to indicate that Alan Cox is creating some kind of fork of the Linux kernel. Actually, -ac kernels are alwasys different from Linux kernels to some extent, since they include slightly more experimental code (e.g. ext3), or code that Linus has not had a chance to review yet. This way, the experimental code gets more testing before going into official Linus kernels. You can read more about -ac kernels at KernelNewbies.Org.

As anyone following LKML knows, Alan thinks that drastic VM changes should be reserved for 2.5, and so continues to keep Rik's VM going. This actually helps quite a bit as both VMs get tested and there have been several comparative tests conducted leading to improvements in both VMs. Competition in this case is certainly helping Linux.

Oh and for all you fork conspirators, here's another fact: Andrea Arcangeli also releases his own kernel releases, called -aa. I don't think any of these are considered forks; everyone understands that this way pacthes get more testing, "crosstalk" between the different flavors is a given.

Much ado about nothing, IMHO...

-Rahul

Re:AC kernels are not a fork

[Webmonger]

The difference between the aa kernel and the ac kernel is that Alan Cox is widely recognized as the number 2 Linux guy, but he's promoting something completely opposed to Linus' decision.

If anyone could start a fork, it's Alan. However, remember that forks aren't necessarily bad. And there seems to be strong argument in favour of both VMs. . .

Re:AC kernels are not a fork

[rakarnik]

You are correct. However, this is neither the first nor that last time that they have disagreed on something. Furthermore, this was hardly a renegade sort of fork (which is what we dread), but one supported by several people against the VM change in a stable kernel series. I would not be surprised if Linus actually asked Alan to keep the old VM till the new one got its wrikles ironed out, so both could be tested side by side.

Anyway, as my other post indicates, Alan will be switching VMs soon.

-Rahul

Re:AC kernels are not a fork

[bwt]

I don't think any of these are considered forks; everyone understands that this way pacthes get more testing, "crosstalk" between the different flavors is a given.

Well, I disagree -- they are ALL forks. Any time you create a patch you are forking. The open source development model relies on perpetual fork and merge to accomplish its development. Most projects are forked this way into a development and a stable branch. I call this a "constructive fork". The AC kernels are perpetually different, but importantly they are generally about the same "distance" away, and "crosstalk" as you call it keeps it that way.

As the "distance" increases, tension increases, and if it isn't resolved it will divide the development camp. If the crosstalk stops, and the idea of eventual merge is abandoned, you have a "true" fork. Developers have to pick sides, and the split can become permanent.

I think the AC kernels have always been the former kind of constructive fork. If he never adopts the new VM, then his kernel will begins to diverge since developing for two VM's is hard. In this way, a small perterbation can become a full blown deviation that divides developer resources. I really doubt that the VM issue will divide the linux kernel team permanently. As AC's kernel gets farther away from the main line, the tension on everyone will increase. Eventually, I predict, the team will force one solution, but there is no guarantee.

Re:AC kernels are not a fork

[Ed Avis]

I believe that Alan Cox prefers GNOME to KDE. Therefore I have a cunning plan for an Alan Cox Trap. I will set up a machine with a specially patched kernel to use Rik's VM when GNOME runs, and Andrea's VM when running KDE. Mwhahaha....

Re:AC kernels are not a fork

[Tumbleweed]

Actually, I'd say they're more like 'sporks' than 'forks'. Nobody who makes them intends for them to take over from the main Linus kernel tree.

Re:The failure of Open Source

[garcia]

that sounds less like ego and more like a mistake on the employees part. Yeah conflicts keep your mouth shut at times but I am sure there was discussion among the rest of you that would have let you know that a good majority of those people you worked w/disagreed w/the route of the project.

Why not all express your opinion instead of being passive participants?

Sounds like it was a management fuck up not an ego issue.

Then again, I really don't know how it works in the real world, or so that's what they always seem to tell me.

Why does the ac tree persist?

[Malc]

The article seems to come out in favour of the new VM code. It makes it sound like it works much more effectively. So, why does Alan Cox continue with the old VM code? There must be some reason why he thinks it's better, or why go through the effort of continually patching the old code into the newer kernel?

Re:Why does the ac tree persist?

[tubby]

The article seems to come out in favour of the new VM code. It makes it sound like it works much more effectively. So, why does Alan Cox continue with the old VM code? There must be some reason why he thinks it's better, or why go through the effort of continually patching the old code into the newer kernel?

Basicly because nether of them are good in all conditions. Each of them is better than the
other in some situations. eg, big systems, little systems or whatever. While i am on the kernel mailing list i haven't been following the discussions closely enough to say any more than that, but it's the gist of it. Also for a while Alan continuing to run the Rik VM gave people a way to run a later version kernel without being lab rats for the new VM, which really hadn't had much testing in 2.4.10/11.

I think that this article overrates the AA VM by a large margin. It cant really be said to have solved the linux VM woes, which is what it implies.

I have now used both of the .13 kernels and personally found the -ac vm to be better for my needs. On the other hand, since i brought 768MB of RAM today, my needs have just changed.

Re:Why does the ac tree persist?

[osiris]

i think it is mainly to do with stability and the proven ability of the old vm code. basically, the new vm code was a complete rewrite from scratch and was incorporated into the main kernel straight away. the problem therein is the fact that the code will not have been as throughly tested and proven as the old vm. it may well be that the new vm is rock solid but it hasnt been in use for as long as the old vm to prove it. what alan is doing is sticking with the old vm as it is pretty much proven to work well and not fall over.

its not exactly trivial to rewrite an entire vm so there are bound to be problems with it. these problems come out through testing. i would have thought such a major rewrite would have been put in a development kernel first rather than into a "stable" kernel tree. that way, developers can test it first and iron out any problems rather than everyone upgrading to the new vm _then_ a major problem found.

the new vm may be brilliant and fast, but alan has a point in sticking with the old code. major rewrites should belong in development trees until fully ready for a stable release.

But it is in the 2.4.10 linus series

[wiredog]

Which is what he's talking about.

Re:But it is in the 2.4.10 linus series

[LinuxGeek8]

Well, I was actually saying that if you compare 2.4.10 with 2.4.9, you're actually comparing 2.4.10 and 2.4.5.

Even though the kernel had gradually evolved from 2.4.0 to 2.4.9, it was evident that the VM design was more of a liability than an advantage.

Point is, the kernel did not gradually evolve to 2.4.9, but only to 2.4.5.
Rik's VM has problems, but in the current ac tree it is doing quite well. Maybe as well or better then Andrea's VM.

Anyway, let's hope that the best VM wins, if there is a best VM.

Re:But it is in the 2.4.10 linus series

[Bun]

Point is, the kernel did not gradually evolve to 2.4.9, but only to 2.4.5.
Rik's VM has problems, but in the current ac tree it is doing quite well. Maybe as well or better then Andrea's VM.

Maybe, but I don't see why I have to assign 512MB of swap space (reccommended in Red Hat's 7.2 install) on a desktop system with 256MB of memory. With that much memory, and simple application usage, the system should operate fine without swap, or at most an equal amount to main memory. If the two systems are performing within 5% of each other, I'll take the one that doesn't want to sack away my hard drive space.

Re:But it is in the 2.4.10 linus series

[Cramer]

I've not done any tests lately, however, the way things were in the past (measured in years), the linux kernel sucked out loud without some swap. 1k or 100M, it didn't matter; simplying having swap available significantly changed the behavior of the system.

Surprisingly, the tivo, based on 2.1.24, performs better with swap disabled. However, that kernel is heavily modified and, well, ancient.

Re:But it is in the 2.4.10 linus series

[morcego]

Maybe, but I don't see why I have to assign 512MB of swap space (reccommended in Red Hat's 7.2 install) on a desktop system with 256MB of memory.

Actually, you don't. This is just the usual "make the swap twice as big as the physical memory" methodology. Old school, but a good recomendation if you don't want to have to calculate how much swap you will need in a case by case bases.

Re:But it is in the 2.4.10 linus series

[Bun]

Actually, you don't. This is just the usual "make the swap twice as big as the physical memory" methodology.

Hrm. The install program during the upgrade all but insisted I increase the swap, so I agreed to let it add a /SWAP file to supplement the 128MB swap partition I already had configured. I'll try it without it for a while and see how it goes. Thanks for the tip.

About time someone figured out this is a CF

[CDWert]

Who says an article cant be biased ?

Its an editorial for crying out loud

I have been wondering when the "masses" would figure out this is a MAJOR issue with the current linux kernel. I use the -ac patches because of various issues not in the main branch, fs support etc. on my workstations , I have to say the VM in the main 2.4.12-13 seems much more predictable and solid, I think im just going to patch against the main for ext3 until this gets hammered out. Alan seems like a very gifted indivdual as does Linus, but it seems as if Alan cant sometimes say hey I was wrong this is a better way to do it, or lets for the goals of the effort, decide and move on improving it as best we can. The author was very correct in his statment of you can fix something that sucks so it dosent suck as much but in the end it still sucks. I still dont get what alans problem is , is it because he didnt write it ? Or is it because its new and less tested ?

Excellent article in all.

Alan will be switching VMs soon...

[rakarnik]

See this posting to LKML:

Alans talking about switching VMs in -ac kernels

swap space?

[archen]

From the article - " All earlier 2.4 kernels (since 2.3.12) needed at least the same amount of RAM in swap and then more to give you additional virtual memory. This meant that on an 8-GB server, you needed to put aside almost a full 9-GB disk just to be able to swap"

Is this accurate? For just about everything I've always gone with 512Mb of swap, regardless of whether I had more or less RAM (not that I'm technically proficient or anything). This would also be a shortcoming of Linux since it would make it a pain in the ass upgrading RAM if you needed to allocate more swap space somewhere else each time. Well I'm all for the newer VM. Simple is good.

Re:swap space?

[osiris]

in general, you want to have about double the amount of swap space as you do ram. i believe one of the main reasons for this is when you get a major crash, the kernel cant dump the memory to disk, only to swap space, so you need the swap space big enough to hold the contents of your ram plus a bit more.

plus, with the price of hard disks these days, why would you quibble about having a 1gb swap partition? its a right pain in the arse to resize a swap partition. dont be stingy and allocate a fair amount to swap.

Re:swap space?

[archen]

Well that sort of depends on the drive too. I know at work there is a computer set up that runs 18 Gb SCSI 15k RPM drives mirrored, and those aren't exactly cheap (condidering a mirror is twice the cost for redundancy for a drive that was already expensive). With twice the swap of a machine that has 2Gb of RAM, we're talking 4Gb of swap - that's 22% of the drive.

Re:swap space?

[osiris]

who says that the swap has to be on the same drive? isnt it feasible to have, say, a separate ide or scsi disk dedicated to swap?

Re:swap space?

[Poppa]

in general, you want to have about double the amount of swap space as you do ram. i believe one of the main reasons for this is when you get a major crash, the kernel cant dump the memory to disk, only to swap space, so you need the swap space big enough to hold the contents of your ram plus a bit more.

I can't see how most users would care, this sounds useful only for debugging the kernel. If my kernel is crashing, I don't care if I run out of swap.

This is an old FAQ and OS legend. In linux, you only need enough swap to cover what your RAM can't handle. Even with pigs like Mozilla and Xemacs running, I barely dent my swap. (512MB RAM, only using 13MB of swap.) Since access to swap is much slower than RAM, it is best to get more RAM than you need. A few hundred MB of swap will then be sufficient.

Re:swap space?

[iabervon]

In the traditional VM formulation (pre-linux), every bit of VM would have a place in swap, and RAM would just keep a fast copy of the data. This greatly simplified the implementation, of course, because all of the data had a location on disk it could keep for its entire lifetime. Linux didn't do this: data would just be put on disk somewhere free, and could lose its place while in RAM. This meant that your total VM would be disk+RAM, not just disk.

As an optimization, and due to hard drive space being cheap, the first 2.4 VM used the traditional scheme, because, if a page hasn't been modified since it was last swapped out, it wouldn't have to be written at all if it was still there.

In any case, it's probably worthwhile to have at least 1.5 times the swap as RAM; if you have just a little bit of swap on a high-memory system, it's unlikely to save you from running out of memory, and will instead cause the machine to swap a lot before running out of memory anyway. You don't need to upgrade memory and swap at the same time, but you might as well upgrade swap first, or just turn it off. (That is, if you're getting more memory so you'll have more space, buy swap first. If you're getting more memory so it will be faster, replace swap with RAM)

These days, hard drives are cheap, and there are old hard drives lying around of reasonable sizes; just use a whole recently-replaced hard drive as swap. This avoids contention with filesystems and is easy to replace.

Re:swap space?

[Andrewkov]

Actually one thing I've thought of but never tried is to make several small swap partitions, maybe one per drive (I don't think it would make sense to have multiple swap partitions on one drive). In theory, synchronized reads from the different drives should speed the system up. Anyone know if this is true?

Re:swap space?

[victwenty]

This is mostly accurate, I believe it was actually fixed somewhere around 2.4.8 in the Linus tree, not sure about the ac tree. Read through the kernel change logs, the fix is explictly mentioned in there.

Re:swap space?

[h2odragon]

This is true.

There was a comment on LKML not all that long ago that dealt with the detials; IIRC it was in response to someone wondering about using software RAID0 for swap.

Re:swap space?

[spacey]

The swap has to be somewhere, and on critical systems you should mirror the swap, and it should be fast. So no matter where you put it (2 external raids, 2 internal disks, 2 silicon disks, whatever) you incurr a high cost for the additional and mostly useless swap space.

-Pete

Re:The failure of Open Source

[Anonymous Coward]

So your Director of Engineering couldn't direct his way out of a paper bag. Now please come up with a real refutation of the point made in the original post? You can't, because I am right.

The claim is not that ego only affects Open Source projects. The fact is that in Open Source projects nobody has the final authority and can keep progress rolling in the face of ego. Even if Linus says no to one option and yes to another, an equally respected leader can choose to ignore that decision and go his own way, taking a fair number of followers with him, dividing the effort.

Against the Truth

[Anonymous Coward]

Moshe Bar argues two points I vehemently disagree with:

(1) Alan made a mistake in not switching to Andrea's VM. Alan is trying to maintain a stable kernel. Switching out large chunks of the VM is the last thing to do to achieve those goals. Alan will switch in due time.

(2) The preemptible kernel is unfit for certain scenarios. Everyone I know loves the preemptible kernel. It gets good reports on lkml and the kernel news sites - Hell, it even got good comments here!

I realize this is an editorial, and I understand everyone has an opinion, but if it isn't true it isn't true. An opinion can't contradict fact.

Tim

Re:Against the Truth

[Flower]

I'll skip point 1. I agree with your assessment and as others have pointed out the switch will be made.

On point 2 however, I just don't agree with you. Moshe does more than a adequate job of explaining his stance on this issue. Between pointinging out the costs of making the kernel fully preemptible, citing his experiences with using it on personal machines (good) and servers (not so good), then noting the preemtible kernel breaks Mosix and LIDS I think he's got a right to his opinion. It's based upon at least as much fact as stating everyone loves the preemptible kernel.

Re:Against the Truth

[Anonymous Coward]

None of us arguments seem valid.

I'll tackle the ones you mention:

The costs aren't what he says. The context switch cost is non-existant because it balances out future context switches. Even if context switching was a cost you must remember the preemptible kernel preempts when it needs to like in userspace. The additional preemption is wise, in other words. It is worth it. If it weren't then why have small timeslices?

He did say his laptop was smoother, but what was wrong with his server? He cites some unknown term : unpredictability of latency. What is that? I bet if he benchmarks it he would see it is the same or better.

The patch can very well break Mosix and LIDS. It is a big change. This is like saying the patch is bad because you have to manually patch it. It is part of the routine. When the patch is part of 2.5 this will not be a problem.

Lots of facts are out there. Reviews, user reports, benchmarks, etc. Plus the mere fact a preemptible kernel works!

Re:Against the Truth

[SurfsUp]

Moshe does more than a adequate job of explaining his stance on this issue. Between pointinging out the costs of making the kernel fully preemptible, citing his experiences with using it on personal machines (good) and servers (not so good), then noting the preemtible kernel breaks Mosix and LIDS I think he's got a right to his opinion. It's based upon at least as much fact as stating everyone loves the preemptible kernel.

His analysis is way wide of the mark. His main argument is that kernel preemption causes more context switches, and that's pure BS. Yes, there may be a few more quantum-expiry switches but these are infrequent enough as to be very difficult to notice. He misses that fact that for some loads, a preemption makes tasks run faster because a user task can continue sooner, following a completed disk read. He also misses the true cost of the preemption patch in that spinlocks are slightly more expensive.

The benefits of preemption in terms of reduction in latency on the other hand are large and measurable. For desktop use it's no contest, and I see non-preemptible kernel mainly being an option for certain types of servers carrying a kind of load that exposes the slight extra cost of the spinlocks, and a percent or two extra throughput somehow matters.

Arguing over the best VM

[wiredog]

Is probably like arguing over the "best text editor".

Re:Arguing over the best VM

[Hiro Antagonist]

Not really; it's hard to quantify the benefits of a text editor; and although one may make a statement such as, "Editor foo enables me to edit files 50% faster," it doesn't mean much; because the reference is subjective to personal preferences.

With VM in the kernel, it's pretty obvious when things aren't working as well as they should -- mainly because it is possible to write test programs (scaffolding) that check to see if a designed system is performing to specifications -- read _The_Mythical_Man-Month_ for an excellent explanation of the merits of specification-based "scaffold" testing.

Re:Arguing over the best VM

[ThatComputerGuy]

Besides, everyone knows that vi kicks emacs's ass.

Should be a compile-time option, then

[Jeppe Salvesen]

If they are to truly compete, then we should be allowed to choose between the Andrea VM code and the Rik VM code when we compile our beloved kernels.

However, a kernel fork would not neccessarily(sp?) be a bad thing, as long as the forking doesn't break the ability to run binaries. I'd hate to have to recompile my entire system just switch between VM-s.

Re:Should be a compile-time option, then

[Anonymous Coward]

In Alan Cox's words, the source would be "too ugly for words". If anyone wants it, go ahead and make it, cos noone else is planning to.

Re:Should be a compile-time option, then

[Anonymous Coward]

> However, a kernel fork would not neccessarily(sp?) be a bad thing, as long as the forking doesn't break the ability to run binaries. I'd hate to have to recompile my entire system just switch between VM-s.

You can switch between VMs now; build one of each kernel, use your bootloader to choose which one to startup. Yes, you do get a lot of other material in the different kernels as well, but I'd argue they're closer to being common than the two VMs.

Re:Should be a compile-time option, then

[be-fan]

Dude, a VM is an awefully low level bit of code to make a compile option! XFS, for example, won't work the the AC kernels due to the different VM. Probably, the two seperate kernels are the best idea ATM.

Re:Should be a compile-time option, then

[Anonymous Coward]

> Dude, a VM is an awefully low level bit of code to make a compile option! XFS, for example, won't work the the AC kernels due to the different VM. Probably, the two seperate kernels are the best idea ATM.

Well, XFS does work on Mandrake maintained kernels (which are -ac based). And come to think of it, Linus behaved irresponsable by not maintaining the VM in his tree and then out of the blue replacing it with something experimental in a supposedly *stable* kernel.

He should just have taken the opportunity to finally start at 2.5 ...

ok, here's the thing

[Velex]

I don't care if you want to swear by the Linus kernel, but it gets killed by IO. I mean, come on, I'm using 2.4.12, and I can't rip a CD an play an MP3. Under the AC series, I can rip CDs, play MP3s, watch divx movies, surf the web, untar a file, and have a compile job going at the same time. Even for more usual setups, like viewing a video without doing anything else, the Linus kernel drops frames left and right, whereas the AC series laughs at it. Don't tell me I need to use mplayer with SDL, because I do.

Because I treat my Linux box as though it were a Windows box (one of the reason I switched over to Linux for everything is that the widgets in GTK are prettier than the widgets in Windows -- it's nice to have people ask me how to get their desktops to look like mine and tell them they have to install linux) and I expect it run at least as well as a Windows machine, I must use the AC series. While I'm sure that the Linus kernel has it's applications, it is simply unacceptable for replacing the Windows kernel.

Mod me flamebait or troll if you want, but I speak the truth. I have a Thunderbird-750 with 224 MB of ram, and I find it simply unacceptable when I can't run Quake or view movies under linux because of the Linus kernel. When mp3s skip because I'm moving some data around, it tells me that something is wrong with the Linus kernel. I'm glad that I had a friend who introduced me to the AC series, or I would have given up on linux. Plain and simple, politics aside, the end user doesn't care that he's being loyal to Linus the Great, he just cares that he can view that movie. If Windows outperforms linux in multimedia, he'll use Windows.

Re:ok, here's the thing

[mrm677]

I'm curious to know how FreeBSD would fare doing the same tasks that you described...

I'm pretty tempted to cross the fence and give it a try!

Re:ok, here's the thing

[cybrthng]

Windows XP does it great :)

So does my Solaris box and my Linux box running Redhat 7.1 + AC Kernel.

I havn't tried a stock kernel in years, i've been flirting with EXT3 and other AC features for a while. AC kernels were the only ones that let my laptop monitor run in AGP mode with XFree86 4.x without much problems.

Re:ok, here's the thing

[hughk]

Try the latest RH 7.2 kernel, it is 2.9-7 or thereabouts with a lot of extra stuff in and much from ac. It is certainly an interesting mixture, but with ext3, it is going to be very interesting how to keep up with later kernels (I was running vanilla 2.4.12 before on top of RH 7.1).

Re:ok, here's the thing

[choward]

I use the Linus stock kernel on a _very_ similar setup (Duron 700, 384MB ram) and I don't have the problem you mention. One thing I've noticed is that with the Linus kernel, DMA is _never_ turned on by default, you must use hdparm explicitly at startup. Once you do that, skipping mp3s are a thing of the past.

Running the hdparm tests,
w/out DMA: 4.01MB/sec
with DMA: 34.96MB/sec

Quite a change.

Craig Howard

Re:ok, here's the thing

[derF024]

i don't know what you're doing to that poor machine, but i have a 366 mhz laptop with 128 megs of ram and i can do all of those things under linus' 2.4.12 just fine. I play video under avifile instead of mplayer, but I never lose a frame, even across 10 mbit ethernet.

Re:ok, here's the thing

[Anonymous Coward]

wrong. Linus's kernel will automatically enable DMA if it is compiled with:

CONFIG_IDEDMA_AUTO=y
next time you recompile your kernel, select that option and you'll be all set.

Re:ok, here's the thing

[ryanvm]

Hmmm - an AMD eh? Would you by chance be using a VIA chipset?

I'll have to admit that I don't keep up on the LKML like I should, but I've been afraid to use DMA on 2.4 because of the file system corruption it was experiencing with VIA chipsets early on. [I have a PIII with a VIA Apollo Pro 133A chipset.] Wasn't that the reason that Linus decided to default to "DMA disabled".

Did that ever get resolved or should I still be wary of DMA?

Re:ok, here's the thing

[To Mega]

I had 2.4.12 running on a Via chipset motherboard (Asus K7M) with DMA and it was just fine. 2.4.7 crashed when I enabled the driver for my controller in the kernel however, so I upgraded to 2.4.12 and everything was awesome. 10MB/s on an older 5.1GB Quantum drive. 20MB/s on a year old 25G Maxtor.

You should back up your important files, and then try it. Immediately after backups, corruption can do you no harm, except for loss of time. But the possible increase in speed using UltraDMA outweighs any risk.

Re:ok, here's the thing

[Rick the Red]

OK, here's the thing: I'm not really competent to comment on kernel internals. However, I am competent to ask questions, and I fail to see that any of your complaints are directly attributable to the VM code. Couldn't some of what you're experiencing be due to other code in the AC series, not the VM code? I'd really be interested to hear what about the Linus kernel's VM code is affecting your performance, especially since you have pleanty of RAM and the article clearly states the performance differences between the Linux kernel VM and the AC kernel VM appear in memory-limited situations. Am I missing something obvious, or did you miss something subtle?

Re:ok, here's the thing

[aussersterne]

I experienced severe corruption in the early 2.4 series on my own personal KT133A board (Asus A7V133) but all recent kernels (I started trying again around 2.4.8) have been perfect with the Via UDMA enabled using two Maxtor and two IBM drives. Very fast and very stable... so in my experience, it's been fixed.

A lot of other naggish problems with the early 2.4 series (broken 2048-byte secotrs in FAT, nonfunctioning or error-prone CDRW output, clock skew and resultant bad DPMS stuff under load) have also been fixed. I found 2.4.10-ac12 to be very stable on my VIA 133a system and have now switched to 2.4.13, which seems equally stable so far.

The system is up 24/7 and is running Samba services for a small network of friends as well as web, mail, dns and a few other things. I run KDE 2.2.1 and routinely have a bucketfull of apps (OpenOffice, many GIMP windows, emacs, etc.) running as well as freeamp playing music files. I also have a script that grabs the nightly mozilla snapshot and builds it once a day.

Since I switched to the late 2.4 series, no problems or complaints. I honestly haven't had trouble with either Rik's VM in the -ac tree or the new AA vm and UDMA has been rock solid.

Agreed.

[Anonymous Coward]

I had a similar problem a while back... The company was designing a commercial software project.... Egos ruled the engineering meetings. Noone could agree on the correct way to write the modules. We couldn't even agree on what language to use. Some wanted to use ColdFusion, some wanted to use Delphi, some wanted to use Python, a few wanted to use PHP and some wanted to use Java. There was only one tech guru who could glue the whole project together...this guy wrote glue components which tied the whole system together, and bugfixed everyone's code (all in different languages). It was all going well, but the guy started to take longer and longer lunch breaks.... eventually the CEO followed him one day when he went on his lunch break ... it turned out that he was having sex with mares on a local farm. Of course, the CEO couldn't keep him on - but he still needed his technical expertise. Unfortuantely, rule of law overruled the need to keep the guy and he was fired. After that, the project just fell to peices and the company went out of business. And you thought Open Source projects had problems. Peh.

Re:Agreed.

[slashdot_commentator]

No wonder why employers only seem to have jobs in the city...

Virtual Memory stinks in Linux

[Apreche]

I have a large ext3 partition to store all my data and a 256MB partition of swap. I also have 384MB of RAM. Occasionally I'll hear the had drive grinding away like it's using the swap. I check and it is using the swap, but my real RAM isn't full yet. It's actually far from full, like 100MB free. I know if I don't have a swap partition it wont use it, but then I'll run out of memory sometimes, like when I have a huge pile of applications open at once. It really needs some work. I don't care about forks or anything, just make it work better.

Re:Virtual Memory stinks in Linux

[diamondc]

are you using any linux kernel 2.4.9 or higher? i remember 2.4.8 NOT releasing swap even when i had loads of memory. it was taking more than half of my 500mb swap partition. and i was leaving X running overnight with the daily cron jobs. with the recent kernels, i havent had that many problems with RAM or swap

question:malloc support?

[CaptnMArk]

Does it support malloc correctly now (returning NULL when out of memory)?

Re:question:malloc support?

[spitzak]

IMHO this is useless, it wastes a lot of memory and makes programs fail sooner (pages that are never written to or are immediately freed due to exec would take up space). More importantly this solves nothing, as memory does not necessarily run out when *your* process calls malloc, it runs out when *some* process calls malloc. Maybe in the old days when system calls did not require memory to be allocated this scheme would have worked but not anymore. I have no idea what a real solution is, but I kind of suspect nobody does...

Ummm, its not the VM

[j3110]

The old linux VM seamed incredibly insiteful to me. It is far superior to any other operating systems that I've seen out there. It keeps good stats on how long its been since the page was last accessed and swaps pages accessed least recently first and gives a bonus to clean pages.

The problems they describe is a fault of the schedular for picking a process that had a lot of pages just swapped. If the memory gets low, you need to try to give processes that have all their pages in memory and aren't blocking priority so they can do their thing and hopefully free some ram up. Thats why the guy made the kill patch it sounds like, but it should rather suspend than kill. There is nothin you can do if you are just running too many processes anyhow though. You have to swap out a bunch of pages for one process to swap in the pages from another. It's called thrashing, and the only thing you can really do is give one priority over the other, or get more ram. Hey, rams cheap, buy more if your gonna use that much :) Don't cause so much strife by switching key elements of the kernel in a stable tree.

I don't know how the guy pulled it off with the fact that paging is VERY arcitecture dependant. (the kernel pretty much ignores some of the features of X86 (segmentation) to make it more portable) I think their both right, but maybe linus should call it 2.6 so people can have some time to make their unofficial patches still work with it and have some time for testing before its used in a production environment.

Re:Ummm, its not the VM

[Anonymous Coward]

"The old linux VM seamed incredibly insiteful to me. It is far superior to any other operating systems that I've seen out there."

Evidently you haven't seen many.

ext3 in 2.4.x

[42forty-two42]

Hearing this, I want to upgrade, but the only ext3 patch(I use redhat 7.2) I have found is for the 2.2.19 kernel. Does anyone know where I can find one for 2.4.13?

Re:ext3 in 2.4.x

[Svenne]

You're looking for this.
I've been running ext3 since kernel 2.4.11 with these patches.

Re:ext3 in 2.4.x

[Anonymous Coward]

I believe this is the homepage:

http://www.uow.edu.au/~andrewm/linux/ext3/

Re:ext3 in 2.4.x

[Stonehead]

The -ac tree has ext3 support patched in by default, since 2.4.7.

Pluggable vm?

[soybean]

I'm not kernel hacker, but this seams like a good opertunity to implement a pluggable vm archetecture. This would allow this debate to become moot, as well as allowing folks to load vm's that were talored for their specific work loads.

Re:Pluggable vm?

[Hiro Antagonist]

No; 2.4 is a stable kernel. Changing the VM as of 2.4.10 without a code fork was bad enough, and the last thing that needs to be done is to introduce yet another radical change in a supposedly stable series of kernels.

It's not as simple as moving the code to a 'vm1.c' and a 'vm2.c'; VM structures are pretty deeply integrated into quite a bit of the kernel, so it would be a major rewrite to set up a pluggable VM architecture -- which is (theoretically) planned for 2.5 (where development code belongs).

Compound errors

[Salamander]

IMO both Rik's code (RVM) and Andrea's (AVM) were accepted prematurely, and Linus's ADD is the root of the problem here. Everyone thought the 2.2 VM was broken, so he jumped on RVM when it really hadn't received adequate testing with various workloads. Then, when that didn't work out, he did something even worse by jumping on AVM in the middle of a "stable" kernel series when it was totally undocumented and even less thoroughly tested than RVM. That's just bad software engineering, regardless of the quality of Rik's or Andrea's work.

Ideally, an "old-fashioned" alternative to RVM would have been maintained throughout the 2.3 process, as a fallback in case RVM turned out not to be ready for 2.4 - which was in fact the case. But this wasn't done, there was no alternative, and so RVM became the basis for 2.4. Once that decision was made it should not have been unmade by replacing RVM with AVM. Andrea's work should have been in the 2.5 tree, which should have been opened a long time ago to deal with precisely this sort of situation. 2.4 is not the last Linux kernel that will ever exist. We don't need to make it perfect. It would be far better to admit its imperfections, band-aid them as best we can, and try to get a head start on creating something better for 2.6. What we have instead is error on top of error, "not ready" replaced with "even less ready".

To clarify, I have nothing but the highest regard for both Rik's and Andrea's work. Obviously they have different ideas and attitudes. Rik has drawn on many sources in his design, resulting in a system that is both very advanced and very complicated. The process of reining in the complexity is still incomplete, but I still have hope that some day Rik will be able to come up with something that's really awesome, and he has always documented his ideas thoroughly. Andrea, by contrast, is much more pragmatic; he wants something that works now even if it's somewhat more limited in scope (e.g. by being almost impossible to reconcile with NUMA). The dark side of that "pragmatism" is that Andrea has skimped on non-code activities such as documenting or explaining the basic ideas on which his system is based. Nonetheless, both have done great work and should continue to do great work...in the 2.5 tree.

Re:Compound errors

[Milican]

Where is the moderation fairy? Why does she only give me dust when there are lame stories to moderate. Please sprinkle thy dust into my hands so that I may bless this post.

JOhn

Re:Compound errors

[puetzk]

FWIW, I think that Andrea's setup is modeled after the 2.2 VM (which he did a fair amount of work on tuning). So this is really more of a pragmatic revive-the-old-approach than it might initially seem.

We all know this simplistic setup had scalability problems (like much of 2.2) but at least it worked right. Hopefully given some more time, Rik can really get his to go, since it seems more sophisticated/scalable long-term.

Re:Compound errors

[Salamander]

FWIW, I think that Andrea's setup is modeled after the 2.2 VM

Yes, mostly. Except for the classzone stuff.

We all know this simplistic setup had scalability problems (like much of 2.2) but at least it worked right. Hopefully given some more time, Rik can really get his to go, since it seems more sophisticated/scalable long-term.

Absolutely. I'd love to see Rik and Andrea (and others?) competing on purely technical merit, in the 2.5 tree. I think that would be great for everyone.

Re:Compound errors

[slashdot_commentator]

Hey, Linus f***ed up with accepting the original RVM for 2.4. And now he was between a rock and a hard place. "Marketing" considerations meant that 2.4 needs to be "non-experimental"; that the client base could go to the 2.4 version, and use it with little concern that their server would crash. That would not have been the case with RVM versions up the last month.

So what was Linus to do? Keep dragging out 2.4 until RVM could fulfill minimum "marketing" requirements of stability? How long is that going to take? Do you want to wait and let M$ marketers talk about how amateurish Linux was; that professionals did not use Linux's current "stable" release, but a version that hasn't improved in 3 years?

So Linus decides to commit A REALLY BAD PRACTICE, and changes to a less tested VM over the initial 2.4 VM. Its another f**kup if AVM is buggier than RVM. (But Linus had reason to believe it wouldn't be so, with relatively limited testing.)

Is it still an f***kup even if AVM turns out to be more stable than RVM? If so, are you saying its preferable for new Linux development to be shutdown for another 6 months to a year? And Zdnet to opine on how the "stable" 2.4 kernel is DEMONSTRABLY unreliable? I'll take a "manager" that makes mistakes and makes decisions based on product survival over a manager that religiously follows an engineering practices manual.

Re:Compound errors

[Anonymous Coward]

Well, just one question. What do you mean by f***kup? As regexp, it expands to whole lot of words, from 'fkup' to 'file markup' and beyond. Infinite list even includes the obvious candidate fuckup. So, why do you obfuscate your text, if you mean fuckup? Is it more elite that way? Or are you just a yellow-livered 14-year old whose mother regularly washes your mouth with soap?

Re:Compound errors

[Salamander]

are you saying its preferable for new Linux development to be shutdown for another 6 months to a year?

No, there have been quite enough delays associated with 2.3/2.4 already. More than enough. And there will continue to be delays until the processes get ironed out.

What would have been preferable, IMO, would have been if more resources had been devoted to fixing and tuning the VM we already had (RVM, for good or ill). Linus could have put his foot down. He could have said "There will be no 2.4 VM except for RVM. The price for admission to the next round of VM redesign is that you help us fix RVM." People - notably Andrea - would have listened, and contributed more constructively. They know that Linus's good will is like currency. But Linus didn't say that. Alan Cox pretty much has, and kudos to him for having the courage to do so. What Linus did was take a bad situation and act in a way that nine times out of ten would make it worse. Maybe he'll get away with it this time because AVM in its current state is more robust than RVM in its current state, but that would actually be a bad thing because it will only reinforce the bad decision-making and we'll get burned next time instead of this time.

And Zdnet to opine on how the "stable" 2.4 kernel is DEMONSTRABLY unreliable?

First off, are good reviews from places like ZDnet the goal for Linux development? Second, do you think it's better for the stable 2.4 kernel to be subtly, unpredictably unreliable? Better the devil you know, and all that.

Most importantly, what if Linus's gamble - and that's what it was - hadn't succeeded? What would the ZDnet reviews be like then? What kind of ammo would that provide for everyone who wanted to claim that open-source development processes weren't all they're cracked up to be? Yeah, it looks (so far, knock wood) like we've been lucky this time, but I don't think relying on luck is a good thing.

I'll take a "manager" that makes mistakes and makes decisions based on product survival over a manager that religiously follows an engineering practices manual.

The two aren't as diametrically opposed as you make them out to be. Good engineering practices are good because they help increase either the speed or the reliability with which product can be delivered. Slavish adherence to any dogma is a bad thing, but so is the belief that everything you're doing is OK just because you managed to win one game of chicken. My point is that this scenario is going to be repeated. I'd rather encourage responsible driving than watch what happens when Linus plays one game of chicken too many and brings everyone else along for the ride.

Re:Compound errors

[Anonymous Coward]

Saying that a decision should never be unmade once made is silly; mistakes happen, corrections are inevitable. Linus isn't just doing these things blindly without a care for the technical reasons behind them. AVM is working well. There is no development kernel branch to test things out on right now (the -ac series is the closest thing we have to that).

Re:Compound errors

[Salamander]

There is no development kernel branch to test things out on right now

And whose fault is that? Who has unilaterally refused to open a 2.5 branch even though there clearly should have been one months ago? The lack of a 2.5 branch is a major piece of what's wrong, and it can be laid at exactly one person's clay feet.

Re:Compound errors

[slashdot_commentator]

What would have been preferable, IMO, would have been if more resources had been devoted to fixing and tuning the VM we already had (RVM, for good or ill). Linus could have put his foot down. He could have said "There will be no 2.4 VM except for RVM. The price for admission to the next round of VM redesign is that you help us fix RVM." People - notably Andrea - would have listened, and contributed more constructively.

I am not sure if I am incorrectly grasping what you are trying to say. Is this a variation of "Linus should have stuck to RVM, and people would have "pounded" it into stable operation?" What the hell were developers doing before 2.4.10?
Lets say developers are correct in the opinion that RVM is complex. Sometimes the "ideal" implementation can be so technically challenging that it takes a longer time to make it function properly. (And I recall the nightmare of Solaris v2.1 to v2.5 when I think this.) That may have been the limiting factor in "resources". Linus dictating the adoption of a (perceived) "not-ready-for-primetime" VM would not have corrected the situation if this were the case. Why do you think that Andrea did not make an honest effort to help fix RVM bugs?
I think (and I can be dead wrong here) Andrea (as did at least Linus) took a good look at Rik's RVM, and saw 2.5 being started a year from now because of time needed to fix a "complex" RVM. He decided to rewrite a dumber version of RVM that was more likely to work bugless. I suspect it was an over-the-top kludge, and Andrea will not be pushing AVM for 2.6 when RVM gets cleaned up in 2.5.

First off, are good reviews from places like ZDnet the goal for Linux development?
ZDnet is a metaphor for the technical print media industry. (Obviously, ZDnet can go f**k themselves, good or bad.) What I don't want to see is any media entity running a web load testing suite and watch a Linux 2.4 server meltdown, while IIS runs like a pig without crashing.

Second, do you think it's better for the stable 2.4 kernel to be subtly, unpredictably unreliable? Better the devil you know, and all that.
Except if more (talented) developers can understand the unknown (AVM) devil, but not the better known devil.

Most importantly, what if Linus's gamble - and that's what it was - hadn't succeeded? What would the ZDnet reviews be like then?
Damned if you do, damned if you don't. Was Linus better off standing pat? I agree, I hope Linus does not make a habit of playing chicken.

Re:Compound errors

[Salamander]

Why do you think that Andrea did not make an honest effort to help fix RVM bugs?

Obviously the only authoritative answer would have to come from Andrea, but my guess can mostly be expressed in one word: ego. He thought his fundamental ideas were better than Rik's.

I think (and I can be dead wrong here) Andrea (as did at least Linus) took a good look at Rik's RVM, and saw 2.5 being started a year from now because of time needed to fix a "complex" RVM.

No matter how long it might have taken to fix RVM in 2.4, that's no excuse for 2.5 not being open. All of the flip-flopping should have taken place in 2.5, and if AVM turned out to be the "winner" in that environment it could have been back-ported to 2.4 if necessary.

Except if more (talented) developers can understand the unknown (AVM) devil, but not the better known devil.

That gets a little harder when AVM - and in particular its features that diverge from 2.2 - is almost totally undocumented. Many of the kernel developers have complained bitterly about this fact.

Was Linus better off standing pat?

Yes. The best and average cases were comparable, and the worst case for standing pat was better as it didn't involve calling the whole open-source development model into question. Perhaps even more importantly, taking a stand on principle would have set a good precedent and example for the future.

Re:Compound errors

[slashdot_commentator]

Why do you think that Andrea did not make an honest effort to help fix RVM bugs?

Obviously the only authoritative answer would have to come from Andrea, but my guess can mostly be expressed in one word: ego. He thought his fundamental ideas were better than Rik's.

Hmmm, improperly worded question. What evidence can you present that supports your conclusion that Andrea did not expend effort in debugging Rik's VM component?

No matter how long it might have taken to fix RVM in 2.4, that's no excuse for 2.5 not being open. All of the flip-flopping should have taken place in 2.5, and if AVM turned out to be the "winner" in that environment it could have been back-ported to 2.4 if necessary.

But 2.4 would still be "broken" until that event occured. And don't you think it would have been more difficult to determine the more stable VM with unstable code being tested at the same time? I understand that you think having an embarassing demonstration of 2.4, the "stable" kernel, crashing with IIS chugging along is preferable to the same event occuring after a bad engineering practice. But to users and the magazine writers, it looks like a failure either way. I know you think its better to lose "by the book", but real world managers would grasp at the straw.

I don't believe this decision "calls the whole open-source development model into question". Its always been understood that the linux kernel is Torvald's benevolent dictatorship, and people were welcome to hit the highway. That has always been a structural consideration of O-S development. The only thing this decision does is put Linus's reputation as a project manager into question. "Stands on principle" and "good precedents" sound rather ideological and they don't benefit you if you're "dead".

Re:Compound errors

[Salamander]

What evidence can you present that supports your conclusion that Andrea did not expend effort in debugging Rik's VM component?

I never claimed that, nor is it relevant, so I don't need such evidence. Andrea did in fact spend considerable time helping debug RVM problems, but obviously at some point gave up and decided to undermine that ongoing effort instead.

don't you think it would have been more difficult to determine the more stable VM with unstable code being tested at the same time?

Odd-second-number kernel series aren't so unstable that it would be impossible to distinguish VM-related problems from other problems.

I know you think its better to lose "by the book", but real world managers would grasp at the straw

"The book" is written the way it is because - in general - the odds of failure are lower when you do things by the book. Real managers who have a clue know that. "Grasping at straws" is bad practice. The folks at SEI will tell you that the primary hallmark of a mature software-development model is that it produces repeatable results (and, at the higher levels, information that can be used to improve the process further. Sometimes you have to pick the best of several bad alternatives, but that should still be done according to proven risk-management principles instead of "grasping at straws".

I don't believe this decision "calls the whole open-source development model into question".

Linux development is often held up as a model of how well open-source development can work. If AVM had proven to be less stable than RVM - and there was, at the time, practically no reason to believe otherwise - and the resulting instability in 2.4 were also associated with an obvious case of bad process management, that most certainly would reflect on open-source development models. I can see it now. "Open source development is fragile, it's vulnerable to the whims of untrained project leaders who make decisions based on politics rather than technical merit..." and so on. You can't seriously believe such polemics would not be written in such a situation, can you?

"Stands on principle" and "good precedents" sound rather ideological and they don't benefit you if you're "dead".

See previous about why the book is written the way it is; the whole point of the book is to reduce the chances of ending up "dead". Try to put yourself in Linus's shoes, at the time the decision was made. You have two choices:

1. You can keep going with a well documented and well tested VM system that has been making steady progress, even if some problems - including some fairly serious ones - still exist and some of its principles are well understood only by a few people.
2. You can accept a patch that implements a whole new VM system that's totally undocumented, that *nobody* except the author understands, and that's very lightly tested so you have no real idea whether it's stable.

Can you seriously suggest that #2 was the responsible choice, most likely to lead to success? I don't think so. Linus made the decision based almost entirely on his personal feelings about the people involved, a lapse of technical objectivity which any "real manager" knows often leads to disaster. Whether disaster actually occurs in the here-and-now is irrelevant; such behavior should be discouraged in the strongest terms by anyone who cares about the future of Linux.

author's new benchmark

[brer_rabbit]

Upon returning home the other week after meeting with Andrea, I went to my lab and searched for the disk images of the server comparison I ran back in January of this year (of FreeBSD 4.1.1 versus Linux 2.4.0). I took the Compaq ML500 server I have been reviewing (2x 1-GHz CPUs, 2-GB RAM) and upgraded both the FreeBSD disk image to 4.4-Stable and the Linux version to 2.4.12.

Good, this would be an interesting benchmark.

Then, I changed the memory down to 192-MB RAM so as to stress the VM system more.

ok, this is fair, but you should also run with the same memory configuration you originally ran.

I also upgraded to the latest stable versions of Sendmail (8.12.1) and MySQL (version 3.23.42). Finally, I compiled everything with the latest version of gcc, 3.0.2, and tuned the two instances to the best of my knowledge (softupdates and increased maxusers for FreeBSD, and untouched default values for Linux).

NO!!!! why would you do this? Don't you want to know how the earlier linux/FreeBSD kernel compares to a later ones? Now instead of modifying one variable you've modified 3,846 variables. It's going to see if any improvements in FreeBSD/Linux are due to an updated kernel, compiler, mysql, etc etc. Go back to your original setup and only change the kernel, since I believe that's what you want to benchmark.

Re:The failure of Open Source

[Hiro Antagonist]

So your Director of Engineering couldn't direct his way out of a paper bag. Now please come up with a real refutation of the point made in the original post? You can't, because I am right.

Don't make pointless, baseless assertions.

The claim is not that ego only affects Open Source projects. The fact is that in Open Source projects nobody has the final authority and can keep progress rolling in the face of ego. Even if Linus says no to one option and yes to another, an equally respected leader can choose to ignore that decision and go his own way, taking a fair number of followers with him, dividing the effort.

The same thing can and does happen in commercial projects; instead of code forks you get mass firings, but the end result is the same. A manager with an overinflated ego is very bad for any project; be it open source or not; the difference is that, in open source, the clash between management (the project maintainer(s)) and programmers exists under the scrutiny of the public eye, whereas in a corporate setting the conflict is kept within the company walls.

One of the qualities, if not the top quality, of a good manager is the capability to admit that they are either wrong, or don't have sufficient information to make a decision.

BSD VM

[Anonymous Coward]

Why is so much time being wasted developing and re-developing a VM system when a very stable and robust VM system has existed for years in the FreeBsd system. Anyone thought about using that code (as per the terms of the license) as a starting point instead of this senseless writing and rewriting and sub-par performance?

Re:BSD VM

[Kymermosst]

This one is simple enough to answer: the "philosophy" of the VM from BSD could be taken, but the code? Not really. Though the license may let the code be used (though there are debates on the compatibility between BSD-licensed code and GPL-licensed code), the BSD and Linux kernels are completely different. You can't just simply take the BSD code and adapt it to Linux, or even use it as a starting point. The VM is a core component of the kernel-- in fact, it's one of the major functions of the kernel along with I/O abstraction. It's really much easier to write a VM system for a kernel from scratch, as opposed to porting one from a completely different system.

There is also the argument, made by some, that the BSD VM system isn't particularly good a low-memory situations, either.

Re:BSD VM

[Anonymous Coward]

For three reasons:

1) The interfaces are completely different
.
2) No one in the linux camp actually understands well FreeBSD's VM.

3) Not Invented Here!

Bring yourself up-to-date

[marm]

The machine freezes EVERY time because of memory shortages. The kernel can't allocate pages for incoming network traffic, causing a backlog, causing processes to hang, causing further backlog.. then powie an unresponsive machine.

This was a common problem with kernels from about 2.4.1 up to 2.4.9 - the machine would gradually eat into swap further and further, failing to release no-longer-used swapspace, until it would go Out Of Memory (OOM) and attempt to kill the process that was eating all the memory. Frequently it would pick the wrong process to kill (sometimes even killing init) or would end up deadlocking.

I agree with you - that is no way for a virtual memory system to behave.

However, the Linux development process moves quickly once people get annoyed enough to actually do something about it, and that's precisely what has happened. Starting with 2.4.10, a new, simpler VM system has been used in the official Linus kernels, and I can say with some confidence that it has solved all the major problems with the 2.4 VM system, and continues to get significantly faster with every release.

If you haven't actually tried a new kernel yet (and from your problems it seems that you haven't), I suggest that you do - it's made the world of difference for me.

At the same time, the old 2.4 VM has lived on in the -ac series of kernels, and has become a great deal better there - some competition has made a big difference. Almost all of the major areas where it behaved badly have been fixed. However, my own impression is that it is still somewhat slower than the new VM.

The choice is yours which you want to run - my own recommendation would be for the new VM in the official Linus kernels, but others may disagree.

[OOM Killer]
NEWS FLASH they took this feature out because it was buggy.

Umm, no they didn't - it continues to exist in both the new VM in 2.4.13 and the old VM in the most recent 2.4.13-ac kernels. It does, however, now work correctly in both VMs. There are some philosphical arguments over whether killing processes is the best way of handling an Out Of Memory situation, but it is surely better than deadlocking the box, which is what most VM systems (including the famed FreeBSD's) do when OOM occurs.

It's been getting better with each dot release but it's still nothing you'd want to bet money on.

All I can say is that the new VM works great for me and lots of other people, even under extreme load. I can certainly understand your pain if you're using an older 2.4 kernel, but please try a recent one - the difference is astounding.

If you're still having problems with recent kernels, then I'm sure linux-kernel@vger.kernel.org would love to hear from you - and would certainly be a lot more useful to you than ranting on Slashdot. Getting the VM right is now priority number 1 for the kernel hackers.

Re:Bring yourself up-to-date

[Laplace]

I will preface this by saying that I am not a kernel developer.

Wouldn't it be possible to label some processes as OOM immune? For example, init could have this flag and would never be killed by the OOM algoritmn. Similarly, users could designate some processes more important than others. For example, my PDE solver which is crunching away at data for my thesis could be immune, but X could die if I ran out of memory.

This whole situation has had an impact on my work. With all of the debate and argument flying around, I'm not sure which kernel to use, if I should upgrade, or if I should revert back to the 2.2 series. Oh well.

Re:Bring yourself up-to-date

[Anonymous Coward]

FreeBSD deadlocks? Granted, I have not tried FreeBSD 4.4-STABLE but 4.3-STABLE on my old laptop runs out of memory all the time and I have never ever had a single deadlock.
In fact, the old 486DX2 runs like a champ with only 16Mb RAM running PostgreSQL, Apache with mod_python, several vi sessions and all kinds of funky software.

Re:Bring yourself up-to-date

[Xzzy]

> If you haven't actually tried a new kernel yet
> (and from your problems it seems that you
> haven't), I suggest that you do - it's made the
> world of difference for me.

We've been trying new kernels as they arrive and we get the time; currently up to 2.4.10 on a brand spanking new build machine and it croaked the other day with the same symptoms. Testing new stuff takes time around here.. lots to do and sometimes trying the latest and greateset takes second priority.

> Umm, no they didn't - it continues to exist in
> both the new VM in 2.4.13

Stand corrected, like I said I'm not a guru on the subject I just read a lot of text trying to figure out what the issues were. ;)

> If you're still having problems with recent
> kernels, then I'm sure linux-kernel@vger.kernel.org
> would love to hear from you

If I had any new information to provide, I would. As it is, "the machine freezes" isn't too helpful and I'm the sort that keeps his mouth shut when other people are already saying pretty much the same thing. ;)

My main point in posting was to try and let people know that the current VM definetly "doesn't work", at least not as well as the original poster appeared to be claiming. :)

Re:Bring yourself up-to-date

[pi_rules]

Not a kernel developer here... but I've got a -tiny- bit of education on the stuff.

Thing is, what kind of mechanism would you use to label stuff OOM immune? You'd certainly have to modify the PCB structure for each process, unless there's some unused bits in there already. If there -is- an unused bit you really wouldn't have any hit at process switch time because you were already copying that unused bit. If you -do- have to add one there's 32 more bits that need to be copied. I dunno how much of a hit this would cause really but I'd imagine keeping process switch time down to an absolute minimum is essential.

Now, you'd need another user-land tool to set these flags which would presumably be run as root only, and make a decision as to whether a process starts as OOM immune or not.

Then again, maybe OOM immune status could be determined simply by the 'nice' value of a process.

I'd imagine the code to do this would be fairly trivial, but the impact would be rather large. Man, thinking about stuff like this really makes me want to get down and dirty and start doing some -real- programming again.

Re:Bring yourself up-to-date

[fferreres]

It is also important for what we do with Linux. We need sshd to live. If the VM kills any needed daemon plus sshd, then we are pretty much screwed because we are 10000 miles away from the server and support is expensive and not time-friendly.

I'd love to mark ssh and inetd as not-killable. Just kill mysql, sendmail and httpd but leave those alone! (maybe it'd be good to have the killing notified /var/messages so i can send me an email).

-

Re:Bring yourself up-to-date

[Salamander]

There are some philosphical arguments over whether killing processes is the best way of handling an Out Of Memory situation, but it is surely better than deadlocking the box, which is what most VM systems (including the famed FreeBSD's) do when OOM occurs.

Untrue. Simply untrue. Most VM systems will slow down, some gracefully and some not, but they will not deadlock.

It's also more than a philosophical issue, BTW; it's a very real-life practical issue. The whole idea of killing a process - any process - completely and irretrievably because the system is low on memory is simply brain-dead. Better solutions might involve working-set limits, emergency reserves (similar to what's already done for disk space in some filesystems), disallowing memory overcommit, long-term suspension of processes so their pages can be stolen, etc. - the possibilities are nearly endless. The OOM-killer approach always carries the possibility of leaving the system truly deadlocked as one member of a temporal-dependency chain - perhaps spanning multiple machines - gets shot in the head. You'll have plenty of memory, but it won't be doing you any good as you're deadlocked nonetheless. There is no problem that the OOM killer solves that is not solved better by a different approach.

How can you get around the VM problems in Linux?

[cbwsdot]

Good article, but what can I do, the end user, to get around some memory management and scheduling issues? I don't know nearly enough to do any kernel tweaking and tailoring for my processor architecture and hardware configuration. Is it unwise to run a desktop system without swap space? I was thinking of giving the performance of my system a boost by leveraging the low RAM prices and eliminating the swap partition. Other /.'s and I have experienced swap space usage during times when some physical RAM was free. Is the kernel thinking "Well, these pages are *so* old and inactive enough that I'll just stick them in swap, regardless of the current system state"? It sounds like paragraphs 16 and 17 attempts to explain this but it flies over my head. Since you can buy 1GB of EEC PC133 RAM for about $150, I was thinking of buying 3GB of RAM, investing in some type of power backup and running my entire system in RAM. Can you do this?

Despite this major issue, a Linux based system is still more stable and in most cases faster than Windows 2000. Also, like the article mentions, take into account that Linux runs many different types of processors. Linux on SPARC is good and 21264 Alpha performance is mind-blowing. Keep up the good work.

Moshe Bar

[Dante]

I have read most of Moshe's articles at Byte and he hits me as the worst kind of dilettante; can't keep his mind made up, and is more interested in fashion then really helping solve any problems. It all about being on the inside and looking clever then anything else.

Am I wrong? I kinda hope so.

Re:Moshe Bar

[Anonymous Coward]

Yeah, all that MOSIX stuff is just a trick, it doesn't really DO anything, really.

Re:The failure of Open Source

[Anonymous Coward]

That company (mine, unfortunately) is most likely going out of business soon. So don't give me this crap that ego only adversely affects Open Source projects.


Gee, it's sad and all, but it's too bad you don't work for Microsoft...
*runs

This Explains a LOT

[Uggy]

We manage a LOT of servers and when we went gold with our new appliance, we decided on the 2.4.8 kernel (because it fixed a bunch of smbfs bugs that would slow down smb transfers to a few bytes/sec). Well, we started getting heavily loaded machines going offline. I'd arrive on site and not even be able to log in on the console. Hard drive light would be on steady and it was just thrashing itself to death.

Problem was, there machines, although supporting a lot of users, weren't supporting them simultaneously. It seemed that they would creep up to their limit of memory and once they arrived at the edge, the kernel couldn't figure out what to release, even though the machine only had one or two users at the time. It was the straw that broke the camel's back. So we're throw more RAM in the sucker, and wonder why the hell this never happened on the 2.2 kernels. I'd mutter that it probably had something to do with the VM, but since I'm not on the kernel mailing list and I hadn't heard any grumblings on any linux based news sites, I figured the problem was us. Well well well! This is a HUGE bug and I'm surprised that it was not addressed more publicly.

My desktop does this too from time to time. Swaps itself to death on a 2.4.8 kernel as well. Open staroffice 6.0 beta, gimp, and bunch of other random things and it eventually goes unstable. Never happened with the 2.2 series.

A compilin' I will go, a compilin' I will go!

Re:This Explains a LOT

[DarkMan]

Um, not meaning to tell you how to do your job, but if your admining Linux machines with new kernels, and _not_ following the lists, or Kernel Traffic at least, then I don't think you can really complain about not hearing about it. It's been well known scince 2.4.[012] that there was a VM issue at heavy load.

Re:This Explains a LOT

[TheLink]

Um, not meaning to tell Linus how to do his job. But if he's releasing a kernel designated as stable (2.4.x) it shouldn't have VM issues at _common_case_ heavy loads.

Might be excusable for problems with unusual loads, but to fail for common cases scenarios like this is disgusting/sad/terrible/disappointing.

Don't they do any regression tests before the release of a designated stable kernel? You don't have to do that for development kernels, but I'd expect more rigorous testing for stable kernels.

If the linux core development team have no respect for what a STABLE release means I think I might have to switch new installs over to FreeBSD till things are safer.

Fortunately Redhat's 2.4.x kernels seem to be ok. Given all the flames on Redhat here, I think it's kind of funny that I'm getting my stable kernels from Redhat.

Cheerio,
Link.

no kidding

[Anonymous Coward]

of course they keep the user info in a database...
anybody who this comes as a surprise to shouldn't come here anyway, because they must be really stupid.
And they have the right to keep information like who is banned, why, and etc.
Otherwise, leave..... i see no hypocrisy. they don't make you tell them your real name address and credit card number like the real privacy abusers.

Please check your facts

[Anonymous Coward]

"...which is what most VM systems (including the famed FreeBSD's)"

This statement is pure bullshit. FreeBSD's VM is lightyears ahead from Linux' one. Same for SCSI subsystem. I once ran Oracle for Linux in a FreeBSD box with 2MB or RAM and 8MB of swap with 1500 concurrent users with no problem at all. A Linux box with 256MB was unable to handle it. Andrea wrote the VM in one month, that's the quality of Linux kernel code, re-code every release because last version sucked ass. Nice, really, nice.

Re:Please check your facts

[Anonymous Coward]

You are a sad little troll, who can't apparently even read. What the poster said is that "if [SHIT HITS THE FAN], BSD chokes". That's not much of a comment on general goodness of BSD kernels, and it sounds true enough. If there are no pages to free, bad things are likely to happen, unless you kill process(es).

THAN

[GrammarPhone]

more technical then the stuff

The word you're thinking of is "THAN". How many times are you going to make this mistake? I don't think I've ever seen Taco post a story using the word "than" properly. It's always "then". Real professional site you got here, Taco-man.

The real question: which will RedHat chose?

[Anonymous Coward]

Which kernel will RedHat use for their next Linux distribution?

Microsoft

[fcd]

It's certainly good to have competition to bring out the best in each system, but it would be horribly inefficient to keep it going in the long run.

Isn't that basically Microsofts argument as to why its ok for them to be a monoply? That competition is not efficient in the software industry?

Why VM is bad

[Animats]

Virtual memory is way overrated, and probably should be phased out, both on servers and desktops.

In Peter Denning's classic paper, The Working Set Model of Program Behavior, Denning concluded that paged virtual memory was, at best, good for an effective 2X increase in memory size. When he wrote that paper in 1968, memory cost about a million dollars a megabyte, so a 2X increase was worth the headaches of a VM system. Today, with memory at a few hundred dollars a gigabyte, it looks less attractive. It's not that expensive to double the size of RAM today. It can be cheaper than adding a fast disk drive just for paging. Uses less power, too.

Disk as backing store gets worse as RAM gets faster. When Denning wrote that paper, the fastest backing devices (drums) rotated at around 10,000 RPM, for a 6,000 microsecond access time, and core memory cycle times were around 4us. So main memory was 1,500 times faster than backing store. Today, RAM cycle times have dropped to around 0.020us, but disks still top out around 10,000 RPM, making main memory 300,000 times faster than backing store. Thus, the relative cost of a page fault has increased by a factor of 200. This makes VM far less attractive today than it used to be. It's not getting any better, either.

The price of having virtual memory is terrible performance once paging between active processes starts. That's called "thrashing". On a server which is processing short transactions, you're much better off throttling at the transaction launch point (as, for example, where CGI programs launch) than going into thrashing. This requires some coordination between applications and memory allocation, but where most of the memory is used by Apache and its child processes, that's a viable option.

The main value of VM today is getting rid of dead code at run-time. A basic problem with shared libraries is that you load in the whole library, needed or not, when you need any function from it. This wastes memory, but after a while, the VM system will notice the unused pages and quietly release them. On a larger scale, the same problem is seen with dormant applications, a problem which has gotten totally out of hand in the Windows world, where far too much unwanted stuff launches at startup. VM ejects them from memory. That's what VM is really used for today.

So if you're actually page-faulting, VM is hurting, not helping.

I'd argue that it's time to go back to a swapping model - all of an app has to be in before it runs. That's where UNIX started; virtual memory didn't come in until 4.1BSD. But in support of this, apps need more information about the current memory situation. And they should be able to designate parts of their space as pageable, at least at the shared object/DLL level. Only a few apps (web servers, window managers) need much memory awareness, so that's feasible. Throttling needs to occur at a smart place, just before allocating substantial resources, such as CGI process launch or connection opening. By the time the VM system becomes involved, it's too late; resources are already overcommitted.

The big win from this is repeatable latency at the memory level. With all the interest in reducing kernel latency at the CPU level, it's time to address it at the memory level too.

QNX, the real-time OS, is worth looking at in this regard.

Re:Why VM is bad

[Hazzl]

This is a very interesting argument. However, RAM prices tend to be quite volatile (just let another earthquake hit Taiwan like two years ago that would send RAM prices soaring). Moreover, sometimes upgrading RAM is not an option: think about Laptops or embedded systems. One of the greatest features of Linux is that it is possible to run modern software on hardware that is less than top notch. While performance will necessarily suffer it still remains possible. This is what sets Linux apart from the "You can't get there from here"-Attitude of the Windows world.

For instance I am running the latest OpenOffice on my Laptop with 32MB of RAM (plus 64MB of swap) once everything is loaded, the working speed is quite acceptable. This is only possible because I have enough VM, so that large programs like OpenOffice will even load.

Re:Why VM is bad

[jovlinger]

but an unused shared library page will be dropped, not swapped (as it will be clean). In fact, it will likely never be read into memory at all, just mapped.

So it seems that you use the two concepts of swapping and mapping as if they were one, when they are not. It is perfectly feasible to have a system with VM but no swap. This system will of course die at unpredicable times because of [temporary] overcommit that you could have survived with a swap file.

I can't see how you could have a system with swap but no VM, tho. Arguably, this is what overlays and banks do for you.

Re:Why VM is bad

[DaveWood]

Alright, I'll bite. What you say is interesting, and I believe your comments regarding the changing relative costs of traditional VM paging algorithms make sense. The problem is that I suppose I don't understand the alternative you are proposing. I am certain this is due to my own ignorance; please give tolerance to my questions, and don't let my inquisitiveness be mistaken for criticism.

You say, "The price of having virtual memory is terrible performance once paging between active processes starts." Assuming the VM algorithm is working correctly (big assumption lately), this means basically that you are trying to run more than your memory can handle, and have reached a load-shearing point with respect to RAM. From this I surmise that we might be talking about a "smarter" VM system that would shear better, perhaps by identifying the condition, and perhaps by better communication with higher levels - in other words, a different/better application-level interface to the VM system.

And, indeed you say, "On a server which is processing short transactions, you're much better off throttling at the transaction launch point [than thrashing]... This requires some coordination between applications and memory allocation." So I think I understand so far.

Then you say: "A basic problem with shared libraries is that you load in the whole library, needed or not, when you need any function from it." This is where I perhaps display my ignorance of the kernel, but that's not what I have understood was going on. My impression of things was that an application was loaded into memory by mapping its data on the disk into "virtual" memory, and that the VM subsystem arbitrated between real and virtual memory by retrieving from the disk only what blocks were "necessary" (i.e. being referenced by the executing code), and that this process naturally extended to libraries, and especially shared libraries (which need only exist in "real" memory in one location, despite being mapped into multiple "virtual" memory environments). Then again, perhaps it is a minor point - if the whole SO image is loaded and then unused pieces are unloaded or vice versa, it seems less important than the contention problem already on my mind...

You say "VM ejects [unused bits of libraries and applications] from memory. That's what VM is really used for today." Absolutely! But regardless of the relative differences, isn't this process of migrating data between different "tiers" of data storage in the computer (each with a different latency, throughput, and cost/availability) always going to be necessary? While I can certainly see a major advantage in creating/improving ways for the application to communicate with the memory management system, is there really some fundamental alternative to the block-based VM "guesswork" that takes place in absence of directives set at compile time?

You say: "So if you're actually page-faulting, VM is hurting, not helping." I am wondering if the VM is either hurting or helping per se, since the real problem is that you don't have enough RAM even for the "active" blocks you want to run. Of course, the quality of your VM will determine how close you can get to "perfect" utilization of your RAM.

Then you say, "I'd argue that it's time to go back to a swapping model - all of an app has to be in before it runs." This is where you lose me, I suspect because I do not understand what you are really proposing. You go on to say "in support of this, apps need more information about the current memory situation. And they should be able to designate parts of their space as pageable, at least at the shared object/DLL level. Only a few apps (web servers, window managers) need much memory awareness, so that's feasible.Throttling needs to occur at a smart place, just before allocating substantial resources, such as CGI process launch or connection opening. By the time the VM system becomes involved, it's too late; resources are already overcommitted."

At first it sounds as though you are saying that you want to eliminate swap altogether. I do not doubt that for some situations this is preferable - you want to have consistent performance and a sharp failure rather than the long thrash in the case where you use up your resources (and you mention QNX). However for general-purpose computing, I'm not so sure this is a good idea, even with RAM as cheap as it is. Depending on what you're trying to do, the slight loss in predictability and overall performance is vastly preferable to sharp failures for many, I would even say, "most" applications, even on the server.

But moving on, it seems you are saying that what you dislike about the VM is that data is broken into arbitrary blocks - and so we should rely on application programmers to designate what it would be a good idea to swap out in case of memory contention ("designat[ing] parts of their space as pageable"). The problem I see with this is that you are relying on the programmer to do something that, if they do not do it, their program will appear to run anyway.

This is therefore automatically classified a frivolous expense by commercial software developers, and even OS people working for the love of the game may be tempted into the same pitfall. This is superficially similar to the argument between malloc/free proponents and garbage collector advocates. Giving the programmer another "lower-level" thing to worry about gives them an opportunity to optimize it, but in practice we often find that on the balance we get more mistakes and the quality of the user experience suffers.

The compiler probably could be coaxed to do it for you. But the various tradeoffs between compile time "pre-blocking" and runtime blocking might leave compile-time computations, whether in the compiler or even in the developer's head, looking inferior to what a good VM system can do while observing actual behavior in real-time.

Your point about throttling occuring "at a smart place" is not lost - obviously many applications could benefit from more transparency by the memory management system in managing their affairs - apache users really don't want to have to guess how many processes/concurrent users should be allowed, they want apache to determine it for them based on what the system can handle. But most application programmers are not going to do this extra work or do it right, and a VM seems like what you need as a "default behavior," even if its benefits (and its audience - those who have enough RAM that they never need fear swap) are lessening over time.

Re:Why VM is bad

[RelliK]

huh? what? This is the most uninformed garbage I have ever read. I don't have time to refute all of the nonsense, so I'll just take on the biggies.

The price of having virtual memory is terrible performance once paging between active processes starts.

When that happens, you are running a lot more processes that can fit into memory. Without VM you would not be able to do that at all.

A basic problem with shared libraries is that you load in the whole library, needed or not, when you need any function from it.

False. Any decent VM does demand paging. Only the pages that are needed are loaded from the executable. The parts of the program that are never executed are never loaded from disk, notwithstanding read-ahead optimization. A shared library is just an extention of the executable so the same rules apply. Further, a shared library can be used by multiple processes and only *one* copy of it is loaded into memory.

I'd argue that it's time to go back to a swapping model - all of an app has to be in before it runs.

That would be absolutely stupid. It would slow down the system tremedously. Se above about demand paging.

Without VM, you would need to increase the memory requirements by a factor of N, where N is the number of processes running concurrently. Further, the startup time of each process would always be slower since all of the code would have to be read in memory. With VM part of it is already there (shared libraries), and the code is loaded on demand.

In short, this is the biggest pile of uninformed garbage. You *really* need to take an OS course before you can talk about OS design.

Re:Why VM is bad

[mickwd]

I don't agree with your point about removing virtual memory from the operating system, but I am surprised that more people using Linux in a server role just don't buy enough RAM to ensure that the minimum amount of swap is actually required.

Desktop use is a different matter - many people who use Linux on the desktop don't have a lot of money to spend on extra hardware (including RAM), and Linux should be able to run decently on as wide a range of hardware as possible (including cheap, old machines).

But for server use, if you can afford the price differential of SCSI over IDE (which you should, for any server supporting any serious load) you should also be able to afford enough RAM to deal with most loading situations.

Re:Why VM is bad

[Johnno74]

No, sorry but I dissagree.

A clever VM system is more important than ever, when you combine it with an effective disk cache.

Yes, Ram is cheaper and faster than ever, but stopping your OS from using a swap file/partition is gonna stop your OS from efficiently using your ram.

Your machine should be allocating as much memory as possibe to a disk cache, even if this reduces the available memory to the extent that active processes start paging, because the swap file paging is optimised by the disk cache too.

It is usually better to swap out pages and keep your cache large than to keep rarely used pages in memory at the expense of cache, because even if you need those pages they will quite likely be in the cache.

Even if you have a shedload of memory (especially so!) you will get better use of your memory if you use some of it as a disk cache than if you don't page rarely-used pages to disk.

Re:Why VM is bad

[Anonymous Coward]

> I don't agree with your point about removing virtual memory from the operating system, but I am surprised that more people using Linux in a server role just don't buy enough RAM to ensure that the minimum amount of swap is actually required.

Alan Cox used to recommend that swap = 2 * RAM for the older 2.4 kernels, so buying more RAM meant needing more swapspace.

Re:Why VM is bad

[slashdot_commentator]

Its not "uninformed garbage", its quite insightful. You're confusing your OS textbook with not-widely-known real world practices.

You also completely ignore a negative with VM; the increased latency in kernel operations (because its screwing around with VM operations).

If companies are pumping away money in hardware resources to prevent cycles lost in paging, there is no need for a sophisticated VM. The VM becomes CPU baggage. The practice of hardware overkill is sometimes referred to as "running in RAM".

One still needs swap to help keep their butts afloat, in case they guess wrong and have too high a request threshold. Swapping will SIGNIFICANTLY slow down the machine (compared to VM), and that is OKAY. It means you need more hardware.

I'm not willing to abandon a Linux VM, because the old swapping scheme requires sharper humans in specialized, tuned purposes. One is better off having a VM in a general purpose environment. You should not be so rude when disagreeing with ideas; it can betray your own ignorance.

I am also not convinced that there are significant gains in a non-VM kernel, not having any numbers in front of me. Perhaps what the original author should do is develop an efficient, primitive memory manager to patch into the 2.4 kernel. And then we could run some benchmarks on it.

Re:Why VM is bad

[Anonymous Coward]

This is just beautiful. You must be some kind of fucking genius.

Re:Why VM is bad

[puetzk]

This was a rule of thumb, because under most ordinary loads it was all the swap the system could ever make any reasonably use of. More RAM (for the same workload) means less swap needed - but if you're going for maximum peak worload you want as much VM space as possible. However, Anything much beyond 2xRAM makes no sense - the system will pretty much totally disintegrate if you've got that that much more data than RAM. Hence the recommendation - that's about as much swap as 2.2's vm could really make use of.

Re:Why VM is bad

[Snowfox]

The main value of VM today is getting rid of dead code at run-time. A basic problem with shared libraries is that you load in the whole library, needed or not, when you need any function from it. This wastes memory, but after a while, the VM system will notice the unused pages and quietly release them. On a larger scale, the same problem is seen with dormant applications, a problem which has gotten totally out of hand in the Windows world, where far too much unwanted stuff launches at startup. VM ejects them from memory. That's what VM is really used for today.

So if you're actually page-faulting, VM is hurting, not helping.

That one-liner made no sense on the tail of the previous paragraph.

Apps are getting larger. A fat chunk of most modern GUI software is dead code, error checking code, or code that only gets used during startup and teardown.

Apart from application code, handling huge datasets also benefits from VM, leaving the most often used parts in memory. Do you really want 2G of memory just to lay out a multi-layer magazine-resolution image in gimp?

That said, if you really want your wish - just set VM page sizes to be larger than your largest application and have a look-see at how performance is affected.

Re:Why VM is bad

[be-fan]

QNX sucks as a desktop OS precisely because it lacks a decent VM (and a decent FS, but that's another can of worms...)
A VM doesn't make the system any slower. The cost of maintaining mapping data and such is negligable compared to the other startup costs associated with applications. Also, most VMs do very little until the system memory becomes stressed. FreeBSD, for example, won't really bother keeping page ages up to date unless there is a lack of available pages. As long as you're not swaping, a VM-based system is no slower than a non-VM system. By the time you're swaping, however, a VM system has a lot more active processes (since the working set of each process is smaller than the total image) than a non VM system. Also, a VM system can deal with temporary demands for a huge amount of memory (such as compiling, in which the disk is getting trashed anyway), while a non-VM system can't. In fact, the QNX developers had to hack GCC to use their swap extensions because it wouldn't run on any sensible RAM config without VM.
You're comments about RAM are also questionable. While RAM might be getting cheaper, the uses for RAM are increasing. The more RAM you add, the more uses you can find for it, such as rendering bigger images, etc. There is no point in wasting a significant amount of it when you don't have to.
Lastly, the comments about process swaping are completely off the wall. App developers are inherently lazy, which is why modern systems take as much responsibility away from them as possible. If you left it up to the application developers to manage memory, then you'd end up with gigantic (more so than now, if that's possible) runtime footprints.

Re:Why VM is bad

[kma]

> Its not "uninformed garbage", its quite insightful.

No, it's uninformed garbage. Animat might have had a point if he was advocating doing away with paging to backing store. Indeed, most modern operating systems treat situations that heavily use the backing store as pathological; they only care that they work, not that they work well or fast.

But, virtual memory is still a boon, even in these days of cheap RAM. I've even written an instructional operating system that uses virtual memory, but doesn't page to backing store. Why? So you don't need to relocate executables when loading them; for shared libraries; to demand-paged executables; to make integration with the buffer cache easier; to support the mmap(2) system call in all its richness; etc. Animat is ignorant of these advantages ("uninformed"); he might also be willfully ignoring them in order to make a more bold-sounding, moderation-friendly post ("garbage").

Re:Why VM is bad

[SurfsUp]

Thus, the relative cost of a page fault has increased by a factor of 200. This makes VM far less attractive today than it used to be. It's not getting any better, either.

You completely failed to think about disk caching and memory-mapped files, your view of VM is way too limited, and that's how you argued yourself to an incorrect conclusion.

Re:Why VM is bad

[jelle]

I guess you speak another language than I do.

In my book, swapping is one of the paging processes in the VM function of an OS. With that definition, your post doesn't make sense.

Care to elaborate?

Re:Why VM is bad

[slashdot_commentator]

Sorry, my bad. I believe this is the offending statement:

Swapping will SIGNIFICANTLY slow down the machine (compared to VM), and that is OKAY. It means you need more hardware.

Change to "Swapping operations will SIGNIFICANTLY slow down the machine (compared to a demand-paging VM management scheme), and that is OKAY."

The context was about paging VM management schemes vs (older or alternative) application/process-swap memory management schemes.

Re:Why VM is bad

[Salamander]

The saddest thing here is that such garbage got modded up as "informative" instead of being modded down into oblivion for being "uninformed".

It's not that expensive to double the size of RAM today. It can be cheaper than adding a fast disk drive just for paging.

Even with RAM at US$0.25/MB or less, that's still untrue. A top of the line hard disk is under a penny per megabyte, which represents a pretty significant difference.

disks still top out around 10,000 RPM, making main memory 300,000 times faster than backing store

Where the hell do you get 300K from? Picking the first drive I see at an online store I see an average seek time of 4.9ms (RPM is totally irrelevant). 4.9ms/0.02ms = 24.5, not 300K. Yeah, yeah, you have to factor in transfer times as well, but 300K is still way out of line.

On a server which is processing short transactions, you're much better off throttling at the transaction launch point

Not all workloads are characterized as short, independent, stateless operations. Yes, there are applications out there that aren't webservers, and they deserve to be well supported by the OS as well.

The main value of VM today is getting rid of dead code at run-time.

Bullshit. But I see others have already addressed that.

I'd argue that it's time to go back to a swapping model - all of an app has to be in before it runs.

So, because the mismatch between RAM and disk access times is so great, it's better to move all of a process's address from RAM to disk and back again instead of just part? Riiiiight.

As someone else said, please take an OS class. While you're at it, read Hennessy and Patterson. You seem to be missing a lot of information about how computers actually work at both the hardware and software levels.

Re:Why VM is bad

[Salamander]

4.9ms/0.02ms = 24.5

Before anyone else points it out, that should be 245. I don't know how that period got in there; I guess it's something I should watch out for when I post at 3am. In any case, this just means that the previous poster was off by three orders of magnitude rather than four. Congratulations. :-P

Re:Why VM is bad

[jelle]

Thanks, that makes it a lot clearer.

And actually, I think now that your last statement about the primitive mm patch for the 2.4 kernel, does sound like a good idea.

His first BSD vs linux comaprison

[Anonymous Coward]

Well, I read his first article and I switched to FreeBSD, so I don't care about those flame wars anymore :)

Linus's choice

[RichiP]

While I normally take issue with the way Linus bullies a kernel issue based on what he perceives is technical merit, I have to agree with him on this one. First, if a better way of doing things has been found, even though it's in the middle of a stable series, it should be changed in order not to propagate wrong coding. I've been coding for a long time now and I still believe that if an error, bug or better coding scheme is found, it be implemented as soon as possible.

The problem with leaving the change till the 2.5 series is that the 2.5 series is nowhere in sight and development kernels usually take more than a year to cycle through (no matter what the kernel hackers say). The fact that 2.5 hasn't even begun may be an indicator of how long 2.5 will take to finish.

it's been ac kernels

[MenTaLguY]

RedHat ships an -ac kernel with RH 7.2, I think 7.1's was also an -ac kernel.

Not pure -ac kernel, though, like most major distributions they also pull stuff from Linus and other kernel trees (there are others) so what they actually ship is really the "RedHat" tree.

unstable kernel, sigh..

[TheGratefulNet]

as a 5+yr linux vet, I'm horrified at this turn of events. I've always counted on linux to be rock stable, yet the last few months have been anything BUT stable.

I really hate to say this, but I'm wondering if jumping ship to freebsd (etc) makes sense. I've been a major linux supporter for quite a long time, but I know that the *bsd guys have had their act together (good smp, good networking under load, etc) for a long time.

would it be all that crazy to adopt the VM system from the 'establishment' (bsd)? frequently the linux codebase DOES borrow from bsd. why is the VM system all that different?

Re:unstable kernel, sigh..

[nathanh]

I really hate to say this, but I'm wondering if jumping ship to freebsd (etc) makes sense. I've been a major linux supporter for quite a long time, but I know that the *bsd guys have had their act together (good smp, good networking under load, etc) for a long time.

*BSD has poorer SMP than Linux. Networking is arguable (some benchmarks show Linux is faster, some don't).

*BSD has also had more than it's own fair share of infighting. Do people forget so quickly why we have *BSD instead of 386BSD?

Go use *BSD if you want to, but do so for the right reasons, not because of rumor and myth.

Re:unstable kernel, sigh..

[TheGratefulNet]

BSD has poorer SMP than Linux

not to start a flame war, but things like netbsd have had to be VERY smart about smp (and multiprocessing, in general) for a long time, simply due to the fact that they deal with much more hardware types than we (linux) do.

freebsd is known by most to be more stable at high network load. hell, its got some 15yrs or so on linux, it has to be more mature simply due to all the time-tested code.

and don't even get me started on nfs. nfs on *bsd has a hope of working. I'd not try nfs on linux unless I like hangs, stale handles and general unpleasantness.

Re:unstable kernel, sigh..

[AntiBasic]

I've noticed a recent trend towards trashing FreeBSD's SMP because of "the giant spinlock." What people don't realize is that one large spinlock can be a viable method of locking for the purposes of threading (that is, multiprocessing). It would seem that someone who has a moderate clue about threading and writing SMP-capable operating systems has commented on this, and feels it's bogus, and one or more of the general breed of "BSD is ubersux" trolls has gotten a hold of this and thinks it's the ultimate death knell for FreeBSD/smp. Obviously, you don't really know much about locking at all. It should at least be pointed out that no matter how many locks you have, it is more important to keep the system OUT of a locked state as much as possible, and FreeBSD does this well enough. It's not as if the system is constantly locked and able to use only one CPU. Most processing occurs in userland, far away from kernel locks, so it doesn't tend to matter all that much. Now, granted, using one spinlock isn't necessarily the best way to do things, at least not in an OS. However, it's not the worst either. Combined with the fact that it allowed fairly rapid updating and deployment of FreeBSD/SMP, I think the choice to use that 'giant spinlock' was valid. It allowed SMP code that by all accounts worked better at least than the 2.0 Linux kernel's (if not 2.2 as well) to be deployed until a better solution could be created.

A better solution will be deployed in FreeBSD 5.0 with the introduction of SMPng. I do not doubt that the 2.4 Linux kernel does a better job at SMP than FreeBSD (release/stable) does, but I think it's worth noting that Linux's SMP has been now five or six years in the making to get to this point, and that the Linux and FreeBSD development and advancement models are significantly different. Where Linux takes gradual steps, FreeBSD (and BSDs in general) tend to take large leaps. That's just a difference in implementation timing. Furthermore, it's perfectly reasonable to expect two open-source systems to leapfrog each other in terms of capability as ideas and code move from one to the other, and it's really not something to gloat over. What one does better today, the other will do better tomorrow. It doesn't really matter. To those of you babbling on and on about 'the giant spinlock', you might want to go do some research into the theory, and practice, of implementing locks in threaded systems. Until then, shut up, please.

Re:unstable kernel, sigh..

[Anonymous Coward]

re SMP: NetBSD right now has development only SMP code that works on the big-ass-lock model, like Linux 2.0.x and FreeBSD 4.x. This is not usable in any real production environment, as it wastes your second+ CPUs unless you are running 2+ 100% compute bound tasks. Linux 2.2 destroys all the freely available BSD's SMP wise, and Linux 2.4 competes very well with BSD/OS.

re Networking: Linux 2.4 has scored some very impressive, world-beating SPECweb numbers. All we get about BSD is anecdotes. My own anecdote is that running apache, the difference between FreeBSD and Linux 2.4 on my hardware is lost below the noise floor, performance wise. I've never had any reliability problems with either. The networking comparison might have been valid in the Linux 2.0 days, but not today.

re NFS: I will never use a NFSd that includes a lockd that simply tells every client that the lock is available. Its one thing not to implement something, but to implement it in a way that is 100% guaranteed not to work (obviously) is not acceptable.

Re:unstable kernel, sigh..

[Anonymous Coward]

FreeBSD 4.x grabs the BAL (big-assed-lock) whenever it is in the kernel.

You make comments like:
"It should at least be pointed out that no matter how many locks you have, it is more important to keep the system OUT of a locked state as much as possible, and FreeBSD does this well enough."and berate others for talking when they don't know what they're talking about. Might I suggest you take your own advice? Making more locks that cover smaller areas is EXACTLY how you "keep the system OUT of a locked state as much as possible."

If subsystems have their own, smaller locks, I can have one process, on one CPU, in one part of the kernel, while another process, on another CPU can be in a seperate part of the kernel. With a big lock, whenever one process is in the kernel, no other process can get any in-kernel work done.

Take the case of a webserver. The vast majority of the time is spent in the disk IO code and the TCP/IP stack. Very little time is spent in the userspace code. FreeBSD will lock apache process A out of queuing any disk reads, simply because apache process B is currently in the TCP/IP stack writing to a client.

So if you're doing parallelizable, exceptionally compute bound tasks (scientific computing, render farms) FreeBSD 4.x's SMP is very reasonable. For servers, it is anything but.

Cursory inspection of whats going on in FreeBSD-current suggest that it will be superiour to Linux 2.2 in terms of lock granularity (and hence potential concurrency) but not really matching Linux 2.4.

Scheduler activations (FreeBSD calls them KSEs) are a much more interesting bit of FreeBSD-current than SMPng.

Re:unstable kernel, sigh..

[spacey]

Don't be too dissapointed, its happened before. The release of 2.0 had massive issues, and almost daily patches to bring it stable. I still recall installing a kernel that caused apache to trigger an oops.

-Peter

Re:unstable kernel, sigh..

[spacey]

*much more hardware types then we (linux) do.*

This is debatable. The mainline netbsd may deal with many more, but its not like it's 5 or 10 times more afaik. AFAIK both the linux and netbsd kernels have been ported to most embdedded controllers, many makes of cel phones, networking gear, etc.

Plus linux does mainframes, remember?

-Peter

Re:unstable kernel, sigh..

[nathanh]

... netbsd have had to be VERY smart about smp (and multiprocessing, in general) for a long time ...

Read the NetBSD feature list by platform. Only 5 platforms have any NetBSD-support for SMP at all and not one of them is stable or usable. Four of the five have "spinup" support only.

http://www.netbsd.org/developers/features/

Myths and rumors.

Re:unstable kernel, sigh..

[nathanh]

I do not doubt that the 2.4 Linux kernel does a better job at SMP than FreeBSD (release/stable) does...

How is this different to my comment of "BSD has poorer SMP than Linux"?

Obviously, you don't really know much about locking at all.

I didn't make any comment about locking.

Most processing occurs in userland, far away from kernel locks, so it doesn't tend to matter all that much.

Really?

To those of you babbling on and on about 'the giant spinlock', you might want to go do some research into the theory, and practice, of implementing locks in threaded systems. Until then, shut up, please.

I suggest you practise what you preach.

Re:unstable kernel, sigh..

[SurfsUp]

would it be all that crazy to adopt the VM system from the 'establishment' (bsd)? frequently the linux codebase DOES borrow from bsd. why is the VM system all that different?

BSD doesn't support >4 Gig memory on IA32, for one thing. Yes, Linux's VM could be more similar to BSD's and it would probably be better that way, but there is no way it can be identical.

Re:unstable kernel, sigh..

[Anonymous Coward]

*BSD has also had more than it's own fair share of infighting. Do people forget so quickly why we have *BSD instead of 386BSD?

Uh, 386bsd developed into Free because it was too large to just be a patch.

Net evolved from 4.4bsd.

You're a moron.

Re:unstable kernel, sigh..

[nathanh]

Uh, 386bsd developed into Free because it was too large to just be a patch.

The name change to FreeBSD was because Bill Jolitz refused to handover the project and would not apply patches. The three developers of Free were going to call it 386BSD 0.5 until Bill went back on his earlier approval of the patchkit.

NetBSD was sprung from 386BSD after a bunch of developers expressed frustration with the slow pace of 386BSD. The first NetBSD release was a combination of Net/2 and 386BSD as a port to the Macintosh.

And OpenBSD forked from NetBSD because of disagreements between Theo and pretty much everyone else. Theo is still disagreeing with everyone else. He's famous for it.

I think that pretty much proves my point that *BSD aren't exempt from infighting.

Net evolved from 4.4bsd.

It evolved from Net/2 and 386BSD. If you'd bothered to read the netbsd.org site you'd have learnt this too.

You're a moron.

You're ignorant.

2.4.13

[josepha48]

2.4.12 has bugs that prevent certain compile options. 2.4.13 is out and it works really sweet.

Oh an djust becuase there are two different vm's (Alan vs Linus) does not mean that there is an official fork. As well there is already companies like Redhat and Suse that release their own patches to the linux kernel that make them unpatchabe against the main tree.

REALLY

[Anonymous Coward]

Real professional site you got here, Taco-man

The word you're thinking of is "really".

What about the AIX VM?

[Sara Chan]

The discussion so far has focussed mainly on Rik's and Andrea's VMs. For the 2.4.x series, that's fair. For 2.5, though, what about considering the AIX VM?

IBM has said that they will open source any part of AIX that we would like. The AIX VM works well under high stress. Obviously it could not just be put as-is into Linux, but there must be a lot of good ideas/algorithms in it that could--arguably should--be moved to Linux. Why isn't anyone looking at doing this?

Re:What about the AIX VM?

[Anonymous Coward]

The AIX VM is bizarre and different - it is almost entirely unlike any other UNIX VM out there. It is hideously ugly, MP support was added as an afterthought and looks like it was written by a pascal programmer.

It also relies heavily on a segment registers based architecture i.e Power/PowerPC (each segment describing 256MB chunks of virtual address space). You start getting into lots of fun when hitting/crossing segment boundaries.

I have some doubts how well this maps to/performs on non-segment based architectures. IBMs inability/unwillingness to put an AIX product out on the Itanium after some heavy investment *may* be related.

Linux Kernel list link

[commanderfoxtrot]

This is a link to the kernel-traffic discussion with details and basic benchmarks: here!.

Re:The failure of Open Source

[Kymermosst]

I'd like to bring up a small example that the SAME thing can happen in your happy little closed-source world, too. I can sum it up like this: Windows - OS/2. Windows happened to be the one that succeeded.

Furthermore, when one authority is in the position to determine where progress goes, people tend to lose their jobs or otherwise be eliminated from the project for having a different philosophy. Rather than having two parts of the effort working on two projects, therefore being 100% productive, you have part of the effort working on one project, therefore not being 100% productive.

But, my real question for YOU is, did you just get up on the wrong side of the bed today? Your lover refuse to have sex with you? Or are you paid by Microsoft/Oracle/whatever to bash the open source development model every chance you get? Away with you, FUD!

Re:ok, here's the thing - threads

[ProtonMotiveForce]

Another area of Linux with problems (albeit mostly non-kernel related) is the threading model. For Christ's sake, grow up. "We can do kernel context switches fast enough for 1:1 threading model!". Pure hubris. Linux is unusable for any serious Java development.

Core design issues.

[bored]

Part of the problem with the design and redesign of the linux VM is an insistance with sticking with a few core design points that make it 100x harder to write. For instance, virtual memory overcommit spawns a whole bunch of ugly problems that must be solved in order to create a stable and fast system. If the core development team spent some time looking at past OS research then they would completly change their design criteria and a bunch of these problems would go away.

Another perfect example is the OOM killer. If the VMM could properly balance the workload (and it didn't overcommit) then there wouldn't be a need for code to select the 'correct' process to kill. Since the VM cannot balance correctly, the kernel developers spend massive amounts of time trying to write an OOM that functions correctly in the case where the VMM is wedged. This time would be better spent fixing the VMM so it never got into these states.

Re:Core design issues.

[AndyS]

You can't avoid overcommit in a usable system.

For example, say I run netscape, and I want it to launch RealPlayer (to play a stream). It then needs to launch realplayer. The only way it can really do this (portably btw) is fork(), and exec().

fork() means that you have to account for the same memory space the program takes up if you want to be overcommit free. There's no equivalent to ExecuteProcess() (or whatever it's called), this is the unix way[tm].

Now, this blows, but it's life, I have lots of forked copies of apache running, and they're all about 2Mb. But, combined I bet you they're taking up far less. However, I would have to allocate space for them otherwise.

I'm personally a non-fan of overcommit, and would like to actually switch it off, but for your average desktop machine, it isn't an option, it requires a *lot* of memory, or stuff will fail, despite not doing anything big.

MS-DOS VM

[RadioheadKid]

Why not just use the 640k limit and we wouldn't have any of these problems...

KidA

Comparisions to NT/XP VM architecture ??

[Anonymous Coward]

I would've thought there would be many posts comparing and contrasting (and most likely flaming) the Linux and Windows' VM architectures. As a primarily Winddows user, this discussion is highly interesting and I would like to learn more about how these issues (VM page allocation, thrashing, kernel pre emption) are handled in NT land.

Re:Comparisions to NT/XP VM architecture ??

[NDPTAL85]

How would we know? Windows isn't open source remember? Therefore only a precious few get to look at how its actually built/made/constructed.

Re:Comparisions to NT/XP VM architecture ??

[jpmorgan]

Err, yes, NT might not be open-sourced, but it's actually pretty well documented. Remember one of the most important laws of software engineering: for any project of reasonable complexity, poorly documented source-code isn't a substantially more usefull than just a compiled image. If I'm not mistaken, NT5's VMM model is pretty complex.

Last I checked it relies heavily on the concept of working sets and its paging algorithm relies not only on statistics about memory accesses, but also about the processes that own pages (it's more likely to page out memory for a process that isn't likely to grab CPU time again for a while than one that's currently running, or will probably be running soon), proactively tries to clear pages out of memory to prevent thrashing and, if it detects thashing, auto-tunes various system and process parameters to try get everything back under control. Does all this actually work? Well, YMMV. :)

Remember, Linux can't just blow past Win9x on the desktop nowdays, it's going to have to compete with the NT 5 system executive on the desktop. You know, the windows kernel that actually...well...works, not that POS kludge.

The tech front may be fairly mute these days, whatever the cause, but the competition is tougher and more intense than ever; hopefully once this disagreement is resolved Linux's VMM will be better, shinier and faster. Ain't competition grand?

For those who are interested in understanding what this discussion is all about, and what the all the terms mean (you know, all that unimportant stuff), Tannenbaum has a fairly good introduction to VMM in 'Modern Operating Systems', and the 2nd edition includes case studies of both Unix&Linux and Win2K (I'm not sure which version of Linux he's looking at though); I'd be shocked if any decent computer bookstore or university library didn't have at least one copy of this. It's a fairly good read for anyone interested in the mystical world of ring 0 (eww...x86ism), and is the kind of thing that's usefull to have read before getting into a discussion of vmm and other complex kernel issues.

2.4 kernel has finally grown up with 2.4.10

[Shanep]

If he says so.

I have never experienced such "system freeze" swapping with Linux, since I started about 4 years ago. Felt like my Win95 days with 8Mb RAM were back.

With 256Mb RAM, I only need to open up a ~1Mb pdf with Acroreader to experience some extreme disk thrashing that takes an age to give back my system. Something I've never seen happen with Linux.

Re:2.4 kernel has finally grown up with 2.4.10

[Jetson]

I have never experienced such "system freeze" swapping with Linux

My laptop (128Mb) did it all the time when I was running 2.4.4 through 2.4.7 or thereabouts. The problem invariably cropped up when I was closing an application under X. I solved the problem by disabling swap completely for a few kernel versions.

Re:2.4 kernel has finally grown up with 2.4.10

[Shanep]

George?

Actually, as you mention, it's when I close Acroreader that the trashing begins.

From what you say, what I've heard from others and from that article, it seems both VM systems are kinda sucky in the 2.4 kernels.

If FreeBSD supported RAID-0 with Vinum for / and /usr, I'd move completely over to 4.4 Release. If the Linux kernel does'nt get the VM and software RAID sorted out before the next major release, then I might upgrade my PII-300/BX using Linux software RAID-0 to maybe an AMD with hardware RAID-0 and move to FreeBSD that way. Software RAID-0 in Linux is also pretty sucky with 2.4.x, I get a reduction in speed over the single drives! : (

One would think that the VM

[Anonymous Coward]

would be hidden behind some interface, and that it's just a matter of plugging in a different implementation. If it's that difficult to switch between the two, something's badly broken in the design.

But what do I know, I'm just another Slashdolt, who couldn't code his way out of a paper bag if I tried.

I want to hear more about music videos, and computer games.

Re:One would think that the VM

[V.P.]

The VM is hidden behind such an interface, so that you can plug-in different implementations easily, without breaking your code.

People call it the "user/kernel interface", or just "system calls".

VM? Do we need it?

[bcboy]

1G ram is under $100. I just turned the VM off. Some folk I know that do large clustering solutions are doing the same thing. Who is using VM these days? And is it really worth the headache, rather than spending a few dollars?

Missing Include In His C Program

[Hornsby]

I guess he forgot to put his C program to stress test the kernel inside of a block. Anyway, the #include line is missing a header file to include. It's supposed to read: #include Hope this helps.

Then Again?

[Anonymous Coward]

Gosh! When are you Slashdot people going to learn the difference between the words "then" and "than"? Learn proper English!

Re:The failure of Open Source

[Anonymous Coward]

Truly you have been brainwashed. Anyone criticizing Open Source does so because:

But, my real question for YOU is, did you just get up on the wrong side of the bed today? Your lover refuse to have sex with you? Or are you paid by Microsoft/Oracle/whatever to bash the open source development model every chance you get?

Why can't you accept that Open Source is not perfect? Why can't you learn not to respond to trolls? Foolish child.

Slightly OT question

[mindstrm]

They mention LIDS (Linux Intrustion Detection System).

My question is..
does the LIDS actually do *any* intrustion detection, or does it just prevent modification of certain files?

Re:Slightly OT question

[AaronMB]

Note: I use LIDS at work, but am not an expert in it.
LIDS among other things can be used to protect or hide files(which is a big reason why we use it at work), protection of processes(so an intruder couldn't kill necessary processes), allow certain programs to have certain capabilities(e.g. servers can bind to protected ports but can't do anything else), etc.(note: this is not an exhaustive list, its just what we use at work). However, it likes to send mail to the other administrator whenever ANYTHING violates those rules. So, if a user did not know you were running LIDS and decided to do system modifications, you'd get a nice email about it, and it would prevent it from happening. Also, it can detect port scans and will send email to the administrator, and a number of other things. However, as to being an IDS like Snort, it is not.
-Aaron

Your absolutly right

[Anonymous Coward]

AIX's VMM assumes way to much about the underlying arch of the machine its running on. Combine that with the fact that it doesn't have a hardware abstraction layer and large pieces of the VMM drops to assembly. The result is something that would be absolutly useless for linux other than maybe as a case study of how not to write a VMM.

Re:ok, here's the thing - threads

[Anonymous Coward]

Wrong. The problem is that Java (at least pre-1.4) is unusable for serious network programming due to its stupid one (or two, depending on the situation) threads per connection model.

There is a reason that professional Java software like BEA weblogic and Websphere use JNI networking layers written in C instead of the brain-dead java.io/java.net APIs.

The linux kernel's threading support is optimized for software not written by absolute morons: software that has roughly the same order of threads as CPUs.
Get a language that doesn't suck (even if that language is Java 1.4, I have no favorites here), and stop whining because other people won't make well written software run more slowly to help out your broken software.

The Byte I knew and loved

[TOCie]

My father kept all his old Bytes from the late 70s and early 80s... I used to spend my afternoons going through them, one by one, reading the technical details and discussions...

... items that were absent when the paper publication finally stopped a few years ago.

That article is exactly the kind of thing that would have fit right in with the old Byte. I hope they continue such things in the future.

Kernel Architechture?

[bucky0]

Hey,

I'm a newbie to linux kernels, and I dont understand a lot of the termonology flying around. I.E. What is a spinlock? What's the difference between a dirty page and a clean page? What is a page? Anyhoo, is there a page that explains some kernel terminology and maybe gives a conceptual map of it?

Thanks-
Buck

Re:Kernel Architechture?

[NDPTAL85]

Google is your friend. Enter any term you want into google and then dig thru the links it returns. Start with "kernel spinlock" or "spinlock" or "Linux VM" or any combination based on what you wanna know.

And what about BSD?

[Ian Bicking]

Moreso than AIX, people have been reading the BSD VM code for a long time. It seems to be regarded very highly, and its design has been stable for quite some time.

So why doesn't Linux just copy BSD?

The code here seems rather incidental, it's the design that is more important. But why not copy a good design? Or do one (or both) of the contending VMs do so?

Re:And what about BSD?

[puetzk]

Rik van Reil's (-ac series kernels) is somewhat modeled off the FreeBSD VM. How much so I don't know.

Re:And what about BSD?

[SurfsUp]

Rik van Reil's (-ac series kernels) is somewhat modeled off the FreeBSD VM. How much so I don't know.

Loosely

VM is good :-)

[jawtheshark]

I was thinking exactly the same when reading the parent post. RAM upgradability of systems like laptops.
I'm posting this from a 5 year old Toshiba laptop: P120/32Meg RAM ( 64Meg swap indeed) running Linux. It does everything I need, and I just don't see a reason replacing the machine. Linux saved one of two years of extra lifetime for that machine, and I saved some cash :-) (which ended up in other hardware, but that's another story)

Strangely enough a lot of people tell me to buy a new one because it is "obsolete"....those are Windows users, yup, no kidding. Obsolete? I can play MP3's while working (okay, I admit, in mono...)

I didn't even consider trying OpenOffice on this machine....now thanks to your post, I'll give it a shot! I really start to wonder why I bought a huge desktop machine. Besides, that one has 768Meg RAM and Windows 2000 still uses swapspace. I don't know why: it has more than enough memory available. It probably just swaps out some unused DLL's.

Re:The failure of Open Source

[Kymermosst]

Actually, it's because I did get up on the wrong side of the bed today, and felt like being pissed off about something. That comment struck me as a something.

I am aware the open source development model isn't perfect, but I am also aware that neither is any commercial development model-- portraying as such (either expressed or implied) is a lie, and that is what I was responding to, not the trolling aspect.

Though you can fault me for a little flamebait, you cannot fault me for anything else. I didn't try to portray the open source development model as perfect, I merely pointed out that commercial devlopment processes have the same flaws.

Re:Trouble with 2.4 .x Kernels with heavy MP load

[parabyte]

We are experiencing a lot of stability problem with our applications, and I am very suspicious that they are kernel related. We switched back from 2.4.12 to 2.4.8 because it is much more stable for our apps.

I am not a kernel hacker, but I make heavy use of many of my system's resources. My target systems have a least 2 Processors, 2GB of RAM, the lastet NVIDIA GPU and fast SCSI disks, and I try to render 3D scenes fast, utilizing every processor, using a lot of IPC, while asynchronously loading hundreds of megabytes per minute from disks and the network.

However, everything I program runs outside the kernel, as a user mode process, and my understanding of one of the main tasks of the kernel is to isolate (misbehaving) user processes, but it seems to me it was never so easy to take out my LINUX-boxes as it with the current 2.4.x kernel series; my app runs fine for days, but after I quit it there is a 50/50 chance the system survives. I get kernel oopses or the system freezes completely; sometimes just the X-Server hangs; when the system continous to operate, sometime the gcc freaks out with internal compiler errors, which disappear after reboot. And I had to switch from Reiserfs to ext2 because of serious filesytem corruptions, and NFS is even more troublesome than usual under LINUX My system logs are full of kernel messages like:

Sep 25 15:02:21 dellomat kernel: Unable to handle kernel paging request at virtual address 002200dc

Sep 25 15:02:21 dellomat kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000000

Oct 30 18:13:38 dellomat kernel: kernel BUG at page_alloc.c:81!

Call Trace: [__delete_from_swap_cache+126/132] [__free_pages+27/28] [delete_from_swap_cache_nolock+106/108] [free_page_and_swap_cache+115/196]

I am not absolutely sure I can blame the kernel, but I can rule out the hardware, as the problems appear on at least five different platform on a about ten systems. And the severity of these problems varies with kernel versions; the most stable we currently use is 2.4.8, and 2.4.12 was almost unusable for us. Before reading the article, my favorite suspect for causing these problems were the NVIDIA drivers, but now everything points to some vital parts of the kernel causing these problems. I do really appreciate the great work many people are doing on the kernel, and I think I can't criticise someone for doing me a favour by working on a free OS, but I am really concerned that all this might seriously endanger the only alternative OS that runs one a broad range of state-of-the-art hardware.

I hope that there are not too much people having the same trouble, and I will get rid of these problems without becoming a kernel hacker.

While writing this I became aware of the "Linux Test Project", and I am going to download and run a bunch of tests now; I never thought this would be necessary as I do not do kernel development, but is seems to be the best thing I can do towards getting rid of the problems. (Except whining on slashdot, of course ;-)

p.

Overcommit is _NOT_ needed

[bored]

You can't avoid overcommit in a usable system.

This is absolute BS. Most OS's don't allow overcommit. Linux is one of the few that does. Basically what this means is that Linux allows an application (or the total system workingset) to exceed the amount of ram+pagefile available. This is completely unacceptable. When an application comes up and asks for memory that is being consumed it should have its memory allocation fail. That is why malloc has the option to return NULL. If it returns NULL then the current application has to deal with it. If it cannot continue then it needs to write to a log and exit. Otherwise the kernel is forced to choose (very ungracefully) what process gets killed when it is unable to find space to swap a page out to get the required page into memory. This is VERY BAD!!!. Sometimes the only pages in RAM are critical to the operating system. Say for instance only kernel pages and pages for init are in memory. Now which do you kill? Hmmm pretty bad hu? Its not a solvable problem. This is why it should be avoided. This discussion has come up in the past but the base kernel developers always say something like "well at that point your system is pretty much screwed up already". Duh! but its not a fatal system reboot. The system that isn't overcommited can unwedge itself with proper scheduling and memory management given enough time, Linux has to reboot. Ever see W2k pop up a dialog that says "Running low on memory increasing page file size"? Thats not because the machine is 'low' on memory. In fact the whole working set may still fit in memory. What that means is that an application just asked for more RAM than will fit into the pagefile if the OS needed to swap it completly out.

Re:Overcommit is _NOT_ needed

[spacey]

Overcommit is an option.

Try:

$ cat /proc/sys/vm/overcommit_memory

The default is 0 now, which means no overcommit. So its at the users discretion.

-Peter

Re:Overcommit is _NOT_ needed

[Jetson]

When an application comes up and asks for memory that is being consumed it should have its memory allocation fail. That is why malloc has the option to return NULL. If it returns NULL then the current application has to deal with it.

That assumes a certain amount of diligence on the part of the programmers (both application and kernel). Every program would have to check every return value (something that isn't done often enough, judging by the number of string buffer overflow exploits), and be able to exit to a recoverable state without requiring any extra memory from the malloc() onward. It's an ancient and ideal proposal that apparently doesn't happen with any regularity.

Perhaps a short-term solution is to allow an overcommit but hedge using a system-wide percentage reservation similar to the root reservation on ext2 partitions. Only when a pending OOM situation is recognized would the kernel dip into the reservation, and then only to bail itself out.

In the long term, maybe Linux should consider adopting one of the few intelligent Microsoft OS features-- a signal sent to user applications that indicates a pending OOM and which implicity requests the application to either give back some memory or exit completely. If and when the OOM killer is needed then it has some knowledge of which processes attempted to reduce their size and which did not. [Insert your Darwin witticism here.:-P]

Sometimes the only pages in RAM are critical to the operating system. Say for instance only kernel pages and pages for init are in memory. Now which do you kill?

Well, obviously you kill the system maintainer for compiling a kernel that won't fit in memory. :-P

Ever see W2k pop up a dialog that says "Running low on memory increasing page file size"? Thats not because the machine is 'low' on memory.

Linux has the ability to dynamically increase the swap memory using regular files if you install swapd. Of course that's yet another user process that requires its own memory....

Re:Overcommit is _NOT_ needed

[bored]

That assumes a certain amount of diligence on the part of the programmers.

Your right! Handing return codes from every function can make a big mess, this is why it rarely happens in user space. This also why you should use C++, Java, or any other language with structured exception handling. That way the programmer can be 'sloppy' and still handle error conditions. User space programs that are considered 'critical' do a lot better job. This is a easy attitude. If your application can crash then be sloppy. If your writing a database then you had better handle failures in a graceful way. On the other hand you will discover that kernel programmers (Linux is becoming an exception, the Linux kernel code is getting a lot sloppier) are a _LOT_ better since unhanded failures cause the machine to crash. The kernel code can be cleaner too since a lot of functions are written to guarantee they will succeed. If they cannot then the function will automatically retry or deschedule till it can.

Well, obviously you kill the system maintainer for compiling a kernel that won't fit in memory. :-P

This is amusing :> but fails to note that the running size of the kernel is different from the compiled 'text' size. One of the problems is the required data the kernel has to allocate to manage the running processes. The more processes that are running, the more IO that requires kernel buffering, and the more memory being backed on disk increases the number of kernel pages that are locked down. One of the solutions to this problem is paging the kernel data structures. This brings up a different set of problems which are reasonably difficult to solve. These kind of problems require a unified view of kernel behavior in order to fully understand and design around. Slapping this kind of thing onto an existing code base that wasn't designed to support if from day one bring about a huge validation nightmare. All kinds of strange bugs start to crop up because critical code doesn't get properly locked down. Depending on the kernel it can actually be harder than say.. making a UP kernel run on an SMP.

Linux has the ability to dynamically increase the swap memory using regular files if you install swapd.

This is also great but doesn't help when its not running or the disk gets full. In these cases the kernel better be able to deal with it. Depending on something like that to never fail is a good recipie for an OS that crashes when the page file gets full.

Re:VM? Do we need it? Yes. We do.

[Anonymous Coward]

VM isn't just "virtual memory". It is the entire memory management system for processes. So, if you want to have more than one process running at a given time, you need it. what you and your friends did is not 'disable VM' but rather 'disable swap' which are easily confused. Disabling swap may be a good idea, but turning off VM will give you DOS, which is a BAD idea.

insightful?

[Tumbleweed]

Okay, I was shooting for funny, but whatever works. :)

This is irrelevant.

[Snafoo]

Who swaps these days, anyway? :)

an obvious solution

[Snafoo]

Since linus is obviously more open to sudden changes in codebases divisible by two than he used to be, I think the obvious next step is to rewrite the kernel in C# as a .NET component.

I mean, I know IE has some sort of swapping system... my disk grinds every time it starts up....

Re:swap space? just depends

[fferreres]

In the past you probably needed a relative swap/ram ratio. Not, it really depends on the case.

If you usually "touch" the swap partitions you can do well with 120mb swap. If you run a big webserver then you need a lot of swap just in case (it can't hurt). If you reach the post where it's not uncommon to need that load of swap, then you need more ram!
In the end, how big your swap must be depends on what you are using the machine for. On a 1 GB Linux system for desktop use you can leave it with NO swap at all.

--

Re:The failure of Open Source

[fferreres]

So? That's life not Open Source. That can happen in your family, your company and can happen to Bill Gates. Or do you think they never lose leaders who know well what they do?

Also, this has happened before for Linux for example in the TCP/IP stack where Linus had to choose and the other guy would fade...

The important thing to keep in mind is that Linus / Alan / Whoever need to choose what's best for the Kernel now while not compomising the kernel's future.

--

Re:What about the AIX VM?-Everyone else.

[Anonymous Coward]

Since the issue has been brought up. How do all the other "Unices" impliment their VM? And how does that compare/contrast to Linuxes?

Developing the VM

[whiterex]

I've been reading through the current threads concerning the "VM", and the different sides of the coin concerning Alans "VM" opposed to Ricks.It suddenly dawned on me that both have put forward commendable work as was mentioned and are doing a great job, but as the saying goes "two heads are better than one" and exactly that! I think that if these two guys got together and formed a joint project drawing equally on each others positive aspects and combined "ideas" Then we would have a project worth writing home about.

2.4.14-pre5aa1

[uffuff]

Try linux 2.4.14-pre5aa1, it works great on my Pentium MMX 166 Mhz, 64MB RAM, 200MB swap. I done some memory allocation stress on both 2.4.14pre5 and 2.4.14pre5aa1, 2.4.14pre5 freezed while 2.4.14pre5aa1 killed away my program as I expected:

__alloc_pages: 0-order allocation failed (gfp=0xf0/0)
__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
VM: killing process test_vm

I have tried almost all kernel versions from 2.2.13 till 2.4.14-pre5aa1 including ac series. When I switched from 2.2 series to 2.4 I have got perception that my system became slower, not to mention that 2.4 VM put every process to swap, using RAM mostly for buffers and cache (e.g., when switching to different WindowMaker workspace you see that xterms on that workspace already got swapped with no reason, i.e. at no memory shortage). And now it seems like those good days of 2.2 perfomance is coming back. Maybe it is not perfect yet, but it is really getting there. It is again possible to run system without swap if you have enough RAM.

Answers to the above

[Animats]

False. Any decent VM does demand paging. Only the pages that are needed are loaded from the executable.

If you implement a VM that way, launching a program takes a very long time. You could, in theory, start out with nothing in memory and page-fault the program in. This requires one disk access per active memory page until enough is loaded for the program to run. The very first virtual memory system, for the Burroughs 5500, worked that way. It worked OK for batch programs, in an era when batch programs ran for minutes or hours, but was terrible for interactive work.

Most operating systems today load most or all of a program at startup, let the app run for a while, then release the unreferenced pages. Deciding how much to load at startup is an interesting question. The BSD UNIX guess was the first N bytes of the executable, where N is a system tuning parameter. (What, exactly, does Linux do about this?) This is a mediocre guess, but an easy one to make. It's OK for long-running programs, but terrible for short-lived ones. Short-lived programs don't run long enough for the least-recently-used page info to become useful. If paging occurs in this situation, the pages removed are ill-chosen, since the LRU info isn't useful until the program has run for a while.

Much of the memory-demanding things servers do look like short-lived programs. CGI programs and Java servlets are short-lived programs. So they're a bad case for a VM environment. If memory gets tight enough that short-lived programs get paged out, thrashing is almost inevitable.

You don't want to page out at all on a server, except (maybe) under transient overload. As soon as paging activity starts, it's time to throttle back the amount of server concurrency until paging stops. This requires coordination between OS and application of a kind not usually seen in the UNIX world, though mainframe transaction systems have had it for decades, all the way back to CICS.

Desktop systems have a different set of issues, but they don't look like classic time-sharing systems either. My main point here is that in the last decade, the memory usage behavior for most programs has changed considerably, but we're still using virtual memory concepts that were developed in the 1960 and mature by 1980.

And remember, even when everything works right, you get the effect of at best 2X the memory.

Here's a basic tutorial on VM, with emphasis on Linux.

Re:Answers to the above

[Salamander]

If you implement a VM that way, launching a program takes a very long time.

And it's better to load the whole thing? Um...hello? Anybody in there? Sure, you can save some latency by transferring the data in large chunks instead of individual pages, but you make up for that in wasted transfer time. Then you have to consider the effect of filling up the cache with pages you actually turn out not to need, evicting other pages you do need to make room for them. Overall, it's a big loss.

Most operating systems today load most or all of a program at startup

Bullshit.

Much of the memory-demanding things servers do look like short-lived programs.

Bullshit again. Maybe it's true for web servers, though with mod_perl and such out there I have my doubts. It's definitely not true for most other common/interesting workloads.

the memory usage behavior for most programs has changed considerably, but we're still using virtual memory concepts that were developed in the 1960 and mature by 1980.

Three strikes, you're out. I was there in 1980, in fact I was working on one of the earliest SMP VM systems (UMAX, at Encore). Things have changed a lot since then.

And remember, even when everything works right, you get the effect of at best 2X the memory.

Hm. Four strikes? The 2X figure is a Linux 2.2-specific red herring. Many systems can make effective use of more under many workloads.

Re:Answers to the above

[Anonymous Coward]

we're still using virtual memory concepts that were developed in the 1960 and mature by 1980.

Actually it seems to be only *your* thinking about VM systems that's stuck in the 80s, not anybody else's. Even for /. the density of factual errors in your posts in astounding.

Here's a basic tutorial on VM, with emphasis on Linux

It's pretty amusing that you'd cite Rik van Riel as an expert even as you're trying to invalidate all of the work that he does.

Re:Answers to the above

[Animats]

Apparently Linux doesn't preload applications, though BSD does. See this little note about a Linux preloading experiment, which reduced the launch time for Netscape Communicator from 14 to 4 seconds by reading the whole executable in at startup. On the desktop, that's probably a win when the user launches a program, since it gives the most resources to the program the user wants right now.

But that's a brute-force approach. A better solution would be to profile applications, feed the profile info into the linker, have it put the most-used stuff first, and put a hint in the executable that indicates how much needs to be preloaded to get through the first few seconds of running without a page fault. That would lead big improvements in application launch times.

The Wind River DIAB System does things like that. So does Sun Workshop. But I don't see the Linux world doing anything like that yet.

I see the point about mod_perl. mod_perl is essentially a way to move some loading and memory management from the OS to the web server. If the OS were good enough at transaction processing and program initial load, the need for mod_perl would be much less. But mod_perl is a good tradeoff, because most CGI scripts are tiny. The poor program load performance inherent in a load-by-page-fault approach may have driven the server world into that solution. mod_perl, after all, was developed as a reaction to slow CGI program load times.

Re:Answers to the above

[DaveWood]

"A better solution would be to profile applications, feed the profile info into the linker, have it put the most-used stuff first, and put a hint in the executable that indicates how much needs to be preloaded to get through the first few seconds of running without a page fault..."

I was just thinking the same thing. This is an interesting idea. I wonder how difficult a patch would be...

It seems, still, though, to come down to better tuning the system we have (and helping it tune itself!)... if you are proposing a fundamentally different alternative I still do not understand it.

Best is not best

[Anonymous Coward]

With a few tweeks, an inferior techmology can be faster than a better written system. The fact that some select video games may run faster in Win98 than Linux does not make Win98 supperior. It could be either VM system that will work better in the future. I would hate for people to choose the (Only Currently) faster technology that may not be the best....

Doh.

[TheLink]

Why don't you just boot different kernels if you want to try stuff out?

Especially if you are just testing the VMs out, the other stuff shouldn't affect your tests much.

Why I need it.

[TheLink]

Because Linux and some other O/Ses don't handle out of memory situations well.

To me it's better to have a gradual degradation of performance as memory runs out, than to run straight into a brickwall, wheels spinning at 133MHz and have Linux kill the wrong process or deadlock.

Another thing about your proposal: if all of the app has to be in then how do you propose to handle process forking?

Cheerio,
Link.

Re:Overcommit is _NOT_ needed (WRONG!!!)

[CaptnMArk]

/proc/sys/vm/overcommit_memory=0

certainly doesn't prevent the overcommit. If it worked, this issue wouldn't come up regularly.

Personally, I want to disable overcommit. Disk is relatively cheap and I have no problem with adding 1gig of swap.

And it wouldn't decrease the efficiency at all.

Ahh! Vindication.

[mindstrm]

Good. I didn't realize it could send mail when these things are attempted. That makes it okay in my books then.

I just thought it was funny to always hear about this 'intrustion detection system' that didn't actually detect anything.