Slashdot Mirror
Linux 2.2 and 2.4 VM Systems Compared
Derek Glidden writes "I got sick of trying to figure out from other people's reports whether or not the 2.4 kernel VM system was broken or not, so I decided to run my own tests, write them up and post them online. The short conclusion is that the 2.4 VM rocks when compared with 2.2, but there's more to it than just that."
Whick 2.4 VM rocks ?
The latest , if so you are correct, the -ac seems to pretty much Suck though.
Sig went tro...aahemmm.....fishing........
"You'll notice that again, 2.4.13 squeaks by 2.4.12-ac6 in terms of compile times"
2.4 VM is, IMO, a significant improvement over the 2.2 VM, but completely rewriting something as important as VM management is intrinsicaly risky and it's difficult to predict with even the slightest confidence many of the consequences of such a change. This sort of thing should be left for major revisions.
From the article:
These were pretty uninteresting - just sitting there watching the kernel compile. Except that at one point, while running the 2.4.13 kernel, the hard drive started grinding away with the drive light pegged on continuously, the display became extremely sluggish and quickly froze up entirely, and about ten minutes later, the hard drive light went off but the machine remained unresponsive, requiring a hard reboot. I don't think this was related to anything I was doing as I wasn't actually doing a compile run at the time - probably just a random occurance, but worth mentioning.
So the machine essentially BSoD'd, but it's not interesting?
it goes like this -
the 2.2 VM 'feels' better for single-user use (which i disagree with) but falls down under 'heavy' load. (which, as i've pushed 2.2 to load avgs above 250, i also disagree with)
but, anyways, that's what he's saying. i found 2.4 to be much nicer in the one userland task that frequently shows off the VM - mp3 decoding under load. 2.4 never, ever skips, 2.2, with or without ESD, skipped frequently.
YMMGV.
Indie rock lives! b-side!
If anyone out there has been having problems with 2.4 vm's (and there have been some problems) you should give 2.4.14-pre7 a try. Things have been moving fast on this front for a while now, but Linus thinks it's pretty much there now.
In his words, "In fact, I'd _really_ like to know of any VM loads that show bad behaviour. If you have a pet peeve about the VM, now is the time to speak up. Because otherwise I think I'm done.
This is an experimental patch to 2.4.13, and you shouldn't run it on an important machine, but the VM by all accounts is much improved.
Even Alan Cox (who has been maintaining the older Rik Van Riel version of the VM in his -ac patches) agrees that the new VM is faster and simpler, and he plans to switch to it as soon as it is reliable enough to pass his stress testing. (which should be really soon, it seems.)
(Yes, I spend an hour a day reading the kernel mailing list.)
Torrey Hoffman (Azog)
"HTML needs a rant tag" - Alan Cox
Anyone have a link to the problems aluded to in the article with early 2.4 kernels? I've got a machine at work that swaps quite a lot and is running 2.4.2 ... I know it's due for an upgrade, but I'd like to know a little more specifically what was wrong back then...Thanks.
... about running 2.2.19 (on RedHat 6.2) on my dual PII server. Guess it's about time to upgrade, though. RedHat 7.2 is looking nice.
How is bigmem support coming along? Is 2.4 still having problems with (32-bit) systems sporting more than 2 GB of ram?
Quite often I get the feeling that Linux and BSD are doing quite a bit of "me-too"-isms in an attempt to catch up with the mainstream OSes--including MS, Apple and commercial Unixen.
I read this story and wonder if I should still be getting the same feeling -- isn't a VM subsystem mostly a solved problem? Or am I reading this wrong, and this is merely tweaking and specialization?
Since I'm no Alan Cox (I'm closer to Alan Thicke), I can't see the truth of the matter, but I get the feeling that we're doing a lot of walking in a tight circle on the path, while others have already left the forest.
Potato chips are a by-yourself food.
to know how XPs kernel would do, and how the different *BSDs, QNX, whatever you have under your sleave.
Heck, know what would be the best? A pluggable kernel system, where anyone could switch WM. Hmm, hurd? Anyways, it's nice to see 2.4 making progress, but we all kinda guessed that.
After all, if we know what is "best", then people could try to break that and become even better. And who could loose from that?;)
I don't think most people really thought the 2.4 VM was a worse performer than 2.2, especially under normal load, and in recent kernels even under high loads.
However, one thing that was not evaluated in this writeup at all was stability, especially on big boxes (as in SMP and >1GB) and heavy workloads. This is where neither VM really seems to be able to hang in there.
I admin seven such boxes that all have 2 or 4 CPUs, and 2 of 4GB of RAM, and during heavy jobs get hit pretty hard. These things run 100% rock solid with 2.2.19, I've achieved uptimes of greated than six months on all boxes simultaneaously. Basically, reboots are for kernel upgraded, nothing more.
With 2.4.x, I'm happy to get a few weeks, and sometimes much less. The machine practically always dies during heavy VM load. It has kept me from upgrading to 2.4 for several months now.
The real kicker, when 2.4 is running correctly, my jobs run as much as 35-50% faster than with 2.2, especially on the 4 CPU server, so I really wish the VM was stable enough to allow me to move.
Anyway, I'm sure it will get there sometime.
BTW, before people write about how they're 2.4 boxes have super long uptimes let me say that I too have some 2.4 based systems that have been up since 2.4.2 was released, but these machines are either single CPU, or SMP but with 512MB of RAM. 2.4 seems to run quite well in this case.
...so why does linux have 1 VM? it seems that 2 of them exist, and the BSD's have more... guys, "gimme a hunk" and "page fault" aren't exactly rocket science anymore, particularly with hardware support... the fact that there is room to make a big deal out of this is the problem, not the VMs.
Under capitalism man exploits man. Under communism it's the other way around.
I am glad to see the 2.4 VMs doing so well. I assume that Linus is not at all satisfied with the VM code and that is the reason the 2.5 branch is not started. Hopefully it will start soon when the VM trouble is solved!
so why does linux have 1 VM? it seems that 2 of them exist, and the BSD's have more... guys, "gimme a hunk" and "page fault" aren't exactly rocket science anymore, particularly with hardware support... the fact that there is room to make a big deal out of this is the problem, not the VMs.
If Linux was a microkernel I'm pretty sure this would be possible but from what I've seen of the Linux kernel code and from some discussions on the linux kernel mailing list, the virtual memory code is too entrenched in various parts of the code to be #ifdefed around with any sort of ease.
But the fact remains is that this VM holy war should have been resolved in the 2.3 series of kernels.
The number of major problems and architectural changes that are being made to the supposedly 'stable' branch of Linux kernel is really run amok.
I'm sure there's plenty of outrages to come as bad bugs are found in the volume manager and other new elements of 2.4
Conformity is the jailer of freedom and enemy of growth. -JFK
...is that Linux's warts are fully out in the open for all to see. Microsoft would never admit to such failings openly, even though anyone who has used Windows extensively is painfully aware of them.
And it's been my experience that you don't hear, "Linux never crashes" that much anymore. At least I don't say it anymore, whereas I used to. I would still say that a properly configured Linux box is more stable than any Windows box, but I've had my share of lockups. (on the desktop anyway. You'll notice my server has been up for 140+ days. The last reboot was when the power supply died [it's a patched together P166] which interrupted 243 days uptime)
All the mailing lists are public, and all of Linux's problems are there for anyone to see. This allows people to make truly informed decisions about which version of Linux to use, or whether to even use it at all. (Yes, of course these things are also true of *BSD) The current issues are why I still run 2.2.19 on my servers, since none of them get anywhere near enough load to need the newer VM's. "Stable" is definitely a relative term.
The link is in this article.
Best Slashdot Co
Win2k locks up when you copy large files? I haven't seen this problem. I have a P4 system with 256 MB RAM, I have Win2k and Mandrake 8.0 dual boot. I have the ISOs for Mandrake on the Win2k partition and I copied them to another folder with no problem. These are 650 MB files. I believe that when you copy a file 2k reads a chunk and then writes it, then repeats. I wouldn't make sense to try to copy the entire contents into memory then write it. Besides there may not be enough concurrent sectors to store it, so it would have to break it into chuncks anyway. Anyway, I think if this were a widespread problem we would have seen a patch a while back. just my opinion.
"Da ist ein Technölüst in mein Unterpanten!"
Seriously, even I have seen Linux die.. it would have been interesting if it kept happening, but it clearly states it was a one-time event.
Just a possibly interesting data point. I played Unreal Tournament with 2.4.12-ac5 and 2.4.10 (both from Debian). 2.4.10 always seems to work fine for extended periods of Lan play (as both a client and server), whereas the 2.4.12-ac5 choked after a few games--the swap ended up being nearly all used up.
Of course, this was hardly a scientific test, but I think I'll stick to something proven for now.
I can't speak for the differences between the two VM layers in the most recent versions of each, but I went from 2.4.7-2 (RH Roswell Beta stock kernel) to 2.4.13 (+ext3 patch), and I've noticed a serious improvement.
My notebook has 192 megs and 256 meg swap partition. I run Mozilla constantly (which seems to constantly grow in memory usage as the days pass). Prior to the upgrade (2.4.7-2, recompiled without the debugging options RH had on by default), swapping was ungodly slow. Switching between Mozilla and an xterm would literally take a few seconds waiting for the window to draw on the screen. Even switching between tabs in Moz was slow.
Since going to 2.4.13 with ext3 patch, I've noticed a serious improvement in this behavior. Under the same conditions (between 20 and 50 megs swap usage), switching between windows is quite fast. I don't know if it's faster at swapping per se, or if it's just swapping different things (eg, more intelligently deciding what to swap out), but for me it "seems" much faster for day-to-day usage.
I haven't yet tested in a server environment... but for desktop usage, 2.4.13 rocks. Can't wait for 2.4.14, to see if any noticable improvements are added...
Though it will be a non-issue once I add another 128 megs to this machine, it's nice to see such great VM performance under (relatively) low memory conditions.
NGWave - Fast Sound Editor for Windows
Your observations run counter to the continuous stream of reports of high latency in 2.4 found in the linux kernel mailing list. Specifically, skipping mp3 playing is the canonical report. 2.2 is often cited as being the less latent of the two.
I don't claim you're wrong. I point this out only to illustrate the subjectivity and lack of real data involved in these anecdotal reports. At the very least, the author has attempted to produce hard data on the matter.
Linus obviously thought poorly enough to of original 2.4 VM to space the mess.
Maw! Fire up the karma burner!
I remember reading about a problem with sblive and via chipsets having timing problems, at one point this caused file corruption as well!
Until SP2 (I believe) Win2k did not natively support ATA100 drives, so many boards either provided their own, or in the case of ATA-RAID controls the drivers we're shown to the system as SCSI. In my experience the native ATA-100 or 3rd party ATA-100 drivers do not perform as well as their SCSI counterparts. long shortly short, if you have a ATA-RAID disk, use the SCSI driver provided from the manufacture... Also if it's a HighPoint(tm) controller don't use the latest version of their drivers, they're is an issue with it where the mouse jumps and the sound card skips.
If you do have a via board, you may want to check out viahardware.com, they have some excellent FAQ's as well as all the latest drivers.
At any rate I am 99% sure it has something to do with your hardware, bios or drivers not Win2k specifically.
late,
-Jon
this is my sig.
Hear! Hear! Linux releases are seriously lacking in professionalism.
Just buy more memory kids, it's cheap. :)
I have half a gig and no swap, and everything runs fine and dandy
To ask the question another way, my big pet peeve with Windows 2K is that copying a big file (100's of megs) locks up the system.
Interesting. I've copied files around that are several gigabytes in size and suffered no ill effects.
What were you using to copy the files? Command line, Explorer? Define 'locks up the system'?
Fear: When you see B8 00 4C CD 21 and know what it means
I believe that when you copy a file 2k reads a chunk and then writes it, then repeats.
I'm pretty sure it mmaps the files and does a memcpy(). Would make the most sense because it then just results in direct access to the cache manager's buffers.
From lots of testing, using mmap'd files is about twice as fast as using read/write on Win2k.
Fear: When you see B8 00 4C CD 21 and know what it means
A switch between mozilla and an xterm took SECONDS with 192 MB of RAM?
Next time tell us you had 100+ mozilla windows open.
(Yes, I spend an hour a day reading the kernel mailing list.)
I'm too lazy to read LKML, but I am interested in the happenings of the Linux kernel development. I highly recommend Linux Weekly News' kernel news (updated every Thursday) and Kernel Traffic , an in depth summary of the week's LKML happenings (usually updated every Sunday or Monday).
cpeterso
I remember having problems with the 2.2 Linux kernel on occasion. Eventually after much trial and error I went into the BIOS and adjusted the system timing to be a bit slower, allowing system control lines more time to stabalize. My problems went away. Adjusting the BIOS is a black art, and it is often beyond even the best engineers, given that the sparse documentation is usually written in some unfathonable Asian "dialect" of English.
While this comparison of VM systems may be somewhat useful, I have yet to see a study done on a statistically significant sample. I would like to see several hundred different machines compared and have those results analyized. Only then do I believe that truly meaningful conclusions can be reached.
Hi,
...
at my former college, a server i'm the administrator of - has an uptime of 98 + 269 + 158 days (the power went down 2 times)..
A sparcStation 10 - running rh 6.1 + +
It's probably the most stable machine I've
ever runned into.
--larsw
I thought it was interesting how the author discussed the mid-release swap out of VM code in the 2.4 series kernel. He mentioned that Linus had felt that the AA version of the code was better than the existing version and wholesale swapped it out.
Does anyone know why Linus did this? Are there some empirical results somewhere that dictate a reason to do this? Certainly, it would seem that there would be, but the author didn't point it out.
Also the author pointed out that 2.2.x kernel's break at very high load levels. Is there documentation somewhere discussing what that might be?
Take care,
Brian
--
100% Linux Web Hosting
--
large file = bigger than 2GB
your warez iso's don't count.
Basically, the people who sided with Linus/Andrea were of the opinion that "things are so bad now [which was between 2.4.5 and 2.4.9] that a complete replacement of the VM even in a 'stable' kernel series is justified", and those who sided with Alan Cox/Ben La Haise/Rik van Riel thought that the existing VM code could be massaged and tweaked enough so that the performance would become acceptable and huge changes could be postponed until 2.5 opened.
This was complicated by the fact that between 2.4.5 and 2.4.9, the -ac series had accepted patches from Rik which weren't applied in the Linus branch and did in fact seem to be fairly successful in increasing performance through much less intrusive code changes. This was one of the main complaints of the Alan/Ben/Rik contingent; that the problems had already been largely resolved in the -ac tree, and that that approach should have been applied in Linus' tree before jumping to a complete rewrite.
At this point, a consensus seems to be forming that the Andrea VM is *much* simpler, the changes haven't had much adverse effect on other subsystems, and the performance is just as good or better than the VM in the -ac series.
The question of whether or not it should have waited until 2.5 is one that will probably never be answered to everyone's satisfaction, but at least will soon be academic.
I have some concern about how all these changes appear to businesses. Linux is supposed to be a high-quality alternative to other operating systems. Yet we've recently had a production kernel that failed to even compile, and there have been major upheavals to the "stable series" VMM, which sometime degrade reliability or performance. This isn't going to impress, and it gives competitors valid ammunition.
Almost all successful enterprises have to overcome a hurdle: how to transition from the "just for fun" start-up stage to the "managed with professionalism" stage. Ideally, fun is kept along with the professionalism. Note that even at Microsoft, B. Gates now lets S. Balmar (CEO) and R. Belluzzo (President and COO) manage things. Gates is the "chief software architect" and Microsoft is still his, but others do the managing.
Linux needs more professionalism in the management of releases, I believe.
~
Tsunami -- You can't bring a good wave down!
I don't know about other shells, but tcsh has some features that provide other useless statistics. You can set a variable called "time" that can provide additional information. From the tcsh man page [edited]:
time: If set to a number, then the time builtin (q.v.) executes automatically after each command which takes more than that many CPU seconds. If there is a second word, it is used as a format string for the output of the time builtin. (u) The following sequences may be used in the format string:
%U The time the process spent in user mode in cpu seconds.
%S The time the process spent in kernel mode in cpu seconds.
%E The elapsed (wall clock) time in seconds.
%P The CPU percentage computed as (%U + %S) / %E.
%W Number of times the process was swapped.
%X The average amount in (shared) text space used in Kbytes.
%D The average amount in (unshared) data/stack space used in Kbytes.
%K The total space used (%X + %D) in Kbytes.
%M The maximum memory the process had in use at any time in Kbytes.
%F The number of major page faults (page needed to be brought from disk).
%R The number of minor page faults.
Particularly, if you could measure the number of swaps/page faults in
the different kernels it would be pretty useful. I've got $time set
to:
# have time report verbose useless statistics
set time= ( 30 "%Uuser %Skernel %Eelapsed %Wswap %Xtxt %Ddata %Ktotal %Mmax %Fmajpf %Rminpf %I+%Oio" )
I do not follow *BSD nearly enough to make this kind of observation, but I thought I recalled that when Universal Virtual Memory was rolled into NetBSD, it was widely regarded as a good design. Anybody with much more VM design knowledge able to comment on how suitable a design like that one (or other well-regarded VM design from other Unixes) would be for the Linux kernel?
- more professionalism? i'm sorry, but that position is somewhat naive... for professionalism, get your distributions from Red Hat, SuSE, Caldera, etc.
... don't be afraid to name names! be specific!
- what was the "production kernel that failed to even compile?"
- of course there have been "major upheavals to the stable series"... remember 1.2.13 -> 2.0.x??? part of the process is going through new development/environment models...
Linux developers don't give a "romeo alpha" about maintaining some sort of business image... it has only been in the last couple of years that the "business" community has even come around to supporting Linux (after much pleading on the part of developers and users)... the 'professionalism' of 'business' in earlier times for Linux was: "Go away. You don't have market share. There aren't enough users." Screw business, and screw "suits.
- Linux is much more than a "successful enterprise," and many developers don't care about maintaining release schedules, trying to impress "customers" and so on...
- What amazes me is the recent turnaround of all the witless computer industry pundits and early nay-sayers who pooh-poohed the idea of using Linux a few years ago, but are now whining about version releases, features for users, etc.
- Want "professionalism"? Use Microsoft software! (just kidding, mind you)...
- p.s. Gates is much, much more than "chief software architect." He's Satan!
:-)
And that professionalism comes in form of distributors like Redhat/Mandrake/whoever. When they release updated kernels those kernels work as you'd expect.
:-)
I don't think you should expect every "business" to go compile every single kernel update that comes along except of course when there are some serious security/whatever issues involved - in which case the distributor releases an updated kernel.
People who like to experiment or live on the edge compile and use all the hottest kernels and they are also the ones who report of problems, which then get fixed. I don't know how professional this is but I'm happy to see new kernels being released often rather than waiting and wondering for weeks/months if there's going to be a new release some day or not.
Hmm, my first post, hope this works..
_________ Be good.
I'm talking about files bigger than 2GB and not warez ISOs. Try database dumps.
Get a clue before you flame.
Fear: When you see B8 00 4C CD 21 and know what it means
Yeah, this post was really Offtopic.
Gawd, that Moderator Crack must be really good stuff!
2.4.12; see the story & comments on Slashdot:
http://slashdot.org/article.pl?sid=01/10/11/11542
And if a closed source vendor completely changes the VM between, say Service Pack 3 and 4, you would have no chance know.
I think Linus does a relatively good job at release management.
reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))
Service pack 3 or 4 have both tested and released as alpha's and beta's before being declared stable and released by Microsoft. Linux really does have this problem and freebsd users do have a valid arguement. You should not do a radical change to the vm in a minor release. There is no valid reason. The Freebsd team only does things like this in the unstable releases. They created a minor release and a unstable or experimental release which both are under development simultaneously. The vm patch would go in the unstable one and new drivers would go in the stable or minor releases. Only bug fixes or device drivers which have been tested throughly are even released. This is how any server OS should be developed. Freebsd users claim there system is more stable partly due to the fact that they have better testing and team management. They are conservative while they view Linux as too bleeding edge. They have a point.
http://saveie6.com/
He notes in his commentary that the 2.2 kernel "felt faster" or something to that effect, while still performing much worse in actual numbers. This is probably the manifestation of the a well-known effect in the world of performance: responsiveness and throughput are often mutually exclusive.
:). On paper it was fast as hell, in practice it
seemed very slow.
In other words, given fixed parameters, it's usually not the case that you can improve both responsiveness and throughput at once. If you don't change memory, CPU speed or I/O bandwidth, and your code is devoid of excess baggage which effectively reduces one of the above, it is almost a given that the two are a tradeoff. I've personally experienced this numerous times in my own performance work, and have read the research of others that corroborate it.
Here are some really interesting fundamental examples. One company I worked at lived and died by disk performance benchmarks, in particular the Neal Nelson benchmark. This test ran multiple concurrent processes, each of which read/wrote its own file. The files were prebuilt to be as contiguous on disk as possible so that sequential I/O operations wouldn't cause disk seeks. By the nature of the test, though, seeking would happen a lot because you had N processes each reading/writing a different contiguous file. So, you lost the benefit of the contiguousness. Until, that is, we came up with a way of scheduling disk I/Os which, given a choice of many pending I/Os in a queue, favored starting I/Os which were close to where the disk head happened to be. This wasn't your father's elevator sort! The disk head would hover in one spot for extended periods, even going backwards if necessary to avoid long seeks. It was a bit more sophisticated than that, but those are the basics.
The effect was, if a process started a series of sequential I/O operations, such as reading a file from beginning to end, no other process could get much of anything through until it was done. So what did this do to performance? Well throughput shot through the roof because disk seeks were nonexistent. The test performed beautifully, as it only measured throughput, and we consistently won the day. However, I/O latency for the processes that had to wait was extremely high, sometimes on the order of minutes.
Needless to say, these "enhancements" were only useful for benchmarking, or perhaps for a filesystem on which the only thing running were batch processes of some kind. It would feel slow as molasses to actual human users, verging on unusable if anyone started pounding the disk. You can't wait 60 seconds for your editor to crank up a one-page file (well, okay, we didn't use MS office in those days
One paper I read on the subject of process scheduling postulated that by increasing the max time slice of a process you could improve performance. The idea was that you would context switch less, would improve the benefits of the CPU cache, and so on. They increased the time slice to something above 5 seconds and ran some tests. Of course, the throughput improved by some nontrivial amount. Predictably, though, the system became unusable by actual human users for the same reason as in my disk test example.
The other extreme would be absolute responsiveness, in which you spend all your time making people happy but not getting any real work done. An example of this would be "thrashing", where the kernel spends most of its time context switching and not actually running any one process for an appreciable amount of time.
The sweet spot for the real world is somewhere inbetween, perhaps a little closer to the throughput side of the spectrum. It sounds like this may be the direction they've gone with the 2.4 kernel, though I'm sure they've done a lot of optimizing and rearchitecting to improve performance overall.
You can count on the following truths:
1) Most managers do not have a clue about the VM issues. I'm sure many journalists are itching to write about 2.4, except the technicalspeak would probably turn off any suit reading it.
2) Linux is not a "professional" product. No "professional" busts their butts to meet a release date for free.
3) Any business who cares about professionalism would not be running any
(web)server on a 2.4 kernel in a production environment. It has nothing to do with the 2.4 release; one almost never put into production something that was implemented weeks ago without extensive testing.
Linux is always going to be a "cult" of Torvalds (though "church heirarchy" would probably a better metaphor). Its going to stay that way until Linux quits, dies, or something like Red Hat or IBM decides to introduce a "schism". If you really feel the need to express your concerns about professionalism, I suggest you email your concerns to Linus. Slashdot readers do not enforce professional standards in Linux kernel releases.
There is no America. There is no democracy. There is only IBM and AT&T and DuPont, Dow, General Electric, and Exxon
Jeez, in an age where a gig of core costs less that 100 bucks, do we really need the abstraction layer of VM any more?
"I bless every day that I continue to live, for every day is pure profit."
** Tests were run using 2.4.13, which includes the new "AA" VM, 2.4.12-ac6, which includes the "Rik" VM, and 2.2.19, which uses the "old"
2.2 series VM. **
Derek, could you run your test suite against 2.2.19pre2 or higher? pre2 was updated with Andrea's VM-global patch. This would be more accurate for those choosing between 2.2.x and 2.4.x.
rgds,
tim.
No.
Can you imagine a beowulf cluster of Alan's Cocks?
The professionalism of which you speak exists, and is the job of the distribution managers in this community. RedHat, SuSE, Debian, etc. all produce stable releases which include vetted kernels. Often these kernels aren't latest stable, but rather the ones which passed the QA and field testing process.
While Linux kernel development has its ups and downs, stable branches definitely do cool down and become very stable (see 2.2.19). This time is a little bumpier than most. Such is life.
On one hand, I do agree that the VM change would have been better in a development branch (2.3 or 2.5). On the other, I view that statement as an valued ideal that didn't mesh with reality in this case. I'll guarantee that such happens in "professional" production environments -- sometimes you simply find yourself between a design flaw and a hard place and *must* make a decision that will enable the product to meet its goals. Yes, Linus took a gamble and ruffled feathers. Yet as hindsight plays out, his gamble appears to be paying off in both performance and maintainability.
That's strange. mmap()'ing the file should be about as fast as read/write, since the Win2K VM implements read/write by mmap()'ing 256KB file regions into kenrel virtual memory and doing memcpy().
A deep unwavering belief is a sure sign you're missing something...
I don't see that as much of an issue. Why not? Well, I've been waiting for the VM thing to settle down, and a week ago I decided to check a couple of my production servers. What kernel did I find? 2.4.0. And they had been up since that was released. With 512M, swap usage was still at zero.
In business, you don't usually just upgrade every time a new -preX release comes out. You find something that works, and use it until you either it's broke, or you find something much better that you also know to work.
Steve
Oh, you're not stuck, you're just unable to let go of the onion rings.
No, it makes sense. Think about what happens to the physical data in each case.
Scenario 1: read/write. The data gets paged into physical memory (which happens to be mapped in the kernel's address space), memcpy'd into a buffer (different piece of phys memory, mapped in user space), and written out from that buffer.
Scenario 2: mmap/write. The data gets paged into physical memory (this time, the physmem is mapped to an address in a user task's address space, not just the kernel's address space), and then written back out from that same piece of physical memory.
Scenario 2 avoids a memory-to-memory copy. So it ought to be a bit faster.
"I don't think this was related to anything I was doing as I wasn't actually doing a compile run at the time - probably just a random occurance, but worth mentioning. "
What is this guy smoking!? A Ferrari might take the least amount of time to get somewhere, *IF* it ever gets there that is!
I've hard-rebooted my Linux machine a few times ever, and those few times were mostly since moving to 2.4.10+2.4.13.
Under high load, "the 2.2 kernel gave the best responsiveness".
Under low load, "2.2 again was the most responsive" AND WHILE IT "used the most swap ".
"Also, after one 2.4.13 test run, the system locked up entirely, requiring a hard reboot."
Another "random occurance" to be sure.
"2.4 VM systems are undeniably better". Say what? Better because it is quicker or better because it brings Linux in-line with Windows 95 stability?
Is it just me, or does this look like a guy trying to advertise himself and his company as smart guys who know Linux and want work, through some long winded test that claims that the VM in 2.4 is much better that 2.2 because it is a quicker, ignoring the glaring fact that 2.4 locked up hard for him more than once in his tests?
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
How is this off-topic? Troll yes, off-topic no. Fucking dumb moderators.
Worse is the information that when he loaded his system up, he got repeated "parse error" messages from GCC. That's unacceptable, and needs to be understood. He suggests that it could be a transient hardware error, but since he got the same error more than once, that's unlikely. The other possibilities are that the VM manager corrupts memory under overload (very bad) or that GCC ignores some reported out-of-memory condition (also very bad.) This needs to be diagnosed.
I have some concern about how all these changes appear to businesses.
I don't, and Linus has made it abundantly clear on several occasions that he doesn't either. I doubt that most of those who use Linux care a whit what 'business' thinks about anything concerning Linux.
It's mostly the bandwagon types and the pundits who go on and on about these things, and why should we cater to them? After all, business could drop Linux altogether, decide to deal with MS until the end of time, and the folks who develop Linux as a hobby will still be plugging away. Works for me.
Linux is supposed to be a high-quality alternative to other operating systems.
Linux isn't 'supposed to be' anything other than what we say it is, especially those in development. This is pundit-speak; the spew that pundits commit to articles in order to make themselves appear far more important than they are.
This isn't going to impress, and it gives competitors valid ammunition.
Linux has no competitors. Red Hat, SuSe, etc., now they might have computers; but Linux does not. And as I said before, most of the developers don't care in the slightest about other operating systems, or what some PR lackey from another company or a self-appointed pundit has to say about the situation.
Almost all successful enterprises have to overcome a hurdle: how to transition from the "just for fun" start-up stage to the "managed with professionalism" stage.
It's clear you're missing the entire point. Linux is not an enterprise; the process that's used to produce Linux doesn't have to 'transition' to anything. You're confusing a bunch of volunteer developers hacking up an OS with an actual business. Apples and oranges.
Linux needs more professionalism in the management of releases, I believe.
Linux is doing just fine the way it is. If you want more 'professionalism' I'd suggest you 'transition' yourself to Windows XP. That OS is produced by a business which seems to be what you're looking for anyway.
Max
My god carries a hammer. Your god died nailed to a tree. Any questions?
Please mod parent up.
Thanks!
Er, maybe I missed something, but didn't Linus say that the goal for Linux was "world domination"? If so, you're all wrong.
Do you really think that if 170 patches are going to give minimal increase in performance that a few more are going to make it any better? I think the benchmarks show a lack of justification for switching to a kernel that still hasn't been fully tested. At least not on a production system.
Also I think the author of the benchmark brings up a good topic about the schedular being as important as the VM. IMHO, I think the the schedular needs a couple of tweeks. The only real way to reduce page faults is to schedual processes that will likely reference pages already in memory over those that have a greater possibility to access swapped pages. In the interim of executing these other processes, you could issue the DMA commands to swap pages so long as the processes you schedualed are unlikely to use parts of the disk that are not cached. (Thus blocking anyhow, just select another process) I'm uncertain of how this might affect old PIO drives, or if it is completely feasible, but it doesn't seam that hard of a task.
Karma Clown
argh, that should be 'competitors' and not 'computers'. I sure hope to hell that wherever Linux resides, it's on a computer.
Max
My god carries a hammer. Your god died nailed to a tree. Any questions?
My computer today has far more memory, than my computer 3 years ago had combined memory and swap, and a few years before that, PC`s atleast didn`t need swap, neither did MacOS or AmigaOS, infact swap was reserved for high end servers with the IO performance to handle it. When microsoft introduced win3.11 with swapfile support, i always had it turned off, for a HUGE performance boost.. then code bloat overtook the price of ram, and more modern o/s`s wont run atall, or will run very badly without swap, even on a system with over a gig of ram. Ram is so cheap nowadays, o/s`s should be designed to run without relying on swap, and should take advantage of the extra speed.. swap should again be relegated to the high end servers. What would you rather have? a 256mb swapfile that severely cuts into your hd io bandwidth, or a 256mb dimm for $60 or less.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
I very much doubt that these kinds of 'political' discussions are less common in large corporations.
In fact, I belive they may have more of them.
The main difference is, in an open source project like Linux,
these debates get fought-out openly instead of in corporate conference rooms.
This means more people are debating and studying the problem, which (hopefully) leads to the better
decision-making.
You're right.
A deep unwavering belief is a sure sign you're missing something...
I'm typically running apache, cups, KDE, a few konsole windows, kppp, konqueror, and xmms (random play of a 6000 song mp3 archive niced at -19),.
To reduce my per-minute ISP fees, I open-up a konqueror window for each topic I'm interested-in (incl. several slashdot discussions) and log-out as soon as the pages finish loading. Then I peruse the news and discussions offline. This technique can easily cut my monthly ISP bill by 50%.
My problem with this technique is that when I open the 12-16th konqueror window, the machine slows to a c r a w l, sometimes even causing my high-priority pre-empting xmms mp3 playback to skip. So I begin very slowly closing konqueror windows... sometimes the swapthrash stops after closing a big (slashdot) window, other times I need to killall konqueror.
My setup is pretty typical: an ABIT BE6 mobo with a Geforce 2 MX (Xfree 4.1.0 with nvidia 1541 binary drivers), Celeron 750 256mb RAM, 512MB swap, running a slightly tweaked Mandrake 8.0 (traktopel) distro.. Oh, and a 1600x1200 24bpp desktop.
I'd be very interested to hear whether anyone else sees this behavior when opening 12-16 konqueror windows.
I'm not blaming the Kernel VM just yet: memory problems can be caused by bad userland software.
Could it be that when Konqueror renders a web-page, it renders the whole thing to a big-ol bitmap (including stuff that's off-screen)? If this is true, than the bitmap representation of a typical slashdot discussion containing 30+ screen-pages of text would require a bitmap of 800 x (1200 x 30) pixels = 28,800,000 pixels.. @ 24bpp -> 86,400,000 bytes! Whoah! Is konqueror that much of a pig?
I rather doubt this, but I still suspect Konqueror is maintaining more 'state information' than it needs-to in the event that a window gets hidden behind another window. Naively, all it really needs to store is the URL and any HTML/Javascript contained therein -- a few hundred kB per page... right?
Oh, and X surely needs to keep a buffer of the window's bitmap (800x1200@24bpp=3MB), but even with 20 open windows, that shouldn't strain a system with 256 MB RAM + 512 MB swap? Rrrright?
Can someone share similar experiences? Or share tips for dealing with this? (other than "open fewer windows" or "buy more RAM")
TIA