Slashdot Mirror
Linux Kernel Benchmarking: 2.4 vs. 2.6-test
frooyo pastes from kerneltrap: "Cliff White recently posted some re-AIM multiuser benchmark results comparing the stable 2.4.23-pre5 kernel against the 2.6.0-test5 and 2.6.0-test5-mm4 development kernels. In his conclusion he makes reference to earlier scheduler tests posted by Mark Wong saying, "Short summary: we mostly rock.""
Comment removed based on user account deletion
The SMP code (written by Linux developers by the way) is supposed to be kicked up a notch in the new kernel. That's what I've heard anyway. I'd love to see Linux being the best OS for multiple CPU scaling.
That will help everyone from the server market, to me when I save up enough for a two processor motherboard.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
now i need another CPU to increase performance!
Go figure. An OS that gets faster with each version.
---
"how can the same street intersect with itself? i must be at the nexus of the universe!" - cosmo kramer
If you thought SCO was mad over 2.4, just wait until they make up evidence for the 2.6 kernel!
Trolling is a art,
OT, but I'm pretty sure I've never seen "real world" and "instant messenging" in the same sentence. Except maybe with the accompanying phrase "no relation to".
Not to be a n00b, but I can't make too much sense of the benchmark the story linked to. Could anyone give a short simple little explanation of what it means? Thanks so much!
I am a viral sig. Please help me spread.
Seriously, its great and all, but when will it be ready for the masses? I.e. the holy 2.6 release? For us, loading a beta (or even alpha) kernel is something that we can do in our sleep, but look at it from this perspective: all of these improvements will only really make an impact once developers can write applications specific for this environment, which requires, at a minimum, an official release.
Children in the backseats don't cause accidents. Accidents in the back seats cause children.
It's only significantly faster if you have 8 processors.
Whereas it is 7% slower if you have one processor.
I suppose they'll have uniprocessor version which runs faster? Lots of people have uniprocessor pcs.
Hyperthreading doesn't really count.
I run 2.4.22 at work and 2.6.0-testX at home. The 2.6.0test(vanilla) series feel much more responsive, especially in X. I have not done any real benchmarks of my systems, but after working with 2.4 all day 2.6 seems to fly.
Just my observation
-the_crowbar
Have you read the Moderator Guidelines
AIM (now at version 7) is not an instant messanger client. It's a benchmarking tool. Click on the link in the story to see what it is/does/etc.
A better comparision would have been against Solaris x86. Solaris scales very linearly with every added processor.
I assume that when they say the 2.4 Kernel outperforms the 2.6 on a uniprocessor computer, but not on a multi processor computer, that they have recompiled the kernel for each hardware environment.
This struck me as strange, because when the kernel is compiled without SMP support, all that code is left out. So it doesn't seem like the 2.4 should outperform the 2.6 on one cpu.
Does anyone know why this might be?
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
Looks like that 1970's UNIX code really increases performance for SMP P-III's.
Now we can appriciate the forsite that our Unix fathers had when developing Xeon SMP code in the late 1970's.
"the general trend in the metric indicates everything has been improving, so I think we rock."
:o)
For some reason, the scheduling seems to get more and more choppy (in that i've noticed) with every iteration of 2.4.x kernel. Currently i'm on 2.4.22, and while i don't have any specific tests, numbers or statistics i'm noticing some issues.
Easiest way to reproduce it is to have the machine do something cpu intensive, such as mkisofs, cdrecord, bzip2 some huge file, cp anything large, installing (via aptitude) or even the "Reading Package Lists...." stage of apt-get update.
Oftentimes, the machine will become unresponsive for about 3 seconds at a time, then jolt back up to speed, then pause for 3, on and on. Even after the command line returns the prompt, or gkrellm's cpu and proc krells show that everything is all done, i will still see lag in responses from the kb, mouse, or whatnot off and on for about 10-15 seconds.
I've gone over my kernel config and tweaked a few things here and there but with no change. I can back down to a 2.4.18 kernel and it's not as bad. Going down to a 2.2.x kernel completely solves the problem, but of course will bring its own issues with some of my newer packages (such as gcc) and a few pieces of newer hardware.
A friend of mine and I have gone over this (on my machine and his) and he experiences a lot of the same issues i do.
Mind you, i'm not complaining. I'm very grateful to all the developers of the world that i even *have* a linux system to run. But this is something that makes me more excited about the kernel 2.6.x series. I haven't tried one out yet, but from what i've heard and read, it should be awesoe.
do() || do_not();
I didn't see anything in the articles to support this, but I'm assuming this is based on x86 architecture. Has 2.6 been ported to other architectures? And if so, have these AIM tests been run ?
I for one welcome our new karma distributing overlords.
USE='clever' emerge -u sig
Gnumeric (which I have on KDE at least) is a non-sucky spreadsheet. In fact, in the course I was TAing last spring the prof had to switch to it from Excel because it could handle the operations better. The only complaint I have about it is that I can't (or at least I haven't figured out) how to cut and paste into a text document (and vice versa). ...But that was point #4 as opposed to #3, so you can strike one off.
The fix to #5 is easy.
I have something in common with Stephen Hawking...
I agree that those things are issues, but they have nothing to do with Linux (the kernel). This is a new release of /the kernel/. You should only get excited about it if you care what kernel you're running. Most people don't, and they shouldn't (as long as the kernel supports all their hardware).
It's always a long day... 86400 doesn't fit into a short.
I'd love to see Linux being the best OS for multiple CPU scaling.
You do need a scalable OS to suport lots of processors, of course, but you also need hardware that scales too (clustering doesn't count). Example - SGI is using Linux with NUMAflex on the Altixes to cluster 64-processor system images, but that kind of hardware isn't commodity in any way, and isn't going to be anytime soon.
Anyway, Linux doesn't scale THAT well...as of 9/2000, SGI was using IRIX for a 1024-processor single-system-image supercomputer; I've heard they can go to 2048 now, but I don't have anything to back that up. Dunno about Solaris, but I imagine it's pretty scalable as well.
Facts do not cease to exist because they are ignored. - Aldous Huxley
how you got insightful I have no idea...
Everything you "want" has nothing to do with linux.
That's like asking ford to build better tires or make higher octane gas.
ASK the right people for the features you want... the people writing the software you want improvements. in...
Because you know, Microsoft Windows Stinks because I can't do live editing in Adobe Premiere, same as I wont get excited about Internet Explorer until www.theonion.com get's better graphics...
do you understand how you sound now?
Do not look at laser with remaining good eye.
The "We mostly rock" statement was referring to a different benchmark (the one in the story's second link), in which the scheduler performance on single processor machines more than tripled (and performance on 8-way machines went up ~50%) between 2.5.30 and 2.6.0-test5. The first link's benchmark isn't very impressive, like you point out, but it's also not the same program.
Just becaues you can't see its use outside of a toy, doesn't mean everybody can't.
Linux IT Consulting and Domino Development in Michigan
Devphaeton, you hit the nail on the head about 2.6.0. Its main advantage over 2.4.x (for this luser anyway) is the smoother multitasking even on a uniprocessor system. I'm running a tweaked 2.6.0-test5 on my laptop, and jobs that would make 2.4.x unusable are barely detectable (from the standpoint of moving the mouse around, typing up slashdot articles, and the like).
:-)
Of course, the ACPI support and swsusp doesn't hurt either
DARL: So, um, hey. It looks like there's this new "too-pointe-six colonel" out on the market from those Lenn-ucks people. We own all that too, right?
SUIT: Well, sir, it's like this. Do you remember how the 2.4 kernel had all of those lines of code in them that are ours, even though they showed up in textbooks before most of our stuff existed?
DARL: Sure, but how does that help us with this new thing?
SUIT: Think about it. Most operating systems, according to my extensive research during years of never having looked at a computer before, contain the same code that they always did, plus a couple of lines of new comments and an extra variable or two that shows how much you're able to charge users for the new features. Just think about the Windows 95 and 98 thing. Perfect example there.
DARL: But...my mansion only has 93 windows. Where is this heading?
SUIT: *blinks* Errr...yeah. Well, it's all the same code, and even those sneaky Linux commies try to pull a fast one on us and put one of those different codes in there, we can always assert our ownership of these "opened sources" files that I just printed out. I asked this guy, you know, and he said that all of these sources are what's in Linux, and since I printed it on paper and stuff, I figure it must be a textbook. Since we own all the words that show up in textbooks, and this has a lot of words, I think we've found ourselves a new angle here.
DARL: Smithers, cry havoc and let slip the Lenn-ucks colonel lawsuit monkeys once more!
I do so hate having to correct you people. *sigh*
"Linux doesn't exist. Everyone knows Linux is an unlicensed version of Unix"- Kieren O'Shaughnessy
linux-2.6.0-test5 - 992.06 - Uni
linux-2.6.0-test5 - 1017.43 - Dual
linux-2.6.0-test5 - 5406.68 - Quad
Does this mean that you only gain 3.49% when adding a 2nd processor? Obviously I don't expect things to scale linear but 3%!? Am I missing something here? And then 81.65% for quad? I'm not trolling, I'm looking for someone to explain what I'm missing.
I've tried the new kernel, and I got more responsiveness issues than improvements. But besides that (I might very well have misconfigured something), I'd like to point out that the kernel itself isn't all that matters: the new drivers that accompany it are just as much important. I noticed a significant increase in X's launch time as well as a whopping 250 FPS with glxgears to be compared to the 150 FPS I got with my 2.4.22 setup. This is probably due to major improvements that were brought to the drivers for my i830M chipset.
This is a simulation of a database load. Basically, larger numbers are better. The numbers are tasks per minute and peak user count. The load adds users each iteration until a max is reached. See http://developer.osdl.org/cliffw/reaim/index.html for more
Second, unicasting looks to be slower. Ugh. I don't like that. That suggests to me that there are segments of code which are optimized for multi-processor use - which is great - but either there aren't uniprocessor versions, or the uniprocessor versions are highly non-optimal.
Third, scaling needs to be improved. I don't know if Linux staggers bus use, to optimize usage, but if it doesn't, it should. Perhaps make use of the existing QoS code, or inject wait states if you know an internal bus is going to be heavily loaded.
Fourth, the scaling between 2-CPU and 4-CPU suggests that something is kicking in at the 4-CPU level that's seriously good. My guess would be NUMA, which IIRC, was tied to the 4+ CPU level. NUMA developers might want to look and see if there's anything which stipulates more CPUs than it actually requires.
Fifth, we need to sort out this damn RTOS issue. Linux 2.6 is supposed to go RT when the priority reaches a certain level, but it seems to be more of an rtsched-type scheduler trick than actual RT code. There are lots of approaches out there (RTAI springs to mind) and it might be good if someone added some good hooks into the OS for real-time operation.
Sixth, and this goes along with the above, HP have a scheduler plugin system. Ok, it seems that pluggable schedulers aren't in vogue, but I do like the concept of a scheduling tree with tunable branches.
Seventh, either use UGASI or don't. Adding in IPSEC is cool, and all that, but the old IPv6 stack is beginning to get stale. (In fact, there's also a lot in the IPv4 stack that's stale, and needs to be replaced.)
Last, we've got the filesystems. I've not seen much serious CODA or Intermezzo work, recently, and I've never known the kernel-provided Intermezzo to work without problems. On the other hand, we've got Lustre, which seems to be a whole lot faster, and seems to be under active development. Yes, it's "late in the day" to go adding whole new components, but the impact would be minimal, and it would make a lot of networked users very happy.
(And happy network users give karma to slashdot freaks!
a quad cpu more performant than 4 * single cpu?
Odd but not impossible.
For example, if in the single cpu config the processes are doing a lot of memory-cache missed then having 4 cpus (with 4 times more the amount cache) could reduce the number a cache misses and so could make the quad configuration more than 4 times faster.
The same reason could explain why 2 cpus are not faster than one: if 2 caches are not large enough and if the processes have a very bad locality then you may get as much cache misses with the dual cpu system than with the single cpu system.
Why knows if Debian lurks in the hearts of men.... The Penguin knows! Debian redefined Linux for me.
It may seem like i was on crack, but I promise they were prescription meds.
I don't post charts when sending to a text-only mailing list such as linux-kernel. Not much point to that. If you'd like charts, see the full reports here: http://developer.osdl.org/cliffw/reaim/index.html