Slashdot Mirror
Linux Kernel Benchmarking: 2.4 vs. 2.6-test
frooyo pastes from kerneltrap: "Cliff White recently posted some re-AIM multiuser benchmark results comparing the stable 2.4.23-pre5 kernel against the 2.6.0-test5 and 2.6.0-test5-mm4 development kernels. In his conclusion he makes reference to earlier scheduler tests posted by Mark Wong saying, "Short summary: we mostly rock.""
If all you're after is karma, just reaffirm the notion that linux is super-fantastic, and MSFT is teh ghey. Toss in an SCO reference.
You will never be "redundant or offtopic". Only Insightful and Interesting.
I don't need no instructions to know how to rock!!!!
Not to be a n00b, but I can't make too much sense of the benchmark the story linked to. Could anyone give a short simple little explanation of what it means? Thanks so much!
I am a viral sig. Please help me spread.
Seriously, its great and all, but when will it be ready for the masses? I.e. the holy 2.6 release? For us, loading a beta (or even alpha) kernel is something that we can do in our sleep, but look at it from this perspective: all of these improvements will only really make an impact once developers can write applications specific for this environment, which requires, at a minimum, an official release.
Children in the backseats don't cause accidents. Accidents in the back seats cause children.
It's only significantly faster if you have 8 processors.
Whereas it is 7% slower if you have one processor.
I suppose they'll have uniprocessor version which runs faster? Lots of people have uniprocessor pcs.
Hyperthreading doesn't really count.
Want to get me excitied about Linux ...
1) Integrate seemless plug-in support into Mozilla.
2) Make Open Office slicker/ handle Mickeysoft documents better.
3) Make a spreadsheet that doesn't suck.
4) Make is to that I can actually cut and paste weird stuff between applications.
5) Make is so that my PC can get updated just by clicking on items and not chasing down library incompatiblites or typing "rpm --force" or "make install" or whatever.
I don't care about 1% improvement in 8 SMP mode on some kooky benchmark.
I'd love to see Linux being the best OS for multiple CPU scaling.
You do need a scalable OS to suport lots of processors, of course, but you also need hardware that scales too (clustering doesn't count). Example - SGI is using Linux with NUMAflex on the Altixes to cluster 64-processor system images, but that kind of hardware isn't commodity in any way, and isn't going to be anytime soon.
Anyway, Linux doesn't scale THAT well...as of 9/2000, SGI was using IRIX for a 1024-processor single-system-image supercomputer; I've heard they can go to 2048 now, but I don't have anything to back that up. Dunno about Solaris, but I imagine it's pretty scalable as well.
Facts do not cease to exist because they are ignored. - Aldous Huxley
The "We mostly rock" statement was referring to a different benchmark (the one in the story's second link), in which the scheduler performance on single processor machines more than tripled (and performance on 8-way machines went up ~50%) between 2.5.30 and 2.6.0-test5. The first link's benchmark isn't very impressive, like you point out, but it's also not the same program.
I've got nothing against Linux improving at SMP in essence, but there is something very bad going on here it seems to me. Notice that while the new kernel 'kicks ass' on SMP systems, on uniprocessor systems the 2.4 kernel is the one kicking ass. Anyone benchmarked 2.4 against some of the pre-SMP kernels on a uniprocessor machine?
Face it, the vast majority of users are uniprocessor, and kernel performance is more of an issue on lower-end machines. Improving performance on big multiprocessor boxes is fine by itself, but not when it harms uniprocessor performance. I'm not a kernel hacker, but I've read many people that this would not happen, that the SMP code would not hurt performance on a uniprocessor machine when the kernel is compiled without it, but that's obviously not turning out to be the case. Anecdotal evidence, at least, suggests that this performance degradation has actually been going on for quite some time, at least back to when SMP code first started being added.
I'm not sure what all the factors here are, so naturally I'm not going to tell you the solution, but it certainly looks like a potential problem that should be discussed. Hopefully someone with more specifics than I have can chime in...
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Just becaues you can't see its use outside of a toy, doesn't mean everybody can't.
Linux IT Consulting and Domino Development in Michigan
2.6 is supposed to be fully preemptable, which should make lots of latencies decrease, leading to better interactive performance on a desktop, even if the overall throughput is lower. What this benchmark shows is that linux kernel 2.6 is a slower uniprocesser server than 2.4. While that is too bad, it doesn't really say much about desktop linux. I just installed 2.6-test5 on my (2 cpu) desktop, but haven't really had time to evaluate its performance relative to 2.4.
linux-2.6.0-test5 - 992.06 - Uni
linux-2.6.0-test5 - 1017.43 - Dual
linux-2.6.0-test5 - 5406.68 - Quad
Does this mean that you only gain 3.49% when adding a 2nd processor? Obviously I don't expect things to scale linear but 3%!? Am I missing something here? And then 81.65% for quad? I'm not trolling, I'm looking for someone to explain what I'm missing.
That statement simply is not true. Granted, you can always find some corner case where the workload is going to be slower between releases (2.x or 2.6), however, as a rule of thumb, 2.6 should still be a huge improvement for even uniprocessor users. Best yet, many, many parameters of the kernel and scheduler are tunable, so, you can always adopt the kernel to work best for your specific workload needs.
While it's true that they are working hard to significantly improve Linux for the server room, by far, they have never lost site of the uniprocessor user. Remember, there is nothing wrong with tuning the kernel for your uniprocessor needs, and specific workloads. They just can't do that when they are benchmarking because it would skew the results, invalidating them. They are not only trying to measure how their improvements effect the overall system, but, what makes for sane initial defaults, which are reflective of a general purpose and broad workload. If you understand what you are doing, there is not a reason to believe that you can't greatly improve things for your specific uses and workloads. It's important to keep all of these in mind when talking about these benchmarks. Furthermore, you should fully expect your favorite distro to come with tuning presents which reflect a targeted workload (file/print server, workstation, database, web server, etc.).
Keep in mind that the benchmark you looked at represents one category of many different types of workloads. So, for that specific workload, it may of been slower, however, that workload my not represent anything you do with your computer. Remember, other types of workloads are significantly faster. One last note, remember, performance is the classic trade off with lower latencies. It trades responsiveness for raw throughput. If, on a uniprocessor workstation, you only see a -7% drop in performance and latency is greatly reduced, chances are, not only will you never notice the loss in performance, but you'll be praising it for how well it works with your mouse, monitor and keyboard (if feels better and makes you a happier user).
Just some food for thought.
Wrong.
One benchmark used for Linux kernels is hammering a system while playing an mp3 to see if they can get it to skip. Low latency is mostly a desktop feature, and the 2.6 kernel is going to have much improved latency.
Other portions of Linux have changed, and may not initially outperform 2.4, but if you think this kernel isn't going to be a dramatic improvement over 2.4 for desktop users and servers, and if you think the kernel developers aren't taking the desktop into consideration, you are mistaken.
Why do I keep typing pythong?
Well, I don't think you can conclude that the SMP changes to the kernel are what's slowing down the 2.5 Uniprocessor performance vs 2.4 kernels. There are many other changes that took place (low latency and an improved scheduler come to mind) that aren't SMP related.
Obviously the SMP performance has been improved, and there was a lot of potential for improvements looking at the 8x test. Another way to interpret the results would be to say that the other changes decreased performance across the board on SMP and Uniprocessor systems. The SMP improvements in SMP machines more than made up for this added cost and improved raw performance on SMP machines.
Hopefully the performance loss on Uniprocessor machines can be decreased or eliminated. Even if it's not, I think you need to remember that raw performance isn't the be-all-end-all thing that's important. 7% is pretty small in the grand scheme of things where processing speed is doubling every 18 months. Responsiveness and better scheduling that doesn't starve processes is more important than a 7% performance decrease IMO, and you don't get that from faster processors.
AccountKiller