Linux Kernel Benchmarks, 2.6.24-2.6.29

Ashmash writes "Phoronix has posted benchmarks of the Linux kernel from versions 2.6.24 to 2.6.29. They ran a number of desktop benchmarks from the Phoronix Test Suite on each of the six kernels on an Ubuntu host with an Intel Core 2 processor. The points they make with the new Linux 2.6.29 kernel are 1. there's a regression with 7-Zip compression 2. OpenSSL has improved significantly 3. a regression drastically impacting the SQLite performance has been fixed 4. the OpenMP GraphicsMagick performance is phenomenally better with this new kernel. In all of their other tests, the kernel performance was the roughly the same."

Comments

The OpenSSL benchmark

[Anonymous Coward]

I find it difficult to believe that the 2x gain in OpenSSL performance for 4K RSA private key operations is solely due to this new kernel. Such operations are, at the core, just CPU-intensive modular exponentiations. Unless the kernel has become significantly better at making use of several cores (or processors) to parallelize such operations, I don't see how that can happen.

Re:The OpenSSL benchmark

[TheRaven64]

There are lots of variables that may affect this. The new kernel may be preempting the CPU-intensive process less frequently, reducing TLB and cache churn, and context switching overhead. I don't think RSA should be using the FPU, but if it is (even for one operation in the 10ms quantum) then switching to lazy FPU context switching would give a performance increase (or, to put it another way, not doing lazy FPU context switching - either by accident or design - would give a performance penalty).

I didn't read TFA (obviously), but on an SMP system, particularly an AMD machine, there are a few other issues that may arise. Without good processor affinity, the OpenSSL process may be being swapped between two cores, increasing cache misses (which can easily slow down a process a lot). The memory allocation routines may have been allocating memory from the memory controller attached to one processor while running the code on the other, increasing memory latency (and, therefore, cache miss cost).

In short, there's nothing a kernel can do to speed up CPU-bound operations, but there is a lot it can do to slow them down, and stopping doing things that slow down operations looks a lot like doing things that speed them up.

Re:Why no comparison to OS X?

[wombat21]

Comparison to OS X purely at the kernel level ? I'd prefer to see real-world benchmarks (a contentious area in itself..) across a range of operating systems, using identical hardware, but it would only spark endless debate that the methodology favoured one OS over another. Personally, I use all 3 and each has its pros and cons : such a benchmark would have to make a pretty compelling case for me to abandon any of my currently installed operating systems.

benchmark snd-hda-intel sound recording too!

[multi io]

Sure, it may have gone from working perfectly in 2.6.21 to not producing a beep in 2.6.28, but look how fast it has become! Priorities! :-P

Why dang it?

[MBCook]

Neat. They benchmarked a bunch of stuff and some real changes obviously took place. I can't help but be comforted by their conclusion (paraphrased): "Stuff changed."

How about telling me why they changed.

• Why did 2.6.29 double it's speed doing SSL signings?
• Why did all the graphics tests speed up some?

• Why did SQLite performance bomb for 3 releases?

• What was the deal with 7-zip performance changing so much? What is it stressing that other tests aren't that cause it to vary?

There are reasons for these things. You could test and find them out. You could read the mailing lists and see if someone else posted explanations (others must have noticed the SQLite thing).

Heck, look at this list of new features and make guesses. I'd prefer "the newly added HyperScheduler v3.732 is probably the source of this" than the article's "things got faster, neat."

That's why I love LWN and the kernel page so much. They post why things changed, or at least reasonable theories.

Re:Why dang it?

[Sancho]

I'd like to know why, too. Drastic changes in performance may mean that faster ways to do a thing were discovered. It may also mean that codepaths are being skipped that are essential to things functioning correctly. Remember the Debian OpenSSL bug?

That's why I'd like to know why SSL signing is so much faster under the new kernel. Seeing a 2x improvement makes me wonder if something's been screwed up that could compromise my certs.

Re:Why dang it?

[Anonymous Coward]

Hey guys. Michael (Larabel, whom owns and runs Phoronix) isn't omnipotent and he does a lot to keep his site running, OS's and tasks benchmarked, and news up to date on top of extra projects like the Phoronix Benchmark Suite of which he has brought togethor almost exclusively by himself. Do yourselves a favour and do some research, and maybe even post it to Phoronix for Michael to update. Assist him and, subsequently, the rest of us to reduce the number of questions.

imagemagic libraries??

[gandhi_2]

in almost all benchmarks, 2.6.29 did the same or a tiny bit worse than the others. Then in imagemagic operations, sometimes 2x faster? what mem / operation combination caused this?

Re:imagemagic libraries??

[blitzkrieg3]

If you're really so curious you can oprofile and find out yourself.

Note: I'm not defending the Phoronix guys. As a previous poster pointed out, they are inherently bad at explaining the why things are slower and sometimes they are flat out wrong

Re:imagemagic libraries??

[RAMMS EIN]

``Note: I'm not defending the Phoronix guys. As a previous poster pointed out, they are inherently bad at explaining the why things are slower and sometimes they are flat out wrong''

In that case, the best they can do is to stop talking about it and just stick to what they know. Knowing just what is faster and what is slower is useful by itself. It can be used as a starting point for investigating what exactly caused the speed-ups and slowdowns. If the Phoronix folks can't or don't want to do this investigation themselves, it's perfectly fine to leave it up to other people.

Re:Why no comparison to OS X?

[blitzkrieg3]

I think you're missing an article.

Besides, how do you expect a different binary to be fair? At least in this test the only variable was the kernel, whereas on OSX you have the kernel, application specific code, and the compiler to consider. They can't exactly copy the ELF binaries and libraries straight over from Linux.

That is the question...

[mtippett]

The Phoronix benchmarking is intended to provide you the answers as to why. It is to highlight the stuff that has happened.

If performance management is going on within the kernel community, then this shouldn't come up as a shock. The whole purpose of independent testing is that you see something that looks out of place, investigate and resolve. A perfect example is http://www.phoronix.com/scan.php?page=article&item=intel_atom_four&num=1 phoronix article, that showed that SuSE was trailing. This causes this http://lizards.opensuse.org/2008/12/16/comments-on-phoronix-benchmarking-opensuse-111/ discussion.

The question and answer don't need to be provided by the same voice. It is when you have someone questioning, and then someone answering, then you have a discussion, then finally you have progress.

To make it worse, there is virtually no reason that any number of the organizations supporting the leading developers can't invest a small amount of infrastructure and run the tests themselves. Phoronix Test Suite is absolutely trivial to use. The amount of "software development in autopilot" is frightening, this applies equally to Open Source as it does to Proprietary.

Boot time is better.

[Blice]

A lot of new code (and old code reformed) was added to try and speed up the boot process, I know that for sure. I saw some of the work Arjan did in the bootfast tree-

fastboot: Asynchronous function calls to speed up kernel boot
fastboot: make scsi probes asynchronous
fastboot: make the libata port scan asynchronous
fastboot: make ACPI bus drivers probe asynchronous
fastboot: Make libata initialization even more async

I don't know for sure that all of this made it upstream for this release but I know some of it did. I think you have to pass the "fastboot" kernel line for it, however. So check your kernel configs and update your grubs!

Or LILOs, if you're weird...


Oh one more thing.. I think the introduction of the asynchronous probing and various other things are going to start a whole new wave of bootfast tricks. For example, before it tries mounting the root file system and continuing on, it waits for device probing to finish. A comment above that code states "Waiting for device probing to finish... This is known to cause long delays in boots, for example this can take 5 seconds for a laptop's touchpad to initialize". The comment was written by Arjan, who obviously has intention to speed things up. So I think what might happen is instead of waiting for EVERYTHING to finish probing (Even if it is async), it'll just wait for the filesystem to become available (Perhaps try after IDE probes, then try after SCSI probes, then after USB, and so on.)

I also remember there was a patch that didn't go upstream (I don't think so anyways) that added a function to be able to initialize things later on (After the boot was done). You changed the initialize() or whatever the function name was to initialize_later(), and then after you're done booting, whenever you want, you do a command and it then initializes anything you did the initialize_later() to. So you would be able to load up the webcam initialization or whatever else you know you don't use right when you boot.

Well, where I'm going with this is that I would like to see them incorporate more of that stuff into the kernel. More boot hacks, more power saving, more efficiency. These things are only going to improve.

Re:Boot time is better.

[MichaelSmith]

Oh goody. A million bizarre race conditions.

Re:Boot time is better.

[Blice]

They did it pretty good actually. They have a function that waits for certain things to sync up before continuing at places.

And they *do* test the shit out of kernels before releasing, you know..