Tuning The Kernel With A Genetic Algorithm

fsck! writes "Jake Moilanen provided a series of four patches against the 2.6.9 Linux kernel that introduce a simple genetic algorithm used for automatic tuning. The patches update the anticipatory IO scheduler and the zaphod CPU scheduler to both use the new in-kernel library, theoretically allowing them to automatically tune themselves for the best possible performance for any given workload. Jake says, 'using these patches, there are small gains (1-3%) in Unixbench & SpecJBB. I am hoping a scheduler guru will able to rework them to give higher gains.'"

Comments

Complexity?

[BurntNickel]

So how much additional complexity is added for a 1-3% perfomance improvement? I'm all for more speed, but keeping thinks simple can often be more improtant when it comes to maintainablity and adding additional features.

Other kernel parameters?

[Feint]

Could this be extended to include other kernel parameters as well? Depending on your app, things like TCP timeouts and other muck can have a large impact. Tuning this stuff is currently somewhat of a black art. Then as the user community of the app becomes familiar after rollout, a lot of the usage patterns change. In a few cases, this means we end up having to re-tune the kernel.

If this package could be extended to the other parameters, it would save my customers a *lot* of time and money.

If nothing else, this could be a deciding factor for some of our clients to use linux instead of windows.

Re:Dear Kernel Coders

[Xpilot]

Go grab the patches. They're commited into the BK repositories already. Sheesh.

Patches for 2.4 can be found in this changeset.

Patches for 2.6 can be found in this changeset.

Click on the little "diff -Nur style" link for a an actual usable patch.

In the course of a few hours, you have the fixes already. Yay for open source.

Btw, nice troll :p

Re:Not worth it...

[Corfe]

It's a unique idea - what's wrong with running it for a while with your typical load (say, for a fileserver), finding some better-than-average parameters for the kernel, then running an unpatched kernel with those parameters manually entered?

What is "on the borders of statistical error" depends on how many times the test was run, and how much variation there had been in his results before. I think it's pretty safe to assume that if he knows how to implement a genetic algorithm into the linux kernel, he knows how to handle statistics properly.

GAs aren't rocket science

[Earlybird]

Because most people aren't intimately familiar with genetic algorithms, and because GAs are associated with machine learning/artificial intelligence, GAs are seen as somewhat mysterious and magical, and are therefore either accepted with "whoa, cool!" or rejected with "whoa, complex!" While GAs are indeed novel compared to many long-established algorithms, both mentalities are overreactions.

In reality, the basic GA framework is "just" another efficient search algorithm, no cooler or more complex than, say, a hash table or a binary search tree; at its simplest, a GA is a way to find an optimal configuration of components without looking at all possible (potentially explosively exponential) combinations; instead, you look at just some permutations, and as you iterate through generations, applying breeding and mutation, you arrive at a generation which is statistically close to optimal.

GAs are also in no way new or unproven technology; a nice example of mainstream use is PostgreSQL's query planner, which uses GAs to optimize query plans.