Alternative To the 200-Line Linux Kernel Patch

climenole writes "Phoronix recently published an article regarding a ~200 line Linux Kernel patch that improves responsiveness under system strain. Well, Lennart Poettering, a Red Hat developer, replied to Linus Torvalds on a mailing list with an alternative to this patch that does the same thing yet all you have to do is run 2 commands and paste 4 lines in your ~/.bashrc file."

Comments

Re:which one is 'right'?

[h4rr4r]

I've done some tests and the result is that Lennart's approach seems to work best. It also _feels_ better interactively compared to the vanilla kernel and in-kernel cgrougs on my machine. Also it's really nice to have an interface to actually see what is going on. With the kernel patch you're totally in the dark about what is going on right now.

-Markus Trippelsdorf

right from the article

Re:What's the catch?

[Monkeedude1212]

I didn't see any Try's, so I don't think so.

Whee...

[Target Practice]

Man, after reading some of that thread, those folks in kernel development make Slashdot users seem downright well-mannered.

Re:Whee...

[gandhi_2]

Ever seen a 50-year-old ER nurse? 90% of the time, they are callused to the suffering around them. It comes with repeated exposure to the environment, and although their demeanor may seem rough to others, they are extremely efficient and skilled.

Sometimes, I think what some mistake for IT snobbishness is just a natural consequence of exposure to the lifestyle.

I thought it would be fun to post some things in answers.yahoo.com in the IT-ish categories... after a while you realize that the REALLY good questions are drowned out by people who REALLY just need to GTFO and RTFSomething.

I work in public ed IT, and can say with NO uncertainty that most people don't want the right answers, they want the nice answers. It's hard not be rude in some cases.... it just comes out your pores after enough exposure to the environment.

Re:Whee...

[__aagmrb7289]

And thus, the cycle perpetuates. Better yet, if you are going to be an ass in your reply - just don't reply. That means the user might NOT get help - and that may be a concern for them - but it's better than getting attacked - which will definitely be a concern for them.

Re:Whee...

[vegiVamp]

This.

I'm on a few mailinglists, and while I do my best to provide clear, concise, correct and helpful answers to questions, I keep being amazed at how some people simply don't bother to do the basics first. Like, you know, even looking in the general direction of the manual.

One of the lists I'm most active on these days is the MySQL one, given that I'm almost fulltime DBAing these days. Note that MyQSL has excellent and comprehensive online manuals for every version you care to run.

I've seen people actually think that list is populated by MySQL employees who are paid to answer their every stupid question, and get impatient and testy if they haven't seen an answer in ten minutes.

i've seen people spam the list with first their inane RTFM questions, followed by a great big stream of "insights" on how he solved it and the most obvious straight-from-the-manual statements.

I've seen people who seem to think that the list is there to write their queries for them. After I got rather miffed and wrote a bit of a sharp mail at one of them about basic manners, what the list is for, how to ask questions properly and what to do before you even think of asking the list; that guy now consistently does his homework, tries to work it out for himself and if he really doesn't find it or wants another opinion, politely puts the question to the list in a clear and concise way. Of course he now gets all the help he needs, and he's even put a few quite interesting things to the list in the mean time. He has, interestingly, also taken to calling me 'Sir' whenever he asks me a question. I never asked for it, but I have to admit I kinda like it :-)

Sometimes it's necessary to make it quite clear to people what they can and cannot expect from online help, and how to behave there. This used to be perfectly acceptable, but since Eternal September began, the flood of ignorance has gotten so vast that I can fully imagine it can sometimes be hard to remember actually helping people in the middle of all the stupidity.

Also from the article

[Weaselmancer]

Two things:

1) There isn't a difference between the kernel patch and the command line hack. They are equivalent. The command line bit was known beforehand because that was the method used to figure out if this kernel hack would be a good idea. The kernel hack just makes the process transparent.

Linus says: Right. And that's basically how this "patch" was actually tested originally - by doing this by hand, without actually having a patch in hand. I told people: this seems to work really well.

2) Linus recommends the kernel patch:

Linus also says:Put another way: if we find a better way to do something, we should _not_ say "well, if users want it, they can do this *technical thing here*". If it really is a better way to do something, we should just do it. Requiring user setup is _not_ a feature.

Source.

Re:Also from the article

[Anonymous Coward]

They are not the same. The kernel patch groups processes by owning TTY. The bash shell change groups them by session. Source: http://lwn.net/Articles/414817/

Re:Also from the article

[SharpFang]

The difference is the kernel patch is 200 lines of C code, which compiles to several kilobytes of machine code. The shell code needs to spawn a bash process upon startup of every other process, that's several megabytes of RAM and interpreting contents of text scripts that perform the operations.

The final effect may be the same but the overhead of performing the operation is much smaller with the kernel patch.

Re:Also from the article

[Anonymous Coward]

How does this effect servers?

I don't believe that it causes any servers to come into existence.

Re:Also from the article

[Geoffreyerffoeg]

No, incorrect. This is a modification to your .bashrc, which is (already) run every time you start a bash process, within that process (i.e., not a new process). Nothing needs to be spawned on every single process.

Admittedly the bash script does spawn some processes, but a) that's the way .bashrc works, and you have dozens of those in there, and b) it's only one process, a mkdir. The echo and the conditional run within bash itself.

The way that the configuration works, whether done in the kernel or in your .bashrc, is to associate all processes spawned from a single bash shell with a single new scheduling group. This gets you better performance when you're running processes from terminals, by associating logically-similar groups of processes in the kernel instead of letting it see all the processes as a giant pile.

The intended use case, which is pretty clear from the LKML discussion, is to make performance between something intensive (like a compilation) in a terminal and something non-terminal-associated (like watching a movie) better-balanced.

Re:Also from the article

[arth1]

What I don't get is why he uses

if [ "$PS1" ]; then

instead of

if [ -t 0 ]; then

Why the latter? Because it works, that's why:

# sh -c '[ "$PS1" ] && echo tty detected'
#

# sh -c '[ -t 0 ] && echo tty detected'
tty detected

The other way around, if a user has set PS1 in .profile, you'll get false positives, and cron jobs that will consume more than their fair share of resources.

Re:Also from the article

[Bloem]

I've read your comment several times and each time I hear the voice of comic book guy from the simpsons in my head. I'm not trying to be rude so I'm hoping you're not offended by this. I think it has to do with the "no, incorrect" part that your comment starts with.

Re:Also from the article

[jesset77]

Agreed. The last thing the Kernel team needs to worry about is improving the Kernel. D:

Re:Also from the article

[Anonymous Coward]

One never sets PS1 for non-interactive shells, and it's the primary way the shell tells the user's startup scripts whether they're interactive. There's a good chance the PS1 method spares a system call, too :-) It's also what the documentation says to do.

Your [ -t 0 ] approach also fails in cases where an interactive shell is being run on a non-tty. Although almost any shell since about 1990 tends to complain in such cases, at least the PS1 method will still run the right .profile code, and the -t method will not.

Re:How does this work?

[greg1104]

It makes every process spawned by the user that passes through the bash shell add their process ID to a per-user task control group. See the documentation on control groups for more information about exactly what that means, and what what some of the commands involved aim to do. I'm not sure if this is exactly the same impact as the kernel-level patch, which aimed at per-TTY control groups. That might includes some processes that don't pass through something that executes the .bashrc file along the way.

Re:What's the catch?

[blair1q]

You can only put a try on the 22nd catch.

Re:How does this work?

[MBCook]

The kernel has a mechanism to schedule groups of processes, and it has for years. By grouping tasks together, you can make one process (video playing) get the same cpu share as a group of processes put together (compiling code). By doing this (instead of the video processing being equal to just one of the compiling processes), everything feels more interactive, even though it's actually slightly slower.

No one uses scheduling groups because they have to be setup by root and it's not the easiest thing in the world (you have to write stuff into sysfs, I think). No distributions set them up.

The magic kernel patch just adds a simple rule to the scheduler. When a process starts, it goes into a group with the rest of the processes in that TTY (virtual terminal). This means the user doesn't have to do anything and the groups are setup automatically.

Poettering thinks this is somewhat hackish, and that things shouldn't be based on what TTY a process is started on. He made the little script to prove that this can easily be done in userspace.

Linus has rejected this, basically saying that we've had years for people to make something like this and no one did until the kernel patch came along. The patch is simple, reasonable, and doesn't require distributors to ship updated userland files to put processes in groups.

I should note that my understanding comes from LWN, which has had excellent coverage of this on their kernel page, as always. You'll be able to see their articles in two weeks if you're not a member (which is worth it if you like this kind of stuff).

Re:While the bashrc approach may seem attractive

[Shimbo]

True, though it could be done at the distro level, which appears to be the author's plans (the person who wrote this script works for Red Hat, and discussed elsewhere in the thread what Red Hat's plans are for rolling out systemd, which will handle this).

Indeed. "Should we be punting this for userspace tools to handle?" isn't the same question as "should we punt it to the user?".

Re:Poettering is pimping systemd

[Chuck Chunder]

so those of you who've experienced broken sound in recent years

In recent years?

Has Linux ever had a stable sound system?
My recollection is a neverending series of different sound related components (OSS, ALSA, ESD, aRts, Jack etc) of which the best you could say is that they worked most of the time but invariably didn't behave very well in certain cases.

Lennart seems to cop a lot of crap over Pulseaudio but as far as I can tell it's a positive contribution in an area with a lot of historical and legacy issues.

Re:How does this work?

[Corbet]

Thanks for the nice words about LWN! Here's a special link to the LWN article on per-tty group scheduling for Slashdot folks. Hopefully a few of you will like what you see and decide to subscribe.

Re:How does this work?

[sjames]

Gooooooood make -j64 whatever.

Roll over!

People come up with the oddest names for their pets sometimes.

Re:How does this work?

[nitehawk214]

Thanks for the nice words about LWN! Here's a special link to the LWN article on per-tty group scheduling for Slashdot folks. Hopefully a few of you will like what you see and decide to subscribe.

You've got a per-tty mouth.

Re:4 Lines Is Not All. Let's Not Forget...

[h4rr4r]

So then why is your UID so high?

I will get off your lawn now.

Putting the code in the wrong place

[ras]

An early comment on LWN captured the technical argument best, I think, which I guess illustrates both the quality of the articles and posters on LWN. The background to this is we are discussing CPU scheduling. If you don't know what CPU scheduling is, think of it as form of mind reading. I'll illustrate.

Lets say you have asked your computer to do several things, in fact so many that if it follows the usual method of simply dividing its time equally between them it is going to annoy you. The video you watching might start flickering, or the music you are listening will drop out. So obviously the computer must now give more CPU time to playing your movie and less to whatever background task you started, such as that MP3 transcode of your 20,000 song library. Except how is the computer is supposed to know this? This is how we get to mind reading.

The hack we are discussing is essentially the discovery of a way to read the minds of one particular type of computer user - the Linux Kernel developer. The Linux Kernel developer is in the habit of starting huge background jobs called kernel compiles. These kernel compiles take a looong while, so the kernel developers, being very clever people, have invented all sorts of ways of speeding them up. One of those ways is to divide the task into lots of little bits, and then fire off separate tasks to do each. This takes maximum advantage of available CPU cores, soaking up every skerrick of available CPU time. This naturally enough leaves none left over for other important tasks like watching a movie while waiting your kernel compile. In this particular case the default CPU scheduling strategy of giving each task an equal share of CPU is woefully poor, because there might be 20 kernel compile tasks and just one movie watching task, so the movie player ends up with 1/20 of the available CPU time. This isn't enough to play a movie.

The mind reading trick discovered boils down to this: Linux Kernel developers use the linux command line interface to fire off the kernel compile. And it turns out that for years now the kernel has been able group the tasks started from a command line and give that group a single portion of CPU time, as opposed to a equal portion to each task in the group. Thus you only have to split up the CPU time into 2, one portion going to the kernel compiler group and the other going to the movie player. Naturally enough the movie player works real well with a 50% allocation of CPU, and so we have a happy kernel developer.

Now we come to the merits of the two hacks. They both do the job I just described equally well. The difference between them is that one, the kernel patch, is automagic, meaning it happens automatically without anybody having to lift a finger. But it comes at the expense of bloating linux kernel a tiny bit, even for users who won't benefit from it. The other way currently has to be done applied manually using a process the vast majority of Linux users will at best find difficult, tedious and error prone.

Seems like a simple decision eh - lets take the tiny bloat hit and not inflict our long suffering desktop users with yet another Linux user-unfriendly idiosyncrasy. But here is the rub: it doesn't help them. In fact, for some it might have a negative impact (a gstreamer pipeline started from the command line strings to mind). The people who will benefit from this are the ones that use the command line heavily and regularly. People like Linus. Which is why he liked it so much I guess. But these are precisely the people who will have no absolutely no trouble doing it the manual way.