Slashdot Mirror
Significant Interactivity Boost in Linux Kernel
An anonymous reader writes "The Linux kernel team is at it again. Linux creator Linus Torvalds recently proposed a patch to offer interactive processes a boost, greatly benefiting the X desktop, as well as music and movie players. O(1) scheduler author Ingo Molnar merged Linus' patch into his own interactivity efforts, the end result nothing short of amazing... The upcoming 2.6 kernel is looking to be a desktop user's dream come true."
There's no reason not to implement both high-throughput scheduling for big servers and low-latency scheduling for desktops in the same kernel... just mark each process table entry with a bit saying whether this process is 'interactive' (favour fairness and low response times) or 'batch' (favour total throughput and bigger timeslices at the expense of fairness).
-- Ed Avis ed@membled.com
For that matter, why are you trying to do two completely different work loads and environments with the same kernel? You compile the kernel tuned to batch workloads for the server, and recompile the kernel tuned to interactive workloads for the desktop. You have the source.
--binkley
Many people are not gonna compile their own kernel, especially those running something like Redhat in a corporate environment. It makes more sense to code it to work either way.
;-) That means a desktop edition could install a low-latency version and a server edition could install a high-throughput version.
How about this radical idea--
Let Red Hat, SuSE, etc. compile different kernels with different options and install them as needed
LedgerSMB: Open source Accounting/ERP
" As an avid Microsoft fan, one of my biggest beefs was the inferior performance of the Linux GUI and its components."
That would depend on exactly what you talking about. Those linux users running something like Blackbox would laugh at you for saying so. I'd also suggest as a user of both, KDE and XP have about the same interactive performance as well.
There's no doubt Windows still has more polish than Linux as a whole when it comes to the desktop. And while anything that improves any of LInux's many "gui's" is a welcome event, Linux's gui's are hardly inferior performance-wise across the board like your implying.
"Maybee this will finally blur the line between OS's enough to get more people to switch over."
Performance doesn't rate very high on why windows users aren't switching over. Lack of familiar apps and games, lack of widespread OEM bundling, and lack of millions in marketing are what's keeping people from switching over.
If you wanna get rich, you know that payback is a bitch
Or using that snazzy bootloader technology, both kernels can be compiled and the kernel that gets loaded is determined by a variable in lilo.conf, making it easier to set desktop or server room performance in a corporate environment.
"Beware of he who would deny you access to information, for in his heart, he dreams himself your master."
Err, these are competing philosophies. You can't have both types of scheduling going on. Think about it: you have an interactive process which wants to use all the CPU all day long, and you have 6 server processes that want to have balanced scheduling for the clients they are handling. No matter what the scheduler chooses, it is being unfaithful to your bit for each process.
The better answer is to either a) make this option compile time, as someone mentioned, or b) make this option configurable (a la sysctl) at runtime. This would allow distribution maintainers to adjust the setting to match the type of installation they are doing, and users on stock installations to quickly adjust the kind of scheduling they have, just like the little check boxes in windows NT/XP.
-- Patience is a virtue, but impatience is an art.
Its not just a UI issues; it does relate to the kernel in that the kernels job is to manipulate process priorities and give CPU cycles where they "should best be given". This is actually best done at the kernel level, and NOT the GUI level, because the GUI does not know about the other non-GUI processes is "competing with". I've felt for a long time that something like this should be done in both Windows AND Linux.
Windows is TERRIBLE at this. Consider the following scenario, which most here who here run Windows XP will be able to identify with. You boot up, you've just logged in. The task bar is there on the screen, the start button there, you click on it. And nothing happens. You wait. Still nothing happens. You wait some more. You start to get annoyed and click the start button a few more times. The hard disk is grinding away while Windows XP does all sorts of "invisible stuff" in the background. The computer is about as responsive as a brick. Then after anything from 20 seconds to a minute, the start menu suddenly opens and closes rapidly in quick succession a half dozen times.
THIS IS NOT HOW COMPUTERS SHOULD BEHAVE. Its pathetic. This is a perfect example of the necessity of this. The task bar process doesn't know about all those other background processes hogging CPU after you log in; there is no elegant way for it to magically know when to set its priority temporarily high, and for how long. But the kernel can say, OK, the user is trying to press a button, we must respond, and temporarily boost the start bar (explorer.exe) process and block the others.
On desktop machines (i.e. not servers), user input is the most important thing. If the user presses a button, something must happen. The kernel should be continually shifting priorities around to where the user is focusing his/her input.
Dropping X11 would be a HUGE mistake. X11 as a system is far more flexible than the gui system of windows, ofcourse this flexibility can introduce a performance hit, but theres many other things in modern os`s which sacrifice performance for features, ease of use, maintainability etc..
But even considering the larger and more featured system, X11 is as fast, or faster, than windows on all but one of my machines, the one where its slower is because the graphics card is very poorly supported by X.
What causes slowness more than X11 itself, is the programs running on top of it... KDE for instance, its hardly a speed demon compared to say, windowmaker.
You dont need a high end videocard to make X smoothe, you just need one that`s well supported... my PCI ATI Rage Pro works perfectly, as does my Elsa GLoria Synergy, both are oldish 8mb pci cards.
Any system will completely suck with poor drivers, try configuring windows to use generic vga or vesa drivers if you want a laugh.
X11 is FAR superior to any local-display-only gui system, i have several machines here, and 1 monitor for X, apps running from each machine and displayed here and interacting smoothly with each other and with locally running apps.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Noone claimed that Linus invented the technique (besides, Linux too has been using it for some time already). The only claim made about him is that he proposed that the way in which Linux uses it be changed. (It would help to actually read the article.)
Linux user since early January 1992.
X is the most important example, since it doesn't help how much CPU your xterms or other X clients you run get if X doesn't get enough CPU time to service them, as if X doesn't get enough time the only thing extra CPU time will give your x clients is the ability to go back to sleep faster and more often.
Realtime scheduling is something else alltogether. Realtime scheduling is about predictability, not about CPU time allocated. With a realtime scheduler you can guarantee that task A get some time at least every 10ms, for instance, but if you're maxing out the CPU you still need a way of deciding which tasks have priority, or reduce their overall time slices.
The kernel patches in question attempts to decide which tasks to give priority automatically, instead of resorting to hacks like using nice on specific processes. It achieves that by making the assumption that if task A is interactive, and it frequently waits on B, then task B needs to get more CPU too.
Since a high load desktop scenario will likely have lots of clients waiting on X the result is that X will get more CPU even if X itself isn't an interactive task, and hence the machine will hopefully feel more responsive.
The way this is being accomplished is good because it doesn't special case - any non-interactive task that provides vital services to interactive tasks will get more CPU (though in this particular implementation, I believe only if they communicate using Unix domain sockets), without the user or developers having to guess which processes should get it.
What's being described here is called a priority inversion, where a high-priority task makes a request of some service running at a lower priority and gets hung up behind it.
Which is wrong. Did you read the article?
Priority inversion is, as you explained yourself, about a high priority task effectively getting a low priority by being dependend and therefore waiting on a low priority task.
The article is about tasks at the same priority[1]. The task scheduler distinguishes between interactive and non-interactive tasks in order to improve latency where the user cares.
Beforehand, the behaviour failed on a slow, loaded system to recognize the X server as interactive, because then X looks like a CPU-hog[2]. That resulted in freezes of several seconds[3]. Simply speaking, the patch solves this by passing some interactivity points between processes.
You could have easily seen that this is not about priority inversion as one of the suggested work-arounds was to simply increase the niceness of the X process (which wouldn't help, if priority inversion had been the problem).
Regarding QNX: As good as it is as a RTOS, as bad it fails to do something sensible when you have too much processes at the same priority. Having a reasonably working system presumes that each task is assigned an appropriate priority. Of course, the people at QNX did a decent job on the default priorities.
[1] It may have an impact on tasks of different priority. I did not care to investigate that aspect, because that is of minor importance to what the patch is about.
[2] And for the scheduler, interactivity is determined by a process going to sleep often (by waiting for interaction).
[3] For non-interactive processes it is beneficial to do them in larger hunks, i.e. let 5 seconds other processes do their work, then work 5 seconds, instead of having 0.01 second slices and do the switching all the time.
Keep an eye on which arguments are silently dropped in replies. Not always, but often times it's very telling.
Ugh. I don't buy it.
To put it in perspective, lots of Unix has a big organization problem. X is just emblematic. It's "lower-level" APIs are a big stinking mess. Ever tried to program against it without a super-high-level bit of middleware? Then let's talk about how nice it is. If you're not up on this, try reading JWZ's rants on it (many written as he was porting Netscape)? X is a 4 foot high sandwich of crap, layer after layer between you and the display, full of massive, sucking complexity, the bugs, inefficiency... even during this supposedly wonderful "network transparent" windowing this foul stew shows its colors, as no combination of two applications or X servers quite looks the same. It's a verifiability nightmare, too, of course (and for instance, disabling X's many attempts to listen and talk on the network are one of the first things you do to secure a machine properly - and for real security, you avoid installing X altogether).
The API design itself is atrocious. The much-touted "flexibility" is really code for laziness - it was a lot of work to do a proper GUI, so no one did it. The mishmash of X server extensions, window managers, font handling systems, etc. that's been cobbled together has led to a nightmare for both programers and users, as any given application doesn't just require "X", but a complex recipe of libraries and versions, and an end-user experience where no two applications look or act the same... or even remotely similar... Where cutting and pasting between windows is a pipe dream, and young geniuses still struggle to configure fonts properly for linux distributors.
Or to just put it plainly, as my friend (who from time to time would write X windows gadgets) would say, it's only about twice as hard as managing the video memory yourself.
"And thank god it's not all standardized, or we'd never have had all those wonderful experiments with different ways to do a GUI that never actually happened." In practice, no system is immune from its initial design choices, and it's been an endless series of awful MacOS knockoffs, multi-button madness, color-pallete spinning goofiness. Is X11 a "GUI experimenters toolbench?" Then I think it's time for something a little more grounded in everyday realities of computer use.
I'm not even warmed up yet. I mean, X is still peppering the filesystem with a hedge-maze of exotically formatted text files describing the hex colors of every pixel of the trim of every window for a variety of appliations and classes in a complex inheritance and assignment scheme that few X developers even understand. Check it out, your XDefaults are "human readable."
Shall we even discuss its security model?
Modern Linux has tried to make its peace with X through wrappers, and we write against Tcl/Tk, Qt, inside the Gnome or KDE framework, and yet still the focus groups come back crying... we try to blame overfamiliarity with windows, but the problems are bigger... all of Unix (and of course Linux) suffers from the same class of problems that X does; as, for instance, an application needs to prompt you to insert a series of CD's, but there is no "single, authoritiative, standard" place to go find out what CD drives are installed on the computer, and what their device names are (yes, we know what they _usually_ are), and finding out if any of the CDs are already inserted involves parsing the text output of a proc file or a mount command, and so on and so forth... And all of this is being done by a messy bash script... so it's no surprise this functionatlity is broken even in, for instance, RedHat's own v8 package manager... I hope you can grasp the metaphor.
It's a mess. Patches won't clean it up. Frankly, it's time we took the whole GUI back to the drawing board. But even if MacOS is the end-all/be-all, we can do it a hell of a lot better than we do in X.
Following are some choice quotes from Don Hopkins' essay:
X-Windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X-Windows is to memory as Ronald Reagan was to money. Years of "Voodoo Ergonomics" have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards.
X has had its share of $5,000 toilet seats -- like Sun's Open Look clock tool, which gobbles up 1.4 megabytes of real memory! If you sacrificed all the RAM from 22 Commodore 64s to clock tool, it still wouldn't have enough to tell you the time. Even the vanilla X11R4 "xclock" utility consumed 656K to run. And X's memory usage is increasing.
...
X was designed to run three programs: xterm, xload, and xclock. (The idea of a window manager was added as an afterthought, and it shows.) For the first few years of its development at MIT, these were, in fact, the only programs that ran under the window system. Notice that none of these program have any semblance of a graphical user interface (except xclock), only one of these programs implements anything in the way of cut-and-paste (and then, only a single data type is supported), and none of them requires a particularly sophisticated approach to color management. Is it any wonder, then, that these are all areas in which modern X falls down?
...
As a result, one of the most amazing pieces of literature to come out of the X Consortium is the "Inter Client Communication Conventions Manual," more fondly known as the "ICCCM", "Ice Cubed," or "I39L" (short for "I, 39 letters, L"). It describes protocols that X clients ust use to communicate with each other via the X server, including diverse topics like window management, selections, keyboard and colormap focus, and session management. In short, it tries to cover everything the X designers forgot and tries to fix everything they got wrong. But it was too late -- by the time ICCCM was published, people were already writing window managers and toolkits, so each new version of the ICCCM was forced to bend over backwards to be backward compatible with the mistakes of the past.
The ICCCM is unbelievably dense, it must be followed to the last letter, and it still doesn't work. ICCCM compliance is one of the most complex ordeals of implementing X toolkits, window managers, and even simple applications. It's so difficult, that many of the benefits just aren't worth the hassle of compliance. And when one program doesn't comply, it screws up other programs. This is the reason cut-and-paste never works properly with X (unless you are cutting and pasting straight ASCII text), drag-and-drop locks up the system, colormaps flash wildly and are never installed at the right time, keyboard focus lags behind the cursor, keys go to the wrong window, and deleting a popup window can quit the whole application. If you want to write an interoperable ICCCM compliant application, you have to crossbar test it with every other application, and with all possible window managers, and then plead with the vendors to fix their problems in the next release.
In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex nonsolutions to obsolete nonproblems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry's standard naked emperor.
Using these toolkits is like trying to make a bookshelf out of mashed potatoes.
- Jamie Zawinski
...
The fundamental problem with X's notion of client/server is that the proper division of labor between the client and the server can only be decided on an application-by-application basis. Some applications (like a flight simulator) require that all mouse movement be sent to the application. Others need only mouse clicks. Still others need a sophisticated combination of the two, depending on the program's state or the region of the screen where the mouse happens to be. Some programs need to update meters or widgets on the screen every second. Other programs just want to display clocks; the server could just as well do the updating, provided that there was some way to tell it to do so.
...
.Xdefaults", but if he happens to have copied Fred's .xsession, he does an xrdb .xresources, so .Xdefaults never gets read. Susie either doesn't xrdb, or was told by someone once to xrdb .Xdefaults. She wonders why when she edits .Xdefaults, the changes don't happen until she 'logs out', since she never reran xrdb to reload the resources. Oh, and when she uses the NCD from home, things act `different', and she doesn't know why. "It's just different sometimes."
.xresources (or was it a file that was #included in .xresources) of the form *fucked*fontList: 10x22, which he copied from Steve who quit last year, and that of course that resources is 'more specific' than his, whatever the fuck that means, so it takes precedence. Sorry, guy. He can't even remember what application that resource was supposed to change anymore. Too bad.
What this means is that the smarter-than-the-average-bear user who actually managed to figure out that
snot.fucked.stupid.widget.fontList: micro
is the resource to change the font in his snot application, could be unable to figure out where to put it. Suzie sitting in the next cubicle will tell him, "just put it in your
Joe Smartass has figured out that XAPPLRESDIR is the way to go, as it allows him to have separate files for each application. But he doesn't know what the class name for this thing is. He knows his copy of the executable is called snot, but when he adds a file Snot or XSnot or Xsnot, nothing happens. He has a man page which forgot to mention the application class name, and always describes resources starting with '*', which is no help. He asks Gardner, who fires up emacs on the executable, and searches for (case insensitve) snot, and finds a few SNot strings, and suggests that. It works, hooray. He figures he can even use SNot*fontList: micro to change all the fonts in the application, but finds that a few widgets don't get that font for some reason. Someone points out that he has a line in his
...
On the whole, X extensions are a failure. The notable exception that proves the rule is the Shaped Window extension, which was specifically designed to implement round clocks and eyeballs. But most application writers just don't bother using proprietarty extensions like Display PostScript, because X terminals and MIT servers don't support them. Many find it too much of a hassle to use more ubiquitous extensions like shared memory, double buffering, or splines: they still don't work in many cases, so you have to be prepared to do without them. If you really don't need the extension, then why complicate your code with the special cases? And most applications that do use extensions just assume they're supported and bomb if they're not.
Want to Know How to Cheat the GPL? Read On!