Slashdot Mirror
Significant Interactivity Boost in Linux Kernel
An anonymous reader writes "The Linux kernel team is at it again. Linux creator Linus Torvalds recently proposed a patch to offer interactive processes a boost, greatly benefiting the X desktop, as well as music and movie players. O(1) scheduler author Ingo Molnar merged Linus' patch into his own interactivity efforts, the end result nothing short of amazing... The upcoming 2.6 kernel is looking to be a desktop user's dream come true."
Absolute astounding. I am in complete and utter shock over this. Truly, truly the most amazing thing I've ever seen or heard.
Now what the hell is this article about?
At it again? At what again? That sorta makes it sound like a girls gone wild video or something. Kernel Dev's Gone Wild volume 3, where Ingo and Linus bare their breasts for beads at a Linux user conference in Tampa Bay - no, that's just too strange...
Oh, one more thing:
Hello, my name is Ingo Molnar. You killed my father: prepare to die.
Robots are everywhere, and they eat old people's medicine for fuel.
There's no reason not to implement both high-throughput scheduling for big servers and low-latency scheduling for desktops in the same kernel... just mark each process table entry with a bit saying whether this process is 'interactive' (favour fairness and low response times) or 'batch' (favour total throughput and bigger timeslices at the expense of fairness).
-- Ed Avis ed@membled.com
Actually, Linus's patch doesn't improve things any better than the scheduler patch it is Linus's patch combined with the scheduler patch that make it such a huge improvement. Again... its the COMBO patch that's arousing so much excitement.
Inconceivable!
Something forms itself from the silent void of the empty mailing lists and the noisy chaos of the crowded mailing lists. It shapes and protects us, it entertains and challenges us, it aids us in our journey through the ether world of software. It is mysterious; it is at once source code and yet object code. I do not know the name, thus I will call it the Tao of Linux.
If the Tao is great, then the box is stable. If the box is stable, then the server is secure. If the server is secure, then the data is safe. If the data is safe, then the users are happy.
In the beginning there was chaos in Unix.
Tanenbaum gave birth to MINIX. MINIX did not have the Tao.
MINIX gave birth to Linux 0.1 and it had promise.
Linux gave birth to v1.3 and it was good.
v1.3 gave birth to v2.0 and it was better.
Linux has evolved greatly from its distant cousins of the old. Linux is embodied by the Tao.
The wise user is told about the Tao and contributes to it. The average user is told about the Tao and compiles it. The foolish user is told about the Tao and laughs and asks who needs it.
If it were not for laughter, there would be no Tao.
Wisdom leads to good code, but experience leads to good use of that code.
The master Cox once dreamed that he was a Kernel. When he awoke he exclaimed: "I don't know whether I am Cox dreaming that I am a Kernel, or a Kernel dreaming that I am Cox!"
The master Linus then said: "The Tao envelopes you. You shall create great code for Linux."
"On the contrary," said Cox, "The Tao has already created the code, I will only have to find it and write it down."
A master was explaining the nature of the Tao to one of his students:
"Is the Tao in the VM subsystem?" he asked. "Yes," replied the master.
"Is the Tao in the scheduler?" he queried again. "The Tao is in the scheduler."
"Is the Tao even in the modules?". "It is even in the modules," said the master.
"Is the Tao in the Low-Latency Patch?"
The master frowned and was silent for much time.
"You fail to understand the Tao. Go away."
The Tao is the yin and the yang. It is the good and the evil, it is everything and yet it is nothing, it is the beginning and the end.
The Tao was there at the kernel compile, and it will be there when the kernel panics.
A novice user once asked a master: "Why compile in C when C++ is more popular?"
"Why a monolythic kernel when Mach is more popular?"
"And why use ReiserFS when ext2 is more popular?"
The master sighed and replied: "Why run Unix when NT is more popular?"
The user was enlightened.
A frustrated user once asked a master: "My kernel has panicked, should I post to lkml?"
"No," replied the master, "You will only bother the Tao."
"Should I rm -rf?"
"No, you will have wasted the Tao's time."
"Well should I search the web?"
"You will search for all eternity," said the master.
"Perhaps I should try FreeBSD?"
"Then you will have disgraced the Tao."
"I suppose I could try gdb," said the user.
The master smiled and replied: "Then you will have made the Tao stronger."
A stubborn user once told a master: "I run version 2.2. I always have, and I always will."
The master replied: "You are foolish and do not understand the Tao. The Tao is dynamic and ever changing. Linux strives for the perfection that is the Tao. It flows from version to version with peace."
"So my Linux does not have the Tao, so what?" said the foolish user. "Oh your Linux is of the Tao," said the master. "However, the Tao of Linux follows the Tao of the C library. One day the C library will change, and your Linux will be left behind." The user was silent.
An angry user once yelled at a master:
"My Linux has panicked! What lousy software it is, I hate it so!"
"You are insulting the Tao," said the master. "The Tao is everywhere bringing order to hundreds of networks, aiding thousands of users, and fighting that of which we call the 'lame.' Do not disrespect the Tao; however, the Tao will forgive you."
"I apologize," said the user, "And I will be more forgiving the next time the Tao fails me."
"The Tao has not failed you, it is you that has failed the Tao," said the master. "The Tao is perfect."
The Tao decides if a kernel shall compile, or if it shall abort.
The Tao decides if a kernel shall boot, or if it shall freeze.
The Tao decides if a kernel shall run, or if it shall panic.
But, the Tao does not decide if a box will have no hardware failures. That is a mystery to everyone.
A young master once approached an old master: "I have a LUG for Linux help. But, I fail to answer my students' problems; they are above me."
The master replied: "Have you taught them of the Tao?" he asked. "How it brings together man and software, yet how it distances them apart; how if flows throughout Linux and transcends its essence?"
"No," exclaimed the apprentice, "These people cannot even get the source untarred."
"Oh, said the master, "In that case, tell them to RTFM."
A master watched as an ambitious user reconstructed his Linux.
"I shall make every bit encrypted," the user said. "I shall use 2048 bit keys, three different algorithms, and make multiple passes."
The master replied: "I think it is unwise."
"Why?" asked the user. "Will my encryption harm the mighty Tao, which gives Linux life and creates the balance between kernel and processes? The mighty Tao, which is the thread that binds the modules and links them with the core? The mighty Tao, which safely guides the TCP/IP packets to and from the network card?"
"No," said the master, "It will hog too much cpu."
The core is like the part of the mind that is static. It is programmed at a child's creation and cannot be changed unless a new child is made; unless a new kernel is compiled.
The modules are like the part of the mind that is dynamic. It is reprogrammed every time one learns new knowledge; every time one learns better code.
One is yin, the other yang. Each is nothing without the other.
A novice came to lkml and inquired to all the masters there: "I wish to become a master. Must I memorize the Linux header files?"
"No," replied a master.
"Must I submit code to Bitkeeper?"
"No," replied the master.
"Must I meditate daily and dedicate my life to Linux?"
"No," replied the master again.
"Must I go on a quest to ponder the meaning of the Tao?"
"No. A master is nothing more than a student who knows something of which he can teach to other students."
The novice understood.
And thus said the master:
"It is the way of the Tao."
A user came to a master who had great status in lkml. The user asked the master: "Which is easier: implementing new features to the kernel or documenting them?"
"Implementing new features," replied the master.
The confused user then exclaimed:
"Surely it is easier to write a few sentences in the man page than it is to write pages of code without error?"
"Not so," said the master. "When coding, the Tao of Linux opens my eyes wide and allows me to see beyond the code, to let the source flow from my fingers, to implement without flaw. When documenting, however, all I have to work with is a C in high school English."
He who compiles from the stable tree is stubborn
and unwilling to change, but is guaranteed reliability.
He who compiles from the current tree is wise but perhaps too conformist, but is guaranteed steadiness.
He who compiles from the unstable tree is adventurous and is guaranteed new innovations: some good, some bad.
He who compiles straight from Bitkeeper is brave but guaranteed turbulence.
They are all of the Tao. One shall respect the old, and debug the new; none shall argue over which is greatest.
There once was a user who scripted in Perl: "Look at what I have to work with here," he said to a master of core, "My code is interpreted dynamically, the syntax is unique and simple, I have sockets, strings, arrays, and everything I could ever need. Why don't you stop meddling in C and come join me?"
The C programmer described his reasoning to the scripter: "Scripting is to C as ebonics is to Latin. If the scripter does not grow beyond that of which he scripts, he will surely {die}. Besides, without C, how can there be script?"
The scripter was enlightened, and the two became close friends.
For that matter, why are you trying to do two completely different work loads and environments with the same kernel? You compile the kernel tuned to batch workloads for the server, and recompile the kernel tuned to interactive workloads for the desktop. You have the source.
--binkley
Im sorry, its not the kernel that makes a desktop OS what it is, its the userinterface on top. Linux isnt going to be truely Desktop friendly untill X11 gets replaced with something that doesnt completely suck. You shouldnt need a high end video card to make X11 nice and smooth or have to use a stripped down UI. If Linux wants to set it self apart from all the other OS's its going to need its own desktop engine. I do agree that the 2.6 kernel is going in the right direction, I just belive the rest of the OS is being left really far in the past.
My 2 1/2 cents Canadian
> Uh...check out Windows 2000 scheduling algos
Sure thing. What was the address of their anoncvs servers again? Oh wait, I forgot the "turn into a powerful government and sign a couple hundred non-disclosure agreements"-requirement.
Now, I am a long-time user of Windows, but am (and have been always) increasingly tempted by switching to a Linux-based distribution, probably Redhat, on my main desktop machine.
With that lack-of-linux-knowledge, could someone explain why precisly this is a "Significant Interactivity Boost in (the) Linux Kernel"? Thank you.
It's:
My name is Ingo Molnar. You kill -9'd my parent process. Prepare to die()
Stop stealing that intellectual property from SCO already. Have you no shame? The gig is up: there's no way you could keep putting this stuff out without ripping off the hard working SCO programmers.
Fermat's other theorem: "I have a simple proof, but I can't write it down as I fear it's a DMCA violation to discuss it"
Except that doesn't work if the X app is being displayed locally but run remotely. Or at least it doesn't seem to.
Well, okay. Look. I just opened up my X server here on my mac os X box, sshed with X tunneling to my university's Solaris box, opened up xchat, selected some text, and attempted to paste it into a nearby xterm. Oh, hey, guess what, didn't work. I tried what you said. I selected some text, i clicked on the xterm, i tried middle-clicking and right-clicking. Nothing there. Care to tell me what i'm doing wrong?
Anyway, the whole select-to-copy thing is HORRIBLE GUI design. What if you want to select something in order to edit it without *blowing away* the clipboard? What if your hand slips and you select a couple letters of text by accident before you can paste something important into its destination? I, personally, have intense problems with the copy-paste thing because at some point i picked up the stupid habit of often scrolling by selecting text and dragging off the edge of the window, which will obliterate the textboard. And, worst of all, there's that nagging little question: let's say i'm editing a file, and i want to select some text in the part of the document i'm editing, "cut" it, scoll up to the top of the document, delete part of a paragraph, write a couple lines, and then paste what i just cut. What X's copy/paste means is that i must select the text i want to move to copy (making sure not to delete it yet, becuase it would be too easy to accidentally select text and copy over what i've written, losing it forever), scroll up, click where i want it to go, paste, and then delete and rewrite the text around it, scroll back down, and then delete the text i copied. Yeah, way to go on making the interface fit the needs of the user. Dammit.
And then there's the fact that, still, mostly due to the broken silliness of X copy&paste, most applications don't quite work the same, becuase they've all fucking implemented the clipboard in nonstandard ways because those unstandard ways are "better". Which they are, unless for some silly reason you want to copy and paste between applications. We've got the "clipboard" and the "cut buffer" and i don't know what either means, and lately some GNOME apps and such have taken to signing up with a sane (i.e., "copy" and "paste" are commands, and as such require a menu use or a key combination). And then vim has like its ten little internal clipboards, and emacs has some clipboard system i don't even pretend to know the first thing about, and i mostly use vim as my text editor in unix. And none of these apps i've ever seen give the option of choosing which copy/paste behavior you want: i mean, none of them will give you a nice little preference that says "select to copy" vs "select 'copy' from menu to copy" vs "have ten little internal cycling cut buffers with some arcane method of manipulation". And i still don't know how copy/paste within vim is supposed to interact with other X apps. I'd test it right now, but for some reason still unknown to me, i can't get gvim to run over my ssh-tunneling setup. When i try, it says:
X11 connection rejected because of wrong authentication.
XIO: fatal IO error 32 (Broken pipe) on X server "localhost:13.0"
after 0 requests (0 known processed) with 0 events remaining.
The connection was probably broken by a server shutdown or KillClient.
Maybe it doesn't like my MIT magic cookies, or something? But I digress. Face it. Copy and paste is still the most broken thing about X, and that's saying a lot. And maybe i'm just dense, but i still can't figure out how to change my X keyboard mapping on these silly Solaris boxes.
-- super ugly ultraman
bashing linux/x/kde/whatever speed, I'm willing to bet you've never doen a full build (ala Gentoo) and actually optimized it for your system...have you? KDE 3.1 is as fast or faster than windows XP on my 1ghz box...it took a while to build, but it's well worth it.
Many people are not gonna compile their own kernel, especially those running something like Redhat in a corporate environment. It makes more sense to code it to work either way.
;-) That means a desktop edition could install a low-latency version and a server edition could install a high-throughput version.
How about this radical idea--
Let Red Hat, SuSE, etc. compile different kernels with different options and install them as needed
LedgerSMB: Open source Accounting/ERP
There was a time when idiots did not walk the earth.
Linux still screams, I have a single server with two gig's of ram in it that runs 100 desktops (KDE) simultaneously. Yes it indeed takes alot of ram to run all of the new software. But for a machine that runs 2200 processes that is a impressive feat. It is a dual processor box and I have yet to see it reach over 30% processor utilization, a testament to the efficency of the kernel.
Software today requires a ton of ram, this has nothing to do with efficency of the linux kernel.
Along with this goes the idiots that think there is something wrong with X. I run this stuff in a corporate environment and X windows is linux's biggest strength. Remove X Windows and I would have to eliminate our corporate use of Linux.
Got Code?
I am curious... which GUI do you use in Linux? What speed processor and how much RAM do you have? Which distribution (or kernel) of Linux do you use?
I ask because it has been my experience that Linux is already considerably more responsive (in terms of GUI performance) than Windows. I use KDE 3.1 with Linux 2.4.20 and I have 512 megs of RAM and a 1.46 Ghz processor.
Now, least people accuse me of trolling (or of pandering to the Linux crowd), I should point out that I am not sure why Windows is so unresponsive. It seems to have something to do with hard drive access. It seems to me that Windows XP is acting like I'd expect it to if I didn't have DMA enabled for my hard drives. Basically, whenever I access the hard drive, the GUI becomes almost completely unresponsive, sometimes taking almost a minute to fire up even a browser. I have checked, though, and I do have DMA enabled.
So I truly do not know what is going on with Windows, but in Linux I just don't have these problems. Under heavy disk access, it may take a few seconds to fire up a browser in Linux, but that's it. MP3s keep playing, my apps are still responsive, etc. etc.
Oceania has always been at war with Eastasia.
" As an avid Microsoft fan, one of my biggest beefs was the inferior performance of the Linux GUI and its components."
That would depend on exactly what you talking about. Those linux users running something like Blackbox would laugh at you for saying so. I'd also suggest as a user of both, KDE and XP have about the same interactive performance as well.
There's no doubt Windows still has more polish than Linux as a whole when it comes to the desktop. And while anything that improves any of LInux's many "gui's" is a welcome event, Linux's gui's are hardly inferior performance-wise across the board like your implying.
"Maybee this will finally blur the line between OS's enough to get more people to switch over."
Performance doesn't rate very high on why windows users aren't switching over. Lack of familiar apps and games, lack of widespread OEM bundling, and lack of millions in marketing are what's keeping people from switching over.
If you wanna get rich, you know that payback is a bitch
Or using that snazzy bootloader technology, both kernels can be compiled and the kernel that gets loaded is determined by a variable in lilo.conf, making it easier to set desktop or server room performance in a corporate environment.
"Beware of he who would deny you access to information, for in his heart, he dreams himself your master."
Every new release of Windows wasn't vastly slower and more bloated then the release before it...
Oh wait. No there wasn't.
If you wanna get rich, you know that payback is a bitch
Err, these are competing philosophies. You can't have both types of scheduling going on. Think about it: you have an interactive process which wants to use all the CPU all day long, and you have 6 server processes that want to have balanced scheduling for the clients they are handling. No matter what the scheduler chooses, it is being unfaithful to your bit for each process.
The better answer is to either a) make this option compile time, as someone mentioned, or b) make this option configurable (a la sysctl) at runtime. This would allow distribution maintainers to adjust the setting to match the type of installation they are doing, and users on stock installations to quickly adjust the kind of scheduling they have, just like the little check boxes in windows NT/XP.
-- Patience is a virtue, but impatience is an art.
AmigaOS had something similar long before windows did, and amigaos was always a FAR more responsive system than windows, even from the first version.
AmigaOS and windows are both fairly similar in purpose and features tho, unix is more tailored to heavy duty server use, and thin clients, and ofcourse its far more powerfull and flexible. Thus you have a powerfull stable kernel, multiuser abilities, and features such as remote displays and authorization in X.
True, windows has tried to copy some of the age old unix features, but the basic design remains the same with extra things kludged in as an afterthought, and theres still no X style remote apps managed by your local wm, its whole desktop or nothing.
So while windows may be faster on a single machine, due to its simpler design, once you scale up.. to say one server serving hundreds of thin clients, unix really pulls into the lead.
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Its not just a UI issues; it does relate to the kernel in that the kernels job is to manipulate process priorities and give CPU cycles where they "should best be given". This is actually best done at the kernel level, and NOT the GUI level, because the GUI does not know about the other non-GUI processes is "competing with". I've felt for a long time that something like this should be done in both Windows AND Linux.
Windows is TERRIBLE at this. Consider the following scenario, which most here who here run Windows XP will be able to identify with. You boot up, you've just logged in. The task bar is there on the screen, the start button there, you click on it. And nothing happens. You wait. Still nothing happens. You wait some more. You start to get annoyed and click the start button a few more times. The hard disk is grinding away while Windows XP does all sorts of "invisible stuff" in the background. The computer is about as responsive as a brick. Then after anything from 20 seconds to a minute, the start menu suddenly opens and closes rapidly in quick succession a half dozen times.
THIS IS NOT HOW COMPUTERS SHOULD BEHAVE. Its pathetic. This is a perfect example of the necessity of this. The task bar process doesn't know about all those other background processes hogging CPU after you log in; there is no elegant way for it to magically know when to set its priority temporarily high, and for how long. But the kernel can say, OK, the user is trying to press a button, we must respond, and temporarily boost the start bar (explorer.exe) process and block the others.
On desktop machines (i.e. not servers), user input is the most important thing. If the user presses a button, something must happen. The kernel should be continually shifting priorities around to where the user is focusing his/her input.
The interactivity boost has been in linux from Linux 0.99. This is a new class of boost, increasing interactive proccess priority and helper proccess priority too.
I have to say that KDE 3.1 is pretty snappy on my measly PII 400 with 320 MB of RAM under Gentoo Linux.
Saying KDE is slow is fudding.
Noone claimed that Linus invented the technique (besides, Linux too has been using it for some time already). The only claim made about him is that he proposed that the way in which Linux uses it be changed. (It would help to actually read the article.)
Linux user since early January 1992.
word! AmigaOS had preemptive multitasking when most people were still using DOS on their PCs.
Now that you say it, I also realize that no Windows machine that I have been using has ever been as responsive as the old Amiga. Of course this is also a hardware issue: the Amiga had pretty strict timing for all I/O operations and memory access. The different subsystems had their own time slots in DMA, which was based on the video refresh timing. To a certain amount other subsystems, like the "Blitter", could steal DMA cycles from the CPU. There was even a chip, the "Copper", that could perform certain actions based on the position of the electron beam of the monitor. In my opinion, this chip was the key to most of the impressive effects that could be produced by the Amiga. OTOH, such a design is pretty hard to scale w.r.t. speed. In the end, the Amiga declined because Commodore neglected hardware development for too long.
There are various solutions to this problem. It sounds like the Linux kernel people are trying priority inheritance via the messaging system (local sockets). QNX has had that for over a decade. Because QNX does almost everything, including all I/O, by message passing, it has to do this right. In the UNIX world, message-passing was added quite late, in BSD, and X is one of the few interactive programs that uses socket communication on the local machine. Sockets are used mostly to talk across the network. So support for time-critical local sockets isn't very good. UNIX pipes were the original UNIX interprocess communication mechanism, and they were intended as batch-like devices. Sockets look, and work, a lot like pipes. This legacy is the real cause of the problem.
Of course, the reason Linux users actually want this feature is so that they can play their pirated MP3s in the background while using X-windows.
Not quite right. Every multi-tasking system since the first few in the 70s has had the concept of running interactive apps with a higher priority. It's a very obvious improvement.
The non-obvious improvements are things like making the applications that depend on, or are depended on, by the interactive app, run faster. There are also additional tweaks to this that that are being considered such as giving interactive programs a smaller time-slice, but more of them, so it'll do things like paint the windows properly in respose to your movements, but it won't bog the rest of the system down.
Technically, scheduling tweaks do add to code complexity, but only in such a tiny way. Linus's patch was five lines. And Linus is very concerned with making sure patches are self-contained and, when possible, aren't spread out, a few lines in many different areas. He's got a very good, very "correct" attitude about design. It comes from him being happy with Linux for years now, he's not rushing to any specific point so it becomes useful. He's willing to put the time in to do it right.
Anyways, this is to say that most kernel patches don't lead to complexity, most decrease the complexity of the code. Linus has often sent patches back to be done the "right way" instead of allowing a hack. This tweak is so small and self-contained that it can't really be said to add complexity to anything.
Err, these are competing philosophies. You can't have both types of scheduling going on. Think about it: you have an interactive process which wants to use all the CPU all day long, and you have 6 server processes that want to have balanced scheduling for the clients they are handling. No matter what the scheduler chooses, it is being unfaithful to your bit for each process.
Check out the Solaris 9 Resource Manager, which can do both types. It allows you specify at a high level how much of the system's resources each group of processes gets under which conditions. You could say for example, group A (interactive) gets up to 100% unless group B (batch) needs some, in which case allow B up to 30% during the day and up to 70% at night. You could do this sort of thing in VMS over a decade ago. Also, even if the underlying OS doesn't give you the capability, an Oracle server running batch and interactive tasks can do it too.
X is the most important example, since it doesn't help how much CPU your xterms or other X clients you run get if X doesn't get enough CPU time to service them, as if X doesn't get enough time the only thing extra CPU time will give your x clients is the ability to go back to sleep faster and more often.
Realtime scheduling is something else alltogether. Realtime scheduling is about predictability, not about CPU time allocated. With a realtime scheduler you can guarantee that task A get some time at least every 10ms, for instance, but if you're maxing out the CPU you still need a way of deciding which tasks have priority, or reduce their overall time slices.
The kernel patches in question attempts to decide which tasks to give priority automatically, instead of resorting to hacks like using nice on specific processes. It achieves that by making the assumption that if task A is interactive, and it frequently waits on B, then task B needs to get more CPU too.
Since a high load desktop scenario will likely have lots of clients waiting on X the result is that X will get more CPU even if X itself isn't an interactive task, and hence the machine will hopefully feel more responsive.
The way this is being accomplished is good because it doesn't special case - any non-interactive task that provides vital services to interactive tasks will get more CPU (though in this particular implementation, I believe only if they communicate using Unix domain sockets), without the user or developers having to guess which processes should get it.
Not at all. Indeed the lkml thread referenced was about getting good performance when there are some non-interactive processes such as gcc and some interactive ones such as MP3 players or the X server.
Suppose there are two CPU-bound processes marked as 'batch' and one process marked as 'interactive' which spends most of its time waiting for user input, but needs to respond quickly in short bursts when that input happens. The interactive process will get high priority and preempt the two batch processes when it needs to run; but when it goes back to sleep the two batch processes are scheduled with long timeslices.
-- Ed Avis ed@membled.com
This example perfectly illustrates what linus was having problems with when Ingo suggested that users just nice X up. It hides the problem, but doesn't actually fix it. Windows XP plays tricks on users to make them think that it's faster than Win2K or even NT. It loads the GUI earlier than the previous OSes, but there is still a lot of shit happening on the system in the background when you first get to the desktop (services starting, background apps loading, etc...). This is what causes the delay in response to clicking on the Start button. To prove this to a friend with Linux, I set X to start up a lot earlier and disabled a few of the non-critical services in the init scripts and compiled a custom kernel. Total APPARENT boot time from "Joe User's" perspective was about 30 seconds. I have to wonder if it would be a hell of a lot faster with this new patch. The thing is that these changes DON'T actually make the system faster at all. It's pretty much the same as before, but the end-user experience is that it APPEARS faster. That seems to be what a lot of people miss in this discussion.
Un-news
What's being described here is called a priority inversion, where a high-priority task makes a request of some service running at a lower priority and gets hung up behind it.
Which is wrong. Did you read the article?
Priority inversion is, as you explained yourself, about a high priority task effectively getting a low priority by being dependend and therefore waiting on a low priority task.
The article is about tasks at the same priority[1]. The task scheduler distinguishes between interactive and non-interactive tasks in order to improve latency where the user cares.
Beforehand, the behaviour failed on a slow, loaded system to recognize the X server as interactive, because then X looks like a CPU-hog[2]. That resulted in freezes of several seconds[3]. Simply speaking, the patch solves this by passing some interactivity points between processes.
You could have easily seen that this is not about priority inversion as one of the suggested work-arounds was to simply increase the niceness of the X process (which wouldn't help, if priority inversion had been the problem).
Regarding QNX: As good as it is as a RTOS, as bad it fails to do something sensible when you have too much processes at the same priority. Having a reasonably working system presumes that each task is assigned an appropriate priority. Of course, the people at QNX did a decent job on the default priorities.
[1] It may have an impact on tasks of different priority. I did not care to investigate that aspect, because that is of minor importance to what the patch is about.
[2] And for the scheduler, interactivity is determined by a process going to sleep often (by waiting for interaction).
[3] For non-interactive processes it is beneficial to do them in larger hunks, i.e. let 5 seconds other processes do their work, then work 5 seconds, instead of having 0.01 second slices and do the switching all the time.
Keep an eye on which arguments are silently dropped in replies. Not always, but often times it's very telling.
Horse hockey. I have.
p df, Microsoft Terminal Svcs is only able to do 8 bits (256 colors). Is that still accurate?
Lower-end stuff (e.g. xterms) run slower over a dialup link (I'm sure I'm not even getting 40-50 kbps here), but it's entirely usable, particularly if I've been using it for a little while. Netscape 4 was lousy. I just tried it. I'm at 16-bit colour, BTW.
Back when I had a cable modem (before I moved to a place where they said I'd have a cable modem by the end of last year. hah!), which was capped at 3Mbps, I ran Mozilla 1 over the cablemodem, over a long distance (they hadn't hooked in to KANREN, so my traffic to the university went from Manhattan, KS through Manhattan, NY and back) from my older Ultra10 workstation (It had, I think, just been upped to 256MB RAM), displaying on my Debian (XFree 4? Or was it still 3?) PII 400 w/ 512 MB RAM, and it ran just fine. I don't recall it being substantially slower than local. I was either running 16bit or 24bit depth. Quite possiby 24bit, since I wasn't trying to run many games then (I made it 16bit for games over winex).
Oh, did I mention that those were over an encrypted connection? (ssh X11 tunneling)
Heck, the university used Sun IPX/IPC with Linux as thin-clients, displaying from a couple of (actually fairly crummy IIRC) central servers. It was pretty usable, too.
Slow at 100Mbit my ass.
And, according to www.ncl.cs.columbia.edu/publications/cucs-022-00.
--
Given enough personal experience, all stereotypes are shallow.
Ugh. I don't buy it.
To put it in perspective, lots of Unix has a big organization problem. X is just emblematic. It's "lower-level" APIs are a big stinking mess. Ever tried to program against it without a super-high-level bit of middleware? Then let's talk about how nice it is. If you're not up on this, try reading JWZ's rants on it (many written as he was porting Netscape)? X is a 4 foot high sandwich of crap, layer after layer between you and the display, full of massive, sucking complexity, the bugs, inefficiency... even during this supposedly wonderful "network transparent" windowing this foul stew shows its colors, as no combination of two applications or X servers quite looks the same. It's a verifiability nightmare, too, of course (and for instance, disabling X's many attempts to listen and talk on the network are one of the first things you do to secure a machine properly - and for real security, you avoid installing X altogether).
The API design itself is atrocious. The much-touted "flexibility" is really code for laziness - it was a lot of work to do a proper GUI, so no one did it. The mishmash of X server extensions, window managers, font handling systems, etc. that's been cobbled together has led to a nightmare for both programers and users, as any given application doesn't just require "X", but a complex recipe of libraries and versions, and an end-user experience where no two applications look or act the same... or even remotely similar... Where cutting and pasting between windows is a pipe dream, and young geniuses still struggle to configure fonts properly for linux distributors.
Or to just put it plainly, as my friend (who from time to time would write X windows gadgets) would say, it's only about twice as hard as managing the video memory yourself.
"And thank god it's not all standardized, or we'd never have had all those wonderful experiments with different ways to do a GUI that never actually happened." In practice, no system is immune from its initial design choices, and it's been an endless series of awful MacOS knockoffs, multi-button madness, color-pallete spinning goofiness. Is X11 a "GUI experimenters toolbench?" Then I think it's time for something a little more grounded in everyday realities of computer use.
I'm not even warmed up yet. I mean, X is still peppering the filesystem with a hedge-maze of exotically formatted text files describing the hex colors of every pixel of the trim of every window for a variety of appliations and classes in a complex inheritance and assignment scheme that few X developers even understand. Check it out, your XDefaults are "human readable."
Shall we even discuss its security model?
Modern Linux has tried to make its peace with X through wrappers, and we write against Tcl/Tk, Qt, inside the Gnome or KDE framework, and yet still the focus groups come back crying... we try to blame overfamiliarity with windows, but the problems are bigger... all of Unix (and of course Linux) suffers from the same class of problems that X does; as, for instance, an application needs to prompt you to insert a series of CD's, but there is no "single, authoritiative, standard" place to go find out what CD drives are installed on the computer, and what their device names are (yes, we know what they _usually_ are), and finding out if any of the CDs are already inserted involves parsing the text output of a proc file or a mount command, and so on and so forth... And all of this is being done by a messy bash script... so it's no surprise this functionatlity is broken even in, for instance, RedHat's own v8 package manager... I hope you can grasp the metaphor.
It's a mess. Patches won't clean it up. Frankly, it's time we took the whole GUI back to the drawing board. But even if MacOS is the end-all/be-all, we can do it a hell of a lot better than we do in X.
Following are some choice quotes from Don Hopkins' essay:
X-Windows is the Iran-Contra of graphical user interfaces: a tragedy of political compromises, entangled alliances, marketing hype, and just plain greed. X-Windows is to memory as Ronald Reagan was to money. Years of "Voodoo Ergonomics" have resulted in an unprecedented memory deficit of gargantuan proportions. Divisive dependencies, distributed deadlocks, and partisan protocols have tightened gridlocks, aggravated race conditions, and promulgated double standards.
X has had its share of $5,000 toilet seats -- like Sun's Open Look clock tool, which gobbles up 1.4 megabytes of real memory! If you sacrificed all the RAM from 22 Commodore 64s to clock tool, it still wouldn't have enough to tell you the time. Even the vanilla X11R4 "xclock" utility consumed 656K to run. And X's memory usage is increasing.
...
X was designed to run three programs: xterm, xload, and xclock. (The idea of a window manager was added as an afterthought, and it shows.) For the first few years of its development at MIT, these were, in fact, the only programs that ran under the window system. Notice that none of these program have any semblance of a graphical user interface (except xclock), only one of these programs implements anything in the way of cut-and-paste (and then, only a single data type is supported), and none of them requires a particularly sophisticated approach to color management. Is it any wonder, then, that these are all areas in which modern X falls down?
...
As a result, one of the most amazing pieces of literature to come out of the X Consortium is the "Inter Client Communication Conventions Manual," more fondly known as the "ICCCM", "Ice Cubed," or "I39L" (short for "I, 39 letters, L"). It describes protocols that X clients ust use to communicate with each other via the X server, including diverse topics like window management, selections, keyboard and colormap focus, and session management. In short, it tries to cover everything the X designers forgot and tries to fix everything they got wrong. But it was too late -- by the time ICCCM was published, people were already writing window managers and toolkits, so each new version of the ICCCM was forced to bend over backwards to be backward compatible with the mistakes of the past.
The ICCCM is unbelievably dense, it must be followed to the last letter, and it still doesn't work. ICCCM compliance is one of the most complex ordeals of implementing X toolkits, window managers, and even simple applications. It's so difficult, that many of the benefits just aren't worth the hassle of compliance. And when one program doesn't comply, it screws up other programs. This is the reason cut-and-paste never works properly with X (unless you are cutting and pasting straight ASCII text), drag-and-drop locks up the system, colormaps flash wildly and are never installed at the right time, keyboard focus lags behind the cursor, keys go to the wrong window, and deleting a popup window can quit the whole application. If you want to write an interoperable ICCCM compliant application, you have to crossbar test it with every other application, and with all possible window managers, and then plead with the vendors to fix their problems in the next release.
In summary, ICCCM is a technological disaster: a toxic waste dump of broken protocols, backward compatibility nightmares, complex nonsolutions to obsolete nonproblems, a twisted mass of scabs and scar tissue intended to cover up the moral and intellectual depravity of the industry's standard naked emperor.
Using these toolkits is like trying to make a bookshelf out of mashed potatoes.
- Jamie Zawinski
...
The fundamental problem with X's notion of client/server is that the proper division of labor between the client and the server can only be decided on an application-by-application basis. Some applications (like a flight simulator) require that all mouse movement be sent to the application. Others need only mouse clicks. Still others need a sophisticated combination of the two, depending on the program's state or the region of the screen where the mouse happens to be. Some programs need to update meters or widgets on the screen every second. Other programs just want to display clocks; the server could just as well do the updating, provided that there was some way to tell it to do so.
...
.Xdefaults", but if he happens to have copied Fred's .xsession, he does an xrdb .xresources, so .Xdefaults never gets read. Susie either doesn't xrdb, or was told by someone once to xrdb .Xdefaults. She wonders why when she edits .Xdefaults, the changes don't happen until she 'logs out', since she never reran xrdb to reload the resources. Oh, and when she uses the NCD from home, things act `different', and she doesn't know why. "It's just different sometimes."
.xresources (or was it a file that was #included in .xresources) of the form *fucked*fontList: 10x22, which he copied from Steve who quit last year, and that of course that resources is 'more specific' than his, whatever the fuck that means, so it takes precedence. Sorry, guy. He can't even remember what application that resource was supposed to change anymore. Too bad.
What this means is that the smarter-than-the-average-bear user who actually managed to figure out that
snot.fucked.stupid.widget.fontList: micro
is the resource to change the font in his snot application, could be unable to figure out where to put it. Suzie sitting in the next cubicle will tell him, "just put it in your
Joe Smartass has figured out that XAPPLRESDIR is the way to go, as it allows him to have separate files for each application. But he doesn't know what the class name for this thing is. He knows his copy of the executable is called snot, but when he adds a file Snot or XSnot or Xsnot, nothing happens. He has a man page which forgot to mention the application class name, and always describes resources starting with '*', which is no help. He asks Gardner, who fires up emacs on the executable, and searches for (case insensitve) snot, and finds a few SNot strings, and suggests that. It works, hooray. He figures he can even use SNot*fontList: micro to change all the fonts in the application, but finds that a few widgets don't get that font for some reason. Someone points out that he has a line in his
...
On the whole, X extensions are a failure. The notable exception that proves the rule is the Shaped Window extension, which was specifically designed to implement round clocks and eyeballs. But most application writers just don't bother using proprietarty extensions like Display PostScript, because X terminals and MIT servers don't support them. Many find it too much of a hassle to use more ubiquitous extensions like shared memory, double buffering, or splines: they still don't work in many cases, so you have to be prepared to do without them. If you really don't need the extension, then why complicate your code with the special cases? And most applications that do use extensions just assume they're supported and bomb if they're not.
Want to Know How to Cheat the GPL? Read On!
If anything, Linux is becoming the next MS
Now, *that*, I have to say, is bordering on the flamebaitish -- yes, I see what you're trying to say, but that's a kind of offensive way to phrase it.
Uh...check out Windows 2000 scheduling algos.
It's not exactly the same thing, though even Linus mentioned it (rather offensively, IMHO, to Ingo). Windows has a simple heavy priority boost it gives to the foreground app. That works fine if you're working in a fairly modal manner on a single-user system and you have a desktop-with-foreground-and-background paradigm as a fundamental part of the OS.
Linux's scheduler takes a somewhat more ambitious (granted, that probably means you can trick it more nastily) approach, partly because it has a more general, more difficult task. From what I can tell from skimming the conversation, Ingo's work is something more along the lines of advancing the traditional UNIX approach of "this app didn't use (or is tending not to use) its full timeslice, maybe because it's blocking on I/O, so give it higher priority to get another timeslice than an app that *did* use its full timeslice". He's just doing somewhat more sophisticated automatic classification of whether an app is "interactive" or not.
Yes, on the very surface, it's similar in goal. Make the task that the user is working with get more cycles at appropriate times to reduce latency of interface response. However, the approach is very much different, and the potential benefits are higher (since this automatically addresses a wide range of apps, not just making the foreground app peppier to keep scrolling snappy).
I *will* give you that this has little to do with open source. I suspect that there are plenty of closed-source systems that have tried to do more advanced classification of apps as interactive or noninteractive.
May we never see th