Slashdot Mirror
The ~200 Line Linux Kernel Patch That Does Wonders
An anonymous reader writes "There is a relatively miniscule patch to the Linux kernel scheduler being queued up for Linux 2.6.38 that is proving to have dramatic results for those multi-tasking on the desktop. Phoronix is reporting the ~200 line Linux kernel patch that does wonders with before and after videos demonstrating the much-improved responsiveness and interactivity of the Linux desktop. While compiling the Linux kernel with 64 parallel jobs, 1080p video playback was still smooth, windows could be moved fluidly, and there was not nearly as much of a slowdown compared to when this patch was applied. Linus Torvalds has shared his thoughts on this patch: So I think this is firmly one of those 'real improvement' patches. Good job. Group scheduling goes from 'useful for some specific server loads' to 'that's a killer feature.'"
I guess it's somewhat objective and dependent on your hardware anyway but even with lots of programs open on my main machine I don't notice any slowdown... I think if linux is really gonna pull ahead of the pack they aught to take a gamble on a new, useful interface. Something like 10gui. The risk could even be mitigated by having a choice to load either type of desktop at the beginning with a quick video to demonstrate the difference. :)
check out the Mp3 Garbler I built!
I thought a few years ago, there was a desktop friendly scheduler rejected because Linus thought the server environment was more important. The details escape me.
Compiling the kernel doesn't prove true userspace improvements, but it does show an improvement with scheduling.
I see. It creates "groups" based on the tty and then tries to even out the CPU utilization between groups. This helps if there is a crazy background process eating up CPU and it might even help control flash crushing system performance a bit.
MidnightBSD: The BSD for Everyone
The patch being talked about is designed to automatically create task groups per TTY in an effort to improve the desktop interactivity under system strain.
I guess this works because the task loading up the machine will have been launched from a konsole, and thus be tied to a specific tty (the make -j64 example given later), so a tty-based scheduler can appropriately downgrade it.
But what if the "loading" task has been launched directly from the GUI (such as dvdstyler)? It won't have a tty attached to it, and will be indistinguishable from all the other tty-less tasks launched from the GUI (such as the firefox where you browse your webmail), or worse, Firefox itself creating the load (such as happens when you've got too many Facebook or Slashdot tabs open at once...)
This has been brought up by others on slashdot before, but Linux tends to be either (A) fine and happy, or (B) pushed into a thrashing state from which it can never recover - like it takes 8 or 10 minutes to move the mouse cursor across the screen. Since there is relatively no warning before this happens, it makes a hard reboot just about the only option.
Will this patch help with that issue? Like the threads below say, once a modern (KDE/Gnome) desktop Linux starts swapping, there is so much new data produced per second by the GUI that it's basically game over. I'd like to see a fix for this: it's the single biggest cause on Linux that makes me do a hard reboot. I just don't have the patients to see if the thing will recover in half an hour or so, even though it might.
http://ask.slashdot.org/comments.pl?sid=1836202&cid=33998198
http://ask.slashdot.org/comments.pl?sid=1836202&cid=33999108
http://www.linuxquestions.org/questions/linux-kernel-70/swap-thrashing-can-nothing-be-done-612945/
It creates "groups" based on the tty and then tries to even out the CPU utilization between groups.
Works fine if there are groups than can be distinguished by tty (such as the kernel compilation which has one, versus most interactive apps, which don't, or have a different one).
But it won't necessarily work if the task creating the load is also a ttyless GUI task (such as programs such as dvdstyler)
As strange as it may seem, I get that feeling when I touch a *used* Windows machine. No matter who used said machine, it eventually starts slowing down to a crawl. When I came to Linux, I came because I wanted a programmer-friendly environment (The Way It's Meant To Be) and I liked Compiz-Fusion (go figure!). Now I run GNU/Linux because it is GNU/LInux -- Free and Open-Source --, with openbox, fbpanel, pcmanfm, gnome-terminal, evince, chromium-browser, LibreOffice, amarok1.4, gedit, vim and some games (some via Wine). By leaving gnome itself, I learned what speed was really all about. However, I can't help noticing that my laptop gets quite a usability hit when I start eating of its CPU(s) -- but when I compare that with the Windows machines under a heavy load, I see that my poor 500€ laptop can top the asses out of those 1500€ lean-mean-Windows-machines. So, I may be an exception, but Linux, for me, in spite of some notorious disadvantages, can still top Windows in terms of these load-related tasks.
Have you heard about SoylentNews?
With make -j64, this is ok: The make will have been started from a terminal (it's not important whether it's a Linux (console) terminal or within an X window), whereas most other GUI programs won't. So they can be distinguished from each other.
But in case of dvdstyler, it won't have a tty if it has been started directly through the menus, and so the scheduler cannot put it in a different group than all the other GUI apps (which also lack a tty).
Of course, if you start dvdstyler from the command line (from a konsole), this won't apply.
You can easily check which tty (if any) a program has associated by doing a ps auxww and looking at the 7th column (where it will say ? for "no tty" or name the tty (such as pts/9)
Con Kolivas worked on his staircase scheduler and various performance patches for years. They were routinely demonstrated to be a major improvement. Linus kept saying he was concerned the tradeoff of desktop performance would come in other environments, even though that wasn't true.
Since benchmark after benchmark showed staircase was vastly superior to what was in mainline, Linus then went to go after the guy rather than the code. He said Kolivas couldn't be trusted to support his code, and thusly it would never be accepted mainline. Reality was that Kolivas had been responding to criticism and updating his patchset for over 3 years, constantly improving it. In addition to the LKML, he also maintained his own support mailing list.
I'm a Linus fan 95% of the time, but it was a really shitty move, and it drove Kolivas away from contributing. He quit coding for a while. Then after Linus argued for years how this was a bad idea, suddenly the mainline kernel developed the Completely Fair Scheduler overnight, which was very similar to Kolivas' Staircase scheduler. Linus never admitted he had been a dick for years arguing against the thery of the scheduler rewrite. He then pushed the brand new, untested scheduler into mainline.
CFS is better than what we had before, but it still lost in benchmarks to Staircase, and it was new, untested code.
Now, Kolivas came out of retirement with a new scheduler called Brainfuck that is even faster, but he has no intention of ever tryint to get it in the mainline kernel.
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
Nice to see somebody thinks I'm a troll.
You know, I'm as pro-open source as the next guy here, but I try not to be dogmatic about it. I've played with a bunch of Linuxes from the first half of the 1990s onward and even used it professionally for a time. While there are many things I like about the OS, it does have it's faults. Like not being as user friendly as Win. Like not being as snappy as Win.
I'm really sorry if I hurt somebody's precious feelings, but a troll I certianly am not. And like I said, I'm very glad this patch is coming out and will continue to run Linux as my secondary OS.
Turn off swap, if you can. The cost of memory is now less than the cost of the stress and lost uptime due to swap-paralysis.
I have 4G of RAM on my desktop (I doubled the RAM for $60 after I bought it) and the only time my system swaps is when I have mmap()'ed an 8G file.
Similarly, on my 512M netbook, I don't exceed RAM with crazy things like "make -j64 bzImage". Even with wear-leveling, swapping to SSD is bad form. I'd rather swap over NFS to spinning platters, than to SSD.
So is Linus a dick?
Nope, you're not a troll. I have the exact same complaint about Linux desktops (and OSX): Windows just seems slightly but noticeably snappier for keyboard and mouse events. I just thought that someone at MS realized that this was an important area to get right, while Linux and OSX struggled against a cap dictated by other design decisions (such as Apple's Quartz library being strongly based in NextStep's display PostScript origins).
It was never a huge deal to me because so many other considerations are also important, but it's always noticeable.
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
Actually Linus lost track of many such things because too self centered or ego driven (which happens to most of us when you such success and things to deal with but anyways)
This very patch is a prime example, if it goes *default* in the kernel.
It's a patch that favors *only* Linus's usage of linux: Browse the web while his kernel compiles. Now imagine you start your video from a tty (mplayer blah_1080p.avi) and it takes 95% cpu to be smooth, then you start your browser.. uho.
In BeOS I could^H^H^H *can* (haiku, too) start 5 videos on and old PC, browse the web and:
- the browser is smooth like butter
- the 5 videos are a bit choppy (no miracles but that the point) but they all run at the same speed
Now _that_ is what I want a desktop scheduler to be like.
With more criticism he could see that some would want this auto grouping option, but the majority wouldn't. Now what does that tell us?
It tell us that it's _either_:
A/ Not the best solution (i.e. our scheduler sux)
B/ Grouping should be smarter or more easily configurable (it's currently configurable in the previous kernel version and can do just what this kernel patch does!)
Kernel benchmarks are always a point of contention. But interbench and all the standard benchmarks did show a marked improvement with the CK patchset and the Staircase scheduler.
If Linus didn't really believe it was an improvement, then why did he eventually call on Ingo to write a very similar scheduler?
Linus didn't want to admit he was a jerk who made a flippant personal decision rather than focusing on the best code.
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
Seriously? I thought you were joking but you got modded Insightful instead of Funny. All day at work (the only time I'm subjected to Windows) I sit around wondering why Windows takes 93 seconds (literally, not figuratively -- I counted) to open directories with a few thousand files in them, whereas Linux takes something on the order of .01 seconds. I have never, not even once, thought of Windows as "snappy", especially when compared to Linux. But, I don't want to argue about it, I don't want to start a flamewar, and I don't doubt the sincerity of your statement; I merely have had a very, very different experience.
Not the mod, but I can see where he's coming from. I can maybe see his claim having merit if he's only tried the heaviest GNOME or KDE distros, but "no matter how many distros he's tried?" That comes very close to trolling, considering there is at least one fairly well-known distro that loads itself entirely into RAM.
Of course it is difficult to argue against anecdotal evidence, especially if you don't know the hardware situation. Personally I'm fairly positive my nicely-tuned Arch install runs circles around my fully-patched Vista install, but that's just my experience.
I agree with you to a point but I wouldn't agree at all that just any gui or none at all would work for someone who only uses three or four programs at a time. You still need to efficiently switch between the apps, or view them at the same time, not to mention all the other types of data that is persistent like time, weather background apps, notifications, etc. I'm certainly not intending to flame you but I'd completely disagree with you that the linux desktop is only good for a browser or screen. I'd argue that it's much better than XP. ( I can't say about win7 ) What specifically about windows desktop is making your experience much better than the Ubuntu desktop? Have you tried the Ubuntu desktop lately?
check out the Mp3 Garbler I built!
All the little sister chips could do most of the work. Which is what killed the Amiga. It was a cluster of ASIC chips, and that doesn't scale well to incremental performance upgrades the way a new, faster x86 chip every six months does. Who's going to design a whole new set of ASICs every six months? Nobody in their right mind.
Whatever happened to "nice-ing" processes? Couldn't the menu icon that launches VLC or mplayer or whatever re-nice itself a few negative steps?
I remember, about 8-10 years ago, how this wasn't the case. It was quite evidently better than Windows in this regard, particularly if you didn't upgrade your hardware on a 2-year cycle (eg. waiting until it died) or tried running on significantly older hardware. Performance, on the desktop, was great. 2.6 seems to have progressively nixed it.
The first time I tried compiling a kernel, I was astounded at how I was still able to play fullscreen 600Mb encoded DVDs (can't remember what they were encoded in, but the quality was decent).
I remember building a kernel in '99-2000 or so on a P133 with 64Mb of RAM (running Stormix). Netscape was still responsive. Switching to a different application did't really take all that long.
These days, Linux performance on the desktop, this regard, is worse than Windows. It's a fucking travesty. Using the anticipatory scheduler helps significantly (or did, until they removed it from the kernel), but it was hardly much more than a stopgap measure.
I am pleased as fucking punch that this is finally 'fixed'. Like, I'm giddy to the point where I doubt I'm going to get any work done today.
Where can I download prebuilt kernels for my distro of choice? Surely someone is building them.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
Yeah, I never really figured it out. These are local directories having, oh, maybe three or four thousand files. It is the first time the directories have been opened since the files were written. But still, what the heck is taking 93 seconds? When I re-open the directory, it only takes a short amount of time, so I figure Windows is creating some kind of cache that first time. Still, it's an inexcusable user experience. But, it's an ongoing problem that I have every day or two, because I deal with a lot of these directories with lots of files written to them.
I know Linux wants to be POSIX compliant, but I don't see any harm in changing nice like so:
1. Store the original nice value of a process in the kernel's process table.
2. Allow priority to be lowered by anybody.
3. Allow priority to be raised by anybody as long as it does not surpass the original nice level.
4. Require superuser to go beyond the original nice level (and reset the saved value if they do).
That would be no more dangerous than the current system, but would allow apps to lower their priority temporarily when they go off to perform 'batch-like' processes, restoring it when they return to interactive mode.
Posted from my Android phone. Oh, I can change this? There, that's better...
I probably haven't tried nearly as many flavors of Linux or Windows as you, but my experience has been the complete opposite. My Ubuntu install, running Gnome on a pretty average system (3.1ghz dual core AMD CPU, 4GB RAM, integrated video), feels lightning-fast and responsive compared to pretty much every other system I've ran so far. Especially the Windows systems I use at work, which admittedly are all XP systems infected with McAfee. I've always been kind of a junkie for really responsive UIs - it drives me nuts when Windows and IE7 always take half a second here and a second there to respond to every little command. My Linux system virtually never does this, which is why I'm happy to keep using it. And of course anything that makes it better yet is more than welcome.
I don't reply to ACs
I've been a Windows user since Windows version 1.01 (that used real mode DOS, and text based graphics). Windows has never, ever, ever been "snappy". At least not out of the box. There used to be third-party utilities (eg Norton Desktop), that made the experience better. But, it does depend on what you use it for. If you don't have tens of thousands of files to maintain, and only use it to watch pr0n and for email, and that Christmas letter. Then yes, you will probably have snappy performance.
If however you use it for business, and write dozen of documents a day and have millions of lines of code in thousands of files, and are creating videos and music, then you will not see snappy Windows performance. That said, unless you've been using the -ck kernel patch (I downloaded and stored on my own server when CK left in storm), you've not seen snappy performance in any full blown WM in Linux either. Now, it looks like they finally got a kernel patch that can compete with Con's 6 year old patch. Congratulations Linus! It only took you and your crew six years to duplicate the work of one man that you bullied out of the kernel group!
Running Linux since 1991! Wow it's almost been 20 years since I attained enlightenment!
Or he is less willing to solve problems than you are. Ever thought of that?
Look, you want easy entertainment, you get a DVD player and a TV. You want to get a bit beyond that, you get a gaming console. You want to run with the Internet, you better be willing to learn how to use a PC.
It never ceases to amaze me how smart my parents can be when I'm not available, and how dumb they can degenerate to when I am. It's not even instant Alzheimers, it's like they suddenly can't even read. Could all that just possibly be... laziness?
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
Wondered this myself -- a very strange bug, since playing MP3s has been a solved since the days of (IIRC) 100Mhz Pentiums :)
I figured there was some option to prebuffer that I've never had the time to google for ... but I look forward lazily to the day when it's fixed ;)
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
Even a somewhat power-user can do this. A distro can do it trivially.
1. Menu item can run as root by default, then assume the identity of the logged in user. Might need to play with sudoers file a bit, possibly remove "requiretty" option.
2. Use ionice if you are worried about I/O and running linux > 2.6.13.
Bingo Dictionary - Pragmatist, n. A myopic idealist.
This is the great battle with distributions like Ubuntu. There's a fine line between auto-configuring and helping the user make decisions (or making decisions for the user) and preventing the user from making decisions on their own.
Yet Another Tech Blog
(but so much more, including game and movie reviews)
http://yanteb.peasantoid.org
In 1980, The IBM mainframe operating systems included "Performance Groups". Workloads were classified into Low CPU interactive and High CPU background and also some must complete performance group(s). It like being born again to see it coming to Linux as a new patch. Now I would love to see a "VM CMS" environment under the linux kernel
Leslie Satenstein Montreal Quebec Canada
But somewhere around the time of Windows 95, things seemed to change and it became expected that the "average user" had no interest or time in learning how anything works.
It was a combination of Windows 95 being comparatively idiot-friendly along with an absolutely massive influx of idiots when public access to the Internet started to take off, and the web began to come into its own.
Those two things happening at about the same time changed the world in a lot of ways. For example, the terms "top posting" and "bottom posting" came into existence right around this time, as millions of new users exposed to email for the first time got their hands on Outlook Express, with its completely broken editor that made formatting a message correctly almost physically impossible.
When I really think about it, the biggest difference between being an old-school DOS hacker and being a modern Linux hacker was that in the DOS days we had BBSes, and those of us with net access had Gopher and Archie. I got out of development around 1995 not simply because everyone demanded GUI applications, and early Windows development sucked, but also because when I first got on the web, I found a good many alternatives to every application I had ever developed. You think KDE vs. GNOME is a big waste of resources? One of the DOS applications I actually made money on was a utility that has a close equivalent as part of the GNU toolchain, and I abandoned development when I realized there were about six alternatives that predated my own, and were further along.
We DOS hackers were working alone for the most part, or in small, isolated teams. The web changed all that, and then sites like SourceForge took it all to the next level. Now, collaboration is easy.