Slashdot Mirror
Torvalds Explains Scheduler Decision
Firedog writes "There's been a lot of recent debate over why Linus Torvalds chose the new CFS process scheduler written by Ingo Molnar over the SD process scheduler written by Con Kolivas, ranging from discussing the quality of the code to favoritism and outright conspiracy theories. KernelTrap is now reporting Linus Torvalds' official stance as to why he chose the code that he did. 'People who think SD was "perfect" were simply ignoring reality,' Linus is quoted as saying. He goes on to explain that he selected the Completely Fair Scheduler because it had a maintainer who has proven himself willing and able to address problems as they are discovered. In the end, the relevance to normal Linux users is twofold: one is the question as to whether or not the Linux development model is working, and the other is the question as to whether the recently released 2.6.23 kernel will deliver an improved desktop experience."
Everyone except you.
Obligatory blog plug: http://www.caseybanner.ca/
Hasn't Linus already said he's very interested in adding it into the Linux kernel (going so far as to say that it could be a reason to consider going GPLv3 in the future if Sun releases Solaris under it), but right now it's tied up in closed source?
As it seems many others don't agree to his opinions of Con, http://lkml.org/lkml/2007/7/27/426
What do CFS and SD stand for in this case? The summary and linked articles do not describe this.
Having had my fair share disagreements with customers over technical issues, it just isn't worth trying to "win." The damage to your working relationships is still there even if you are shown to be 100% right. Try and help them address their problems as much as you possibly can, while trying to compromise as little as possible of the design. It's called diplomacy, and it's the difference between being given huge amounts of responsibility, and wanting to quit. You don't even have to agree with them, you just have to make them think you care.
Finally, it is common for programmers to try to avoid a subset of the problems in an area because it gives them the ability to write something "correct." Certainly a very satisfying experience for a programmer. However, that is exactly why it can be a bad idea to let a programmer rewrite a messy module. Very soon you can find the users of that module asking why a laundry list of things don't work anymore and an idealist developer trying to argue that they shouldn't... And it is exactly those idealists that like to rewrite working code. Not that major rewrites are bad, just that they have to be approached by someone mature enough to both expect a list of things they overlooked, and be willing to work with customers to resolve them.
-- http://thegirlorthecar.com funny dating game for guys
It's not closed source. It's available as part of OpenSolaris (CDDL). FreeBSD didn't have a problem integrating it.
Do you even lift?
These aren't the 'roids you're looking for.
The only important points to me are:
1) Games are basic to linux desktops, we need linux games, so if Con scheduler was best on games, please, incorporate those changes.
2) EGO's are a pain in the ass, it seems that Con has been refused by Linus , because he didn't take the whole history into acount, too bad, that kind of things happend in any not trivial project management. Don't let EGO's rule the ball, that's the most basic point a manager (Linus) has to respect to keep people on the project.
What's in a sig?
Actually, I have no idea how an OS works and even _I_ know what this article is about.
I'm sorry but the flame wars over whether you call it zee eff ess vs. zed eff ess would make the vi vs. emacs flamewar look like a watergun fight. It would likely tear the Linux project apart as radical Zeddites suicide bomb the Zee Alliance.
It's about Completely fair scheduler and staircase-deadline cpu scheduler and problems with their developers. If you don't know what a cpu scheduler is, you are not slashdot's target audience, "Tere is nothing for you to see here, please move along".
Extreme Programming - Redundant Array of Inexpensive Developers
The guy walked away. It's like quitting the Internet. Obviously if your reaction is to take your ball and go home (and I know, the ball is everybody's in this situation, but go with it) then you aren't mature enough to handle the responsibility Linus requires.
Slashdot - where whining about luck is the new way to make the world you want.
The real "Libtards" are the Libertarians!
Is CDDL compatible with GPLv2? I may have mispoke in my first post, but I think the gist of what I was trying to say is still there. ZFS is behind a license which isn't compatible with the license Linux is under. Or am I completely misunderstanding things? I'll admit I'm far from an expert on these matters...
Look at the front page much?
Do you even lift?
These aren't the 'roids you're looking for.
which kernel scheduler is pretty low on the list of factors affecting what the Linux desktop experience is all about...
frankly, really high quality experiences take organizational planning and leveraging the expereince of huge groups in way that the "bazzaar" model of software developemnt in open source does not do well. Would someone please just build a mutual benefit corporation for open source users and maintainers? Let's start paying for project managers and the other experienced professionals required to make a "desktop experience" and you will see Linux take over.
The official kernel site says 2.6.23 is only on release candidate 1.
Having lurked on http://www.lkml.org/ for several years, I find Linus to be rather rude. May be it's because English is not his first language...so words are not well chosen. I must say though, that I excuse him because he produces, [or helps produce] a very useful product on the world today. That is the Linux kernel.
I find Linus' comments very interesting, and very honest. He has good arguments, and to me it seems like he did the right thing. He is even open to change scheduler if someone can prove that SD does a better job than CFS, and get someone reliable to maintain it.
It already is an option in recent kernels to change to the completely fair scheduler. In fact, I'm using it right now, and it seems better at some things, and worse at other things, which is to be expected.
I suffer from attention surplus disorder.
Oh, I'm sorry. Please go under your preferences and change them to show apple stories only.
It's an interesting set of emails. In addition to admitting that he actually didn't have any problem with the SD code, Linus points out that he made a gut call that Con is difficult to deal with without even looking into it because, apparently on near religious grounds, he doesn't believe in reading "specialized" mailing lists. What i find of concern is that he'd express such strong opinions about people basically without having even spent an hour or two browsing some list archives. Further, he seems perfectly aware that he may have heard just one side of the story, and yet he STILL doesn't feel he needs to look into it further or to soften his view? WTF ?
... arbitrary ?
Has Linux kernel development always been this
From TFA (actually form the quoted emails) after several mails where Linus has been bashing this Con Kolivas guy for not taking feedback and being argumentative, and then offers some statements about the virtues of a good maintainer some guy "Kasper Sandberg" asks him:
"Okay, i wasn't going to ask, but i'll do it anyway, did you even read the
threads about SD?"
to which Linus responds:
"I don't _ever_ go on specialty mailing lists. I don't read -mm, and I
don't read the -fs mailing lists. I don't think they are interesting.
And I tried to explain why: people who concentrate on one thing tend to
become this self-selecting group that never looks at anything else, and
then rejects outside input from people who hadn't become part of the 'mind
meld'.
That's what I think I saw - I saw the reactions from where external people
were talking and cc'ing me.
And yes, it's quite possible that I also got a very one-sided picture of
it. I'm not disputing that.
"
You can be pretty damn right that the desktop experience is improved with Ingo Molnar's scheduler! If you have done any serious audio work on any platform you know that Linux kernel + Ingo Molnar's IO scheduler = the best platform for serious audio work. This combination has the lowest latencies. Linux kernel+Ingo Molnar's IO scheduler+Ardour offers currently lowest latencies and the best of all - it's completely free! It is pretty amazing - really. Every true professional audio engineer will agree with me.
That's not really fair. Slashdot pretty much covers all things science, be it computer science, or quantum physics. There are frequently articles posted which I, being in the field of computer science, do not necessarily understand or comprehend. I'm not going to tell all of the quantum physicists out there that "slashdot is not the place for them" because Slashdot IS for them. While it might have a technical slant to it, I think it's a place for anyone more interested in hearing news of advancements in various scientific fields, rather than what prison Paris Hilton is going to next.
For that matter, even when I don't understand what an article is talking about, I am still usually more than interested to read about it.
Whatever way you want to paint it, that's exactly what he did.
He believes that Ingo is a more reliable maintainer, so he chose Ingo's few-day old hack instead of Con's very mature and well tested scheduler.
Personally, I think that the person who is at fault here is Ingo, because he has a "Not Invented Here / By Me" mentality, and instead of developing Con's scheduler further, he totally objected to Con's work for ages (which prevented it from getting into mainline), and then suddenly saw the light and wrote his own quick hack based on the same design.
Ingo may be a good developer and maintainer, but he sure as hell isn't a friendly co-developer.
the Linux desktop responsiveness definitely improved (in my subjective experience) in the past two years or so.
If that was Con's scheduler, then thanks.
At this point I blame GNOME over the kernel for any remaining sluggishness, but then maybe my 1.2GHz athlon is just obso1ee7.
(Disclaimer - I have never met Linus or heard him speak on anything )
After reading over the article, I came away with the impression that Linus appears to be an angry person. I have also noticed this in past topics where emails from Linus have been mentioned. So I am asking the question as to why he appears this way. Is it because I always see such references at particular time like now with hot button issues? Or perhaps because English is not his native tongue? Or is it because he really does have an anger management issue?
Based on his own comments in those emails (and the standards that he seems to hold everyone to) it seems to me that Linus would not work with himself as he himself appears very pig-headed and stubborn.
Anyone care to explain?
I am Slashdot. Are you Slashdot as well?
Why can't there be a flag that determines what scheduler is used at runtime, with both schedulers built into the kernel? I thought the whole point of Linux is that it is customizable and modular - I know this doesn't necessarily apply to the kernel, but why not?
;)
I know very little about operating systems, schedulers, and maintaining large projects, so please excuse any ignorance in my post
Come on, it's quite obvious that Linus's secret superhero alter-ego has done battle with Kolivas' supervillain identity before (I mean, Con Kolivas? Do the writers even try to make these things sound authentic?) and is now trying to thwart his evil plans of global domination.
... I find a bit of hypocrisy.
.... consideration about inclusion of code they wrote.. Uhhh did I forget to say its "Open Source"?
Three is right, as its Con and an email exchange between Linus and another.
Whooo hooo..
That settles it.... everybody is accounted for..... right??
Its open source but with all the talk about having a maintainer of certain character as a part of the consideration of
Its not uncommon for pioneers to be forgotten as what comes next, takes over...
Or does this mean that when a maintainer dies, so does what they were maintaining?
The general message Con seemed to be expressing was more interesting as a general observation than of specific code.
The response from Linus suggest that although Linus does not frequent specific topic lists because of inherent bias, he has his bias none the less.
There is a general across the board bias, proprietary and open source, and it is one of exclusion of the end users.
And it comes across as arrogance motivated by money and/or ego.
To explain, programming is an act that includes creating functionality that is then accessible thru an easier to use interface such as a function call with arguements and expected return value. The general concept is common knowledge in coding regardless of what programming language you are using,
However, this concept is not typically provided to the end user, but instead kept away from the user and certainly not provided to the user, when some distortion of it is provided the user, in any sort of easy common consistent manner.
To clarify, users access programs typically via a command line or GUI. Neither of which are so conducive to allowing teh users to put things together for themselves. All the functionality if available to the user via the programs GUI or command line. But the same functionality is not available in an mode that allows user to call the functionality in the program and make use of the results outside of teh programs command line or GUI interface.
Con mentioned the Amiga. The Amiga had all three user interfaces. The command Line, The GUI and the missing user interface an every other system today, The IPC port interface, most commonly known as the AREXX port but did not need AREXX running in order to use the port for "user putting things together for themselves".
SO YOU DON'T LET USER PUT THINGS TOGETHER FOR THEMSELVES! WHY NOT?
Cause you can dumb down the user and get ideas from them and sell those ideas implemented, back to them.
But when you take away from the users, the ability to put things together for themselves, then that makes you a hypocrite when you then call them ignorant, armchair coders or any other demeaning term. As it is you who have created that self supported dependency of trying to justify your lack of inclusion of others.
Con outright stated how he got started.
Maybe You linus and a whole world of other coders, need to pull you head out of your asses and SERIOUSLY realize, THERE ARE OTHERS you are not considering.
Ultimately, if people want to optimize their system for their needs, they should be able to. But there is serious prevention of that across the board.
Is the Linux development model more like Surf's Up or Happy Feet?
Have you improved it to the point that when the system is borderline out of main memory or has a moderately high load average it actually *works* as a desktop system?
E.g. when Firefox is consuming 65-70% of main memory and slower than #%#$ and you know it is waiting on swapped out pages and your swap rate is measured in the dozens to hundreds rather than hundreds to thousands (on vmstat)? (I mean really, how can one take an operating system seriously when only memory is at 100% and not CPU + memory + Disk I/O?)
The real issue, for those who have read comments that Con has made in interviews, seems to be the lack of concern on the part of most of the "in-crowd" Linux developers for performance on the desktop. In part this seems driven by the fact that the people who actually get paid to maintain Linux, benchmark it and "improve" it only care about its performance in server farms and *not* on the individual desktop. I will weigh in on the side of desktop user out there (that wants the Linux sitting beneath their desk to devote its every waking minute to making *them* happy) by saying that if my mplayer "hangs" in the middle of a song (only to continue with a loud burst of noise 10 seconds later) when the CPU is busy with "nice -19" processes, my Firefox browser takes half a minute to scroll a page or open a screen) when memory is tight, and it takes minutes to bring up a tab or minimized program I haven't touched in 3 days and return them to a functional state then the operating system *Has a PROBLEM*.
Con was very clear in his interviews that the problem is the lack of caring about *desktop* performance. Given my comments in the previous paragraph -- some of these areas may be very difficult to benchmark and as a result one is left with nothing but handwaving and loud voices when it comes to discussions about whether the problem exists and how it should be fixed.
I will say this, in the mid-'90s I used X-windows under Unixware on *Pentium 1s* as a desktop machine. I now use X-windows under Linux on a Pentium 4 (with 5-10x more main memory) as a desktop machine. I would argue that my desktop user experience is as problematic now as it was then *despite* the hardware improvements. That IMO is what Con felt was the problem he was trying to address. That is what it would appear the core Linux developers may be failing to address. Con's points raised my awareness level to the extent that I actually went investigating to see whether there were open source distributions of the BeOS and/or Darwin (which is based on Mach) available since they are based on different OS architecture models and might be more end-user friendly [1]. I was hoping to find something I could run in a VM under Linux on my current hardware without major file system surgery. But I have little confidence that such an approach would fix core problems with the Linux scheduling and paging systems. I would *love* to see a real side-by-side comparison of Linux vs. FreeBSD for desktop users with an emphasis on how BSD scheduling, paging and swapping may be different (better?).
(And as a side note, I could care *zero* about the performance of Linux in file server applications!)
1. I did use both Nextstep (on Pentiums) and IRIX (on a R4000) for a while and found both to provide better end-user experience than Unixware (X) or Windows (95-98). I am disappointed that Linux barely manages to match those experiences given the hardware available nearly a decade later.
I am not a kernel developer and do not follow the mailing list. I was under the impression that a new scheduler that was supposed to drastically improve performance went into the kernel a couple years ago? I know there has been a huge difference in desktop performance from that time.
If all people would react as parent did here, I would indeed not think of myself as slashdot audience. However, luckily there are a bunch of slashdot users that understand that slashdot has a mix of topics, and a mixed audience. Someone that knows about the topic at hand and is not too short-sighted to explain to the interested readers that are not acquainted to the current topic what it is about, that seems to me slashdot's target audience. The main reason why I come here is that I tend to learn new stuff from some of the more insightful comments, not because I already know everything of cpu schedulers so I can bash other users without having to explain it to them.
molmod.com - computing tips from a molecular modeling
The scheduler is reponsible for deciding which programs can use the processor when. So, lets says you're running Word (not likely on linux, for quickest example I could come up with), and Word is for some reason using 100% cpu. Well, depending on how the scheduler works, that is how the cpu is shared between processes, other applications might continue working ok, or might freeze up entirely whilst they wait for Word to finish.
So, the scheduler controls how well different applications work together at the same time.
I don't know how well FreeBSD's ZFS port works (I'm using UFS2), but proper support for ZFS requires radically re-thinking your layering, and moving away from the traditional block-device, filesystem, VFS layers.
I am TheRaven on Soylent News
Well, your scheduler certainly wasn't fair on that one.
[emphasis mine]
I'm gonna have to go with Linus on this one.
Any "reasonable" reason why they couldn't integrate both?
Upward mobility is a slippery slope - the higher you climb the more you show your ass.
That's not true. Non-GPLed kernel modules are "tolerated" by the Linux kernel developers, and in principle, a ZFS module could be created and loaded with no problems, assuming it doesn't rely on GPL-only symbols. AFAIK, the VFS doesn't have many of those.
What can't happen under the CDDL is the ZFS code being included in the kernel source tree the same way XFS, ext3 and so on are. That doesn't mean you can't maintain and distribute a module separately! The only reason a ZFS module doesn't exist today is that nobody's gone through the trouble of creating one.
Whether Con was aware of it, when he tried to integrate into mainline Ingo was his main customer. Specifically the person he was trying to deliver work to. And Con committed the cardinal sin of telling a customer that the customer was wrong about what he wanted. Even if Ingo were too coked up to operate a keyboard reliably and had it all wrong, trying that never seems to work.
Did Con gain anything by refusing to re-introduce the hack to get X working the way it had previously under load? Even if he'd just put in a #define that allowed it, and then spent the next year arguing to take it out, there wouldn't have been this breakdown.
-- http://thegirlorthecar.com funny dating game for guys
Yes, but he has it filtered to only "Linux Flamewars"
whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
Start profiling the damn thing! Write performance tests, good ones, write really evil stress tests, stress the crap out of it, and then you will know what is *wrong*. Might not be the kernel at all, in fact, I think that a lot will be because of applications (always a huge source for problems), the UI/Graphics subsystem (again, huge source for trouble, X11, drivers, UI toolkits, they all tend to be far from perfect) and such.
:(
But pissing and moaning won't do you any good. At least Con did try to write stuff, but not being a professional software engineer hurt his efforts I'm assuming. The guy would probably make a good technical manager though. It is a shame he felt he had to quit, it would have been much better if he could have gotten a few other kernel hackers on with him to go on. I think he ended up with a lot of users backing him, but no coders
Not that I am a Linux person, but I always find it sad to see people who are really into something quitting for bad reasons (bad in the sense that the shouldn't have to, not that he did something unwise).
Please. You've got to be joking. Wow. I guess I just imagined all of those other articles. I guess there's never articles about biology, or physics, or genetics.
My affliction must have become REALLY bad, because now I'm seeing sections titled politics and science!
I'll grant you many of the articles do pertain to computers, but there are plenty that do not.
More like Surf's Up, but with emphasis on kernel development, instead of surfing. By the way, stay tuned for Dreamworks' Pictures "Tux". An animated comedy, starring yet even more penguins, and the lovable little guy himself. Tux is an ordinary penguin, working away at his operating system all day, like good penguins should. A chance meeting with the BSD Daemon and Gnu marks the start of a grand adventure that takes Tux all over the internet (including a brief visit to Slashdot itself). Tux makes friends with a host of colourful characters, meets the girl of his dreams, and learns an important lesson in process prioritization and task scheduling.
Unlike porn, which yada yada rimshot hey-ooh!
And what's with the massive ego? It's as if suddenly Linus thinks he invented compilers or something. I think he needs to take a vacation and regain some perspective.
This post expresses my opinion, not that of my employer. And yes, IAAL.
The CDDL was designed to be GPL incompatible, Solaris didn't want its crown jewels to be in Linux.
Furtunately, you're offering just an uninformed (and trivial at that) opinion. Unfortunately, you opinion seems to resonate with a large segment of similarly narrow-minded slashdot users. It is particularly tragic when folks here gravitate toward one steep world view without any desire to explore anything else outside of their comfort zone. It is a good thing that vocal narrow-minded elitist views (irritating as they are) are easy to ignore. The fact that diverse stories make it onto slashdot for all to discuss speaks volumes about that kind of site Slashdot actually is. We choose which stories we wish to participate in. The reality is that no one person (or even groups of persons) gets to define what slashdot is or isn't. Instead of helping to insulting others for not knowing much about the inner workings of the kernel, wouldn't it be more instructive to educate them on a subject which you (presumably) are more knowledgeble of? Instead of pushing your own agenda on a public community, why not take the time to look around and listen to what others may have to offer and then make the effort to make *your* contribution to other who need it? Please understand, I am not trying to attack you personally. I'm sure you're a decent upstanding guy. But the perspective of the opinion you've just expressed comes from a direction that hails from the most unsavory part of this great community, one that insults inteligent people and gives us all a bad name.
Stay sentient. Don't drink bad milk.
Hi, it eeems that the nerd card was issued to you by mistake. Please turn it in, and security will escort you out the building.
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
Quoting someone on ck mailing list, i think this is worth reading:
---8----
I don't really want to keep all that -ck flamewar going but this sum-up is
a little strange for me:
If Con was thinking SD was "perfect" why he released 30+ versions of it?
And who knows how many versions of his previous scheduler?
Besides Con always tried to help people and improve his code if some bugs
or problems were reported. Archives of this list prove that. I reported
several problems (on list and privately) and all were fixed very fast and
with very kind responses. I had run -ck for months and years and it was
always very stable (I remember one broken "stable" version).
I don't know what exactly are you refering to when you say about those
unaddressed reports but maybe it depends on who was asking, how and to do
what (for example - purely theoretical one, I don't remember exact emails
you refering to so I am not saying it happened - stating at the beginning
that the whole design is unacceptable and interactivity hacks are a
must-have won't make a friend from any maintainer and for sure lowers his
desire to get anything fixed for that guy). Or maybe Con had some bad day
or was depressed. Happens. But I really don't remember Con ignoring too
many valuable user reports in last 3 years...
And no - I am not thinking that SD was "perfect". Nothing is perfect,
especially not software. But it was based on months and years of Con's
experience with desktop and gaming workloads and extensively tested in
similar uses by _many_ others. In nearly all possible desktop
configurations, with most games and all video drivers. This is why it was
perfectly designed and tuned for such workloads while still being general
enough and without any ugly hacks. And because of these tests and Con's
believe that the desktop is very (most?) important all bugs and problems
in this area were probably killed long ago. I think even design was
changed and tuned a little at the early stages to help solve such
interactivity/dekstop/gaming problems.
So it does not surprise me that CFS is worse in such workloads (at least
for some people) because I strongly suspect that the number of people who
played games with current version of CFS is limited to about 5, maybe 10.
And I also suspect that you (and Ingo) will get many regression reports
when 2.6.23 is released (and months later too... or maybe you won't
because users will be to "scared" to report such hard to mensure and
reproduce "unimportant" bugs). Hopefully such problems when reported will
be addressed as soon as they can. And hopefully they will be easy enough
to solve without rewriting or redesigning CFS and causing that way even
more regressions in other areas. If not people will probably be patching
O(1) scheduler back privately...
Software projects need a benevolent dictatorship. If anything, there is too little in Linux. Look at MS, even a poor standard is better than none. Too many GUIs lead to people not liking to use Linux. I just want my OS to work, I don't want to have to make 8,000 choices. I want 1 answer to thc question of how do I do 'X', not a dissertation on the 8,000 ways that may apply depending on the settings I have chosen.
Your ad here. Ask me how!
"Linux without Linus cannot succeed. A lot of young folks don't realize this. RMS can't advance the cause of Linux like Linus does."
What does age have to do with it? You seem to be staking out some sort of claim to having a few more years and wisdom than some, but you're running an OS whose future you believe would end if Torvalds were hit by a bus tomorrow. This doesn't seem the pinnacle of old and wise.
The reality is that there would be a great disturbance in the force, and another maintainer would step in. Obviously, that wouldn't be Stallman, who has nothing to do with the Linux kernel, and never has. This the case with most major projects: kernel, GNU tools, X, Apache, PostgreSQL, Samba, or what have you. That's very nearly a defining characteristic of a major project.
As for your arguments about GPL versions, the facts of the matter are that different people have different goals, and different takes on what is the most moral thing to do--or exactly how much morality should even be part of the issue. As the creators, they certainly have that right.
The Linux kernel crowd and the Free Software Foundation generate a lot of press, arguing back and forth. This seems to have lead you to conflate the kernel and the GNU tools that surround it. They're very much different things. As is X, etc. So long as all the software produced by these major projects, and sharply different and philosophies, plays together well enough to run your hardware, and provide a userland, what's your beef?
What you do with a computer does not constitute the whole of computing.
Thread through many mails in the thread archived at lkml.org.
Overall, it looks like almost everyone feels that Linus's
portrayal of Con is absolutely unfair. Linus seems to have
made a decision & is then doing character attacks on Con
to justify his decision. There is even one mail where he
talks about Con's illness & says that it didn't help.
My personal opinion is that Linus comes out looking worse
in this Linus vs Con debate.
"You don't find the grail, the grail finds you." -- The Da Vinci Code
Missing... 'In Sovier Russia.. ' preamble! :)
What's in a sig?
I didn't want it to be too obvious.
I suffer from attention surplus disorder.
over MS ones any day.
Sounds to me either scheduler will do the job just fine.
The decision between two good alternatives is always a difficult decision - someone, no matter how good the ideas, will feel slighted.
Not read the comments, so refrain from marking down as redundant... that option should be removed anyway, except for abuses (e.g direct copies of other posts).
The statement "one is the question as to whether or not the Linux development model is working"
is ill thought.
The Linux development model is directly based up evolutionary theory - if it fails it will die, something else will take it's place.
THINK
A few months before Ingo wrote the O(1) scheduler, he flamed anyone who dared to suggest that an O(n) scheduler is a bad idea. He was *very* aggressive about it, going on and on about why O(n) is best and how O(1) would be worthless. Using Linus's words (about Con), Ingo "ended up arguing against people who reported problems [scheduler linearity], rather than trying to work with them". It therefore seems a bit strange that Linus uses this statement to describe Con, arguing this is why he favors Ingo...
Importantly, Ingo was dead wrong back then (indeed, this is why months after, Ingo came up with the O(1), announcing it as if it was his idea and as if nothing ever happened, not *ever* saying something like "I was wrong, sorry for the flames").
In contrast, Con was right in refusing to pollute the design of SD with Ingo's unfairness discipline. (This is what Linus referred to when he made the "arguing against" statement.) And what do you know? A few years after, Ingo comes up with a "Completely Fair Scheduler"...
I'm in scheduling research for many years. I followed the long Linux scheduling saga, which actually started way before Con was in business. Please believe when I tell you: Linus comments about Con are ludicrous, and petty. This is not Linus's finest hour.
Note however that this does not mean that Linus made the wrong decision: Even though SD is somewhat better than CFS, Ingo is orders of magnitude a better programmer than Con, orders of magnitude more knowledgeable, he gets paid to do the work, has gotten along with Linus for years, and will eventually make CFS as good as SD and even better. This is the real reason for Linus's decision. (Or at least, it should be.)
But the stuff Linus said about Con... well, that's just Linus being small.
As I understand it ZFS' design also doesn't fit well into Linux's abstract filesystem design, because of the different containers and whatnot, which makes it all the more difficult to port.
// MD_Update(&m,buf,j);
Ideally, tickless will obviate the need to play with HZ. I think this a good overall strategy: reduce the number of knobs, and replace them with intelligent algorithms that work for everyone.
One only has to look at the reaction of a programmer who quits because his pet project didn't get into the mainline kernel. He obviously was not the best choice for a long term maintainer.
Arguements over which code was superior are irrelevant at this point. The decision was made and will be made again when next revisited, after extensive testing and reporting. Right now the independent reports recieved by Linus were that they were pretty much equal. That only left him with the decision of who would be the better maintainer. He is the guy who gets to choose and I think he chose wisely. No offense to Con. I wish him all the best and hope he reconsiders as he matures.
I play games under Linux and I appreciate Con's efforts, but I understand that it is not the OS only focus. He didn't, Ingo does.
All Hail the Benovolent Dictator Linus. May his pragmatic glory shine upon thee.
OSGGFG - Open Source Gamers Guide to Free Games
If anything's tied up in closed source, it's Linux. There isn't any provision in the CDDL that prohibits combination with non-CDDL works; it'd be a Linux copyright holder that'd have to sue you to stop it.
A few months before Ingo wrote the O(1) scheduler, he flamed anyone who dared to suggest that an O(n) scheduler is a bad idea. He was *very* aggressive about it, going on and on about why O(n) is best and how O(1) would be worthless. Using Linus's words (about Con), Ingo "ended up arguing against people who reported problems [scheduler linearity], rather than trying to work with them". It therefore seems a bit strange that Linus uses this statement to describe Con, arguing this is why he favors Ingo...
Importantly, Ingo was dead wrong back then (indeed, this is why months after, Ingo came up with the O(1), announcing it as if it was his idea and as if nothing ever happened, not *ever* saying something like "I was wrong, sorry for the flames").
In contrast, Con was right in refusing to pollute the design of SD with Ingo's unfairness discipline. (This is what Linus referred to when he made the "arguing against" statement.) And what do you know? A few years after, Ingo comes up with a "Completely Fair Scheduler"...
I'm in scheduling research for many years. I followed the long Linux scheduling saga, which actually started way before Con was in business. Please believe when I tell you: Linus comments about Con are ludicrous, and petty. This is not Linus's finest hour.
Note however that this does not mean that Linus made the wrong decision: Even though SD is somewhat better than CFS, Ingo is orders of magnitude a better programmer than Con, orders of magnitude more knowledgeable, he gets paid to do the work, has gotten along with Linus for years, and will eventually make CFS as good as SD and even better. This is the real reason for Linus's decision. (Or at least, it should be.)
But the stuff Linus said about Con... well, that's just Linus being small.
Don't worry, there are enough non-knowledgeable people here to mod you down. I must admit that this debate really boils down to Linus and Ingo taking a long time to reach Con's conclusions, and then not being as gracious about it as they could be. Con, of course, was insulted by this (who wouldn't be?) and all the while we had two major camps of Linux users: the pro-CK one (holy crap! My system works with this patchset!) and the anti-CK one (damn! My server doesn't need this, wtf?).
In the end this just demonstrates that having egotistical bastards running the show isn't always going to yield the best results. Linus and Ingo do fantastic work, and they do sub-par work. But when someone points the sub-par stuff out, head for the hills! They will always have a legion of fans defending them, more than willing to dumb-down the real issue in favor of pretending their infallible leaders are never wrong.
So, lets says you're running Word (not likely on linux, for quickest example I could come up with)
What the fuck.
After all, I am strangely colored.
Thank you! I may not know what "staircase deadline" means yet (though the name gives some idea), but it's a lot easier to Google than "SD".
I read TFA, but I was still puzzled.
Agree 100%
Cheers
Ben
This is written by Linus in one of his emails to LKML:
> That was where the SD patches fell down. They didn't have a maintainer
> that I could trust to actually care about any other issues than his own.
I have been on Con's SD mailing list for >2 years now, and not once ever have i seen this to be the case.
He has always (time, etc permitting) followed up on user reported problems.
So... what happens if Ingo suddenly dissapears?
Im sure others will continue his work.
The same should be said for Con's scheduler also, which makes Linus's comments even stranger.
> Has Linux kernel development always been this ... arbitrary ?
Once upon a time... When "Linux" came on a stack of floppies.
There was just this one kernel developer. And whatever the fuck he felt like that day, shipped.
It's LESS arbitrary these days by a significant margin.
Do daemons dream of electric sleep()?
This whole situation reminds me a lot of devfs. The developer of devfs (Richard Gooch) was maybe a bit of an outsider with good ideas and strong opinions that sometimes clashed with those of kernel developers that were more insiders or closer to Linus. The story is a little different in that devfs had actually made it into the mainline kernel, but then later was replaced by udev (the first draft of which was by Greg KH in a day or so... again very much like Ingo's whipping up the first draft of CFS).
/lot/ of energy into their work. But that energy was not lost to the rest of us, as without the experience of their groundbreaking we would not have gotten the solutions we eventually did. Even if Greg KH's udev and Ingo's CFS share no appreciable amount of code or algorithms with devfs and SD, if they are honorable, I'm sure they would admit that they could not have so quickly whipped up their solutions without the example and inspiration of RG and CK's work.
/Maybe/ instead of losing them in the long run we could have gotten some more innovation from talented developers like RG and CK. The problem, I think, wasn't the decision adopt CFS instead of SD. Rather, regardless of whether or not is true, the problem is Linus's public judgement that CK is not a "responsive maintainer". I didn't follow the CK or the lkml lists in that time frame enough to know if he is right... but a really good leader would not have made such a judgement public even if hed believed it, and instead would have worked to find a way to keep such talented persons contributing.
Then as now with CK, eventually Richard stopped doing linux kernel work altogether. I thought it was a sad loss of a talented kernel hacker, and I had been a devfs user, but I must say that in retrospect I do think udev is a better solution. It is simpler, has less impact on the rest of the kernel, but has proven itself to solve all the problems devfs tried to solve that actually mattered.
What's the moral of the story? That both sides are right... on the one hand, there's something sad here, because at least several times in linux history an outsider had to fight for innovation and in the end was pushed away even as his innovation was grudgingly adopted by reinventing it. On the other hand the actual results do seem to indicate that linux is NOT resistant to change, and maybe that the better, more maintainable solution tends to win out.
There's also another thing to keep in mind... it is a pattern in this history of technology that the first attempt to solve a problem is rarely the one that becomes dominant. Both Con Kolivas and Richard Gooch should be recognized for the innovators that they are... and if they were wise they should also not begrudge the fact that it wasn't their exact solution which eventually got adopted by the mainstream. I know this is difficult... they both put a
Finally, I would like to add that although the way I see all this, it has little if anything to do with Linus's personality, nevertheless I think Linus could have handled these cases better.
I can't believe you read the articles. 1053r
Please stop stalking me, bro.
And so it becomes obvious what has happened here. There were two schedulers competing for inclusion in the mainline Linus kernel. Linus evaluated both and selected one. Having lost this race, Con declared Linux a failure and stormed off in a huff.
In other words: nothing to see here, move along, the increase in desktop Linux adoption will continue its slow but steady pace.
Tired of FB/Google censorship? Visit UNCENSORED!
You have to give Google some context. If you use con kolivas sd as your search terms you can just hit "I'm Feeling Lucky" and go right to this.
He lives in San Jose these days . . .
Actually, he moved to Oregon in 2004. I thought he lived in West Linn, but on at least one website, Dunthorpe claimed him. They're both suburbs of Portland.
Oh, come on, man! Zed's dead.
Do you also go as "Charles Goodwin" or "Free Gamer". The parent was also posted to http://kerneltrap.org/node/14008 , apparently 8 hours earlier.
If the answer is war, you are asking the wrong question
Somebody directing a large effort does not have the ability to fully investigate every subsidiary dispute fairly without the effort grinding to a halt. Shorthand criteria are a necessity, as are rules of thumb as to what sources to bother investigating. This will inevitably lead to a number of less-than-optimum technical decisions along the way, but the results will be markedly superior to those where the manager stops everything to thoroughly investigate every aspect of every decision.
Con Kolivas's reaction to "losing" was not to continue to maintain SD and try to get it in later, or to try to improve CFS, but to quit kernel-hacking entirely. Which means he is not of a temperament that can accept that large projects will have arbitrary decisions that go against him, which means he would be a bad choice for the maintainer of a major kernel system. His actions in retrospect justify Torvalds's judgment that he couldn't trust him as a maintainer. Kolivas proved Torvalds correct on the management question, even if Torvalds is wrong on the technical one.
He also said that as far as he knew, only part of ZFS was being opened via GPLv3, so it may not be worthwhile.
Sun is moving towards GPLv3 and hoping that Linux does the same so they can suddenly grab all the Linux device drivers.
http://blindscribblings.com - Tasty pop-culture in conceptual fashion.
I fully agree with the first part of your post, but I don't buy this 'Con's reaction justify Linus judgment' argument:
1) It seems clear on the mailing lists that Con was really a good maintainer (only one 'problem' reported).
2) Con's reaction seems quite understandable, after a very long time working on a project you see it not only refused by the maintainers (perfecty ok on that), but suddenly 'copied/inspired/wahtereryouwant' by that very same maintainer (not quite right).
EGO leads to sectarism, that's all. The problem is that it seems that Con's scheduler was very good at gamming, and it's a shame that Linux dimissed a good piece of code on a specially sensible area for personal motives.
What's in a sig?
It would be? It was! C.f. http://linux.slashdot.org/article.pl?sid=07/07/24/ 1432245, and if you search on lkml.org you can find the actual email where ck submits his last contribution and asks not to be "contacted regarding this or any other kernel development" issues. Really rather disappointing. And by the way, the interview linked from the article is very much worth reading.
Well, before this article I didn't know what SD is, I googled it. This summary nor article didn't tell me it. If you don't want to learn new things, you are not target audience. It has little to do with knowing little from computer field.
Extreme Programming - Redundant Array of Inexpensive Developers
I don't know everything about schedulers either. But I DON'T wait for others to explain it to me. I didn't even knew staircase-deadline scheduler exists. There are many information sources out there in vast internet, just use it more often.
Extreme Programming - Redundant Array of Inexpensive Developers
You have a userid that's half of mine, and you still haven't gotten what kind of stories people here like? This story is, as I see it, "old school" Slashdot like the late 90's Slashdot -- Linux kernel development news and discussion. And science. And interesting cool things.
How often do you read this site? Any controversy regarding the kernel and Linus' comments on said controversy is sure to end up on the front page.
Yours is a very good post. Shame that you did so anonymously. I would like to add that I think the overall sentiment here on slashdot (well, mine anyway) is that it's a shame that _nowhere_ on these lists does Linus come off as anywhere near apologetic. As others have pointed out: a valuable kernel developer was lost through rudeness and incomplete information gathering and apparent cronyism and petty politics, and in the development process we all support (that of open source over the internet), that is something that should be avoidable and avoided. Why wasn't it ?
Oh and don't come back to me and say he can fork the kernel - he can make his own distribution - he can come back when there is a specialized distro which does allow for pluggable scheduling. The fact remains that a superior solution at this point in time was rejected over an inferior one, that the developer of the inferior one suffered from NIH-syndrome and made vague claims of other people's hard work and twisted and turned which made it look almost like he was trying to revise history, and that the developer of the superior solution walked away frustrated.
Open source lost a little bit - the process needs revision IMHO. And Linus should issue some kind of statement that he is at least a little bit sorry.
Religion is what happens when nature strikes and groupthink goes wrong.
I don't disagree that a story about the scheduler is potentially interesting, and that it's what Slashdot used to be about. But as you can see from the above comments, it's not exactly stuff that "nerds" these days care to Google when they don't understand what it's about. Personally, I started reading this site because I ran (and run) Linux, not the other way around. So I've got nothing against this particular story, even though most of the discussion is very poor, and, sadly, typical for this site.
That said, I do disagree that Slashdot has ever been a science site. There's a lot more to science than what interests the nerd crowds (the proper nerd crowds, not the Apple fanboys and gadget freaks, who could care less about how stuff works). Science, as it concerns Slashdot, is mainly science that concerns technology. Throw in a few dinosaurs and volcanoes for good measure. It's boys' stuff. Science fiction stuff. You won't find anything about the travel patterns of herring or anything that takes the social sciences seriously (plenty of people here would even claim "it's not science", demonstrating their own lack of insight into what the scientific method(s) constitute).
If you like Con scheduler apply the patch and STFU. The Desktop user experience is subjective. A scheduler design has little to with hour glasses and smooth scrolling. The scheduler is blind to weather you are handling a http request or pushing a msg into the sound driver.
Should it be?
Monolithic design anyone?
Poor product integration? oops! Linux is not a product. Kolivas nails the price the PC architecture
has paid by de-coupling software and hardware. Hardware on steroids and the half ass software that goes with it.
But I deride those who whine about poor desktop experience and demand the OS community to do something about.
In Spanish we have a phrase for it:
"Quieren mamar y dar topes"
- these are not the droids you are looking for -
linus has lost this PR war for once. now give us the #ifdef DESKTOP so that we can compile our own desktop-optimized kernels and see for ourselves if anything breaks like hell. the one size fits all principle is stupid is so very much not taking into account the power of the C preprocessor. the cpp is a tool for allowing choice. use it. you can even call it #ifdef DESKTOP_UNSUPPORTED if you want to make it clear, that users are going to go for potentially unsupported or broken code.
Hell, what about the crap they did with MFC to make it difficult for people to make multi-threaded GUI applications so they're always responsive?
Who are you trying to kid?If only there was such a thing under Kubuntu, Redhat, CentOS, Mandriva, SuSE... Oh wait!I think it works very well as a general use OS actually.
I think it fails where it needs to use specialist Windows only programs that don't run under Wine out of the box. It also fails where Windows is the only option available in shops, I'm sure if Linux were the only option in shops, it would gain such great momentum too.Well according to your reasoning, there isn't a problem.
Change is certain; progress is not obligatory.
from a psychological point of view...
SD = staircase deadline
sounds like an algorhythm.
CFS = completely fair scheduler
sounds like a hopefully self-fulfilling promise, not an algorhythm.
this of course says nothing about the code.. just noticing the detail...
My read is different, but it'll take a few paragraphs to explain why, so please bare with me. The O(1) so called "improved interactivity" was just a hack upon a hack upon a hack that never worked. If you ever read the code you know what I'm talking about. It was a complete mess. The mess was the result of trying to obtain a goal that is unobtainable: very roughly speaking, Ingo (and others) believed that it is possible to deduce how "important" a process is, based on the frequency of the process' sleep events (not the duration of the sleep). I can point you to a few research papers that show that this can simply *not* be done. Briefly, it has been shown that for each "important" application you can find an "unimportant" application such that the CPU usage pattern of the two is very similar; Hence, you would never be able to distinguish the important from the unimportant (if you are only basing you decision on CPU usage). All you can do is (1) divide the CPU equally, and (2) provide good response time to applications that sleep for long periods of time (like editors). According to several of his emails, Ingo is not reading scheduling papers, and proud of it.
I suspect Con doesn't read such papers either. But he knew this is the case nevertheless. For years Ingo and his followers attacked Con on this. Note that this is a purely technical issue, and Con was *completely* right, e.g. take a look at CFS, or at Linus's original scheduler that lasted until 2.2, or, Solaris, *BSD (with the exception of ULE), HPUX, etc. They are all much closer to the SD's philosophy than to the O(1)'s. (BTW, the only other OS that tried to do what Ingo wanted is... the Windows family).
The discussion between Con and others regarding this issue was not about minor details. Ingo and his followers view towards Con was plain and simple: "your design is crap, it doesn't do what schedulers are supposed to do". Con resisted. And I agree, after years in the business, he was occasionally emotional about it. But this is really understandable and reasonable. Especially when you compare it to the way Ingo and Linus express themselves from time to time (talk about "drama queens"). For example, notice how Linus talks to Ingo when Ingo doesn't understand something immediately.
The bottom line is that most of them are "drama queens" from time to time. Nobody is perfect. In absolute terms, however, I think Con usually expresses himself in a calm, quiet, and relatively humble manner. Certainly in comparison to Ingo/Linus.
>Even though SD is somewhat better than CFS
Ah ok a solution is better... We use that solution, right? Because after all is that not what Open Source is all about? (http://en.wikipedia.org/wiki/Meritocracy)
Meritocracy is a system of government or other organization based on demonstrated ability (merit) and talent rather than by wealth (plutocracy), family connections (nepotism), class privilege, cronyism, popularity (as in democracy) or other historical determinants of social position and political power.
>Ingo is orders of magnitude a better programmer than Con, orders of magnitude more knowledgeable, he gets paid to do the work, has gotten along with Linus for years, and will eventually make CFS as good as SD and even better.
Ok so demonstrated was a scheduler that was better, but chosen was somebody who is perceived as being more knowledgeable, and gets along with Linus and EVENTUALLY will make CFS as good or better... And until EVENTUALLY hits I should wait around and suffer the problems? And how is this different from say Microsoft who refuses to fix bugs?
Let's call this what it is! Corporatism at the open source level with Linus's nepotism!
"You can't make a race horse of a pig"
"No," said Samuel, "but you can make very fast pig"
I'm only 19, and I've never contributed a single patch to any software project, but even I know exactly what this is about. The worry here is that the Linux kernel developers are acting like snobby elitists, and that they care more about patting each other on the back than they care about improving the kernel. They're rather shit up the kernel than listen to someone who isn't part of their inner circle. It's pretty major stuff.
I just wish someone else had the brains to actually start another one. It seems like a fiendishly difficult process.
However, Linus is apparently massively overworked, his ego has gone berserk, there are apparently any number of technical problems, and then there's been the whole brouhaha about whether or not the thing is going to be pushed to the GPL 3 over the heads of people who don't actually want that. In other words, it's a mess.
Sorry, but this is people acting like typical people. Possibly enlightened sages don't act this way...if they don't, though, it's because you chose a definition of enlightened that excludes those who do.
People trust their friends, and are wary, sometimes unreasonably so, of strangers. That's one part of what's going on.
Now, is it the right decision? I'm not scheduling specialist, and I don't know the people involved. Linus has a track record of usually making choices that will work well, so that's the way I'd bet. Was Linus fair, honest, etc.? Others have said that he is here acting petty and spiteful. He human, so that's quite possible. Was the other guy offended? I'd bet *very*.
People often act in ways that are designed to keep their groups at the right size. Often by driving off strangers. Maybe the group of kernel hackers is at around the limit of what Linus can manage (given current techniques). Everything has multiple limits. If that's what's going on, it's too bad, but may be necessary. (Note that even if this is what's happening, it may well be that nobody realizes it. People often don't.)
I think we've pushed this "anyone can grow up to be president" thing too far.
Thanks for the clarification -- I agree. Mind you, I'd be interested in a news story about the travel patterns of herring ;)
I read the post. There's a big difference between sleep-duration (to which you refer) and sleep-frequency (to which I refer). Basing decisions about sleep-duration is fine, and is equivalent in every respect to basing decision on runtime (as the two are complementary). The O(1) was largely based on sleep-frequency. In contrast to what the lwn post claims, the two schedulers have a lot in common.
Well for one thing, I know this whole history has scared me away from ever considering to contribute to the Linux Kernel.
RogerWilco the Adventurous Janitor
How can a person be "orders of magnitude a better programmer" if they repeatedly make significant errors of judgement?
Ingo wrote the bad scheduler that Con dramatically improved! Con only had work to do because Ingo dug Linux into this poor scheduler design to begin with.
Maybe you could argue that Ingo writes cleaner code or something, but to me good judgement is what makes a programmer good.
I guess I would say that everybody that is actually doing something, makes mistakes. But I do agree that the O(1) was a big mistake, which no doubt reflects badly on the Linux development process. And Con's work has indeed managed to put the scheduling subsystem back on track. Plus, I think it's safe to say that Con is orders of magnitude a nicer guy than Ingo.
But I'm being realistic here: Please consider that even though Ingo implemented the O(1), Linus was the one that decided that all this messy interactivity mambo jumbo would make it into the mainline and replace the scheduler Linus implemented more than a decade ago. At the bottom line, it's Linus's mistake and Linus's responsibility. Also, a lot of really sharp guys didn't fight against the O(1). So there's something of a collective responsibility here.
In light of this, if you were Linus, who would you choose? I'm not at all suggesting that the answer is carved in stone. I'm merely saying that the Ingo alternative has significant benefits. He is after all a very talented professional (Con used the work "brilliant" to describe him) that gets paid to do the work. I argue that these pros shouldn't be dismissed.
in my life God comes first.... but Linux is pretty high after that
Francis Smit
As i understand it...
There is the scheduler made by CK, which has been developed for months, maybe even years in the -ck patchsets...
Then there is the scheduler made by Ingo, which performs about the same and is designed in much the same way, but which was developed in under a week...
Does this not strike you as wrong?
Firstly, that something would be merged into the mainline kernel with so little testing...
Second that Ingo would develop a new scheduler, rather than modifying the existing and fairly similar one by ck? Or is that one somehow so flawed it needed to be junked?
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
This is pure speculation, as the real work will be in comparing the two schedulers and if one really is better than the other, adopting the better one, but I'd say that if one took a very long time to make, and the other a very short amount of time to make, that perhaps the shorter one has less code. If it does the same thing, then I'd pick the one with the less code, unless the other one provided extra features, including perhaps modularity.
:)
Even though you may not know what exactly all went down with CFS and CK's version, you can look at the code and compare them yourself. There are lots and lots and lots of people with an interest in making Linux a good OS, so if Linus made the wrong choice, he should be made aware.
Promote true freedom - support standards and interoperability.