Slashdot Mirror
Intel Chief Evangelist Comments on Linux Scheduler
ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a
recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
XML is like violence. If it doesn't solve the problem, use more.
Oh, until the second page, which I didn't see.
Yes, and has for at least a decade. As has SGI's IRIX and probably a lot of other operating systems. (Linux would be included in that list based on the comments here.)
Try Robert Love's "run" command... Processes can be started or moved to any CPU or collection of CPUs.
First of all, any job with the word 'evangelist' directly implies 'highly paid fanboy.'
I worked with the Evangelists in Apple Developer Relations, and my direct personal experience tells me that you're full of shit.
Apple's Technology Evangelists are some of the most talented technical marketing professionals I've ever met, and now that I'm an outside developer, they do a great job for me pulling the levers and turning the gears that I need to get my products out the door, connecting our company with customer prospects, and finding and introducing us to other companies that we can collaborate with. I'd hire any of those guys in a heartbeat for either a development or a marketing position, if I could get them out of Apple.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
I think that what Intel's Evangelist wants is already available through standard POSIX pthreads and the POSIX-RT extensions.
man pset_create, pset_assign, or pset_bind.
I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html
"Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."
what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.
the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.
there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.
Not only that, Sun junked the multi-level scheduler idea in Solaris 9. In Solaris 8 (unless you linked to the alternate thread library named something like /usr/lib/thread/libthread.so) a multithreaded process could have M threads, that would run on N kernel processes (light-weight processes or LWPs). The one process (the one started from main(), etc...) would schedule the threads itself on its N LWPs.
In other words, while a call to pthread_create() would start a new, separate thread of execution in *your* process, but may or may not start a new kernel-level thread that would run your new execution thread. And a user-level scheduler would schedule your threads on your LWPs. The kernel would schedule the LWPs.
Sun junked this because it doesn't scale well when you get into hundreds of heavyweight processes and thousands of threads and/or LWPs.
In Solaris 9 (and the "alternate thread library in Solaris 8), each separate call to pthread_create() spawns an LWP, and there's only one scheduler - the kernel's. There's no longer on in each multithreaded app. (And yes, there were ways to get this one-to-one behavior in Solaris 8 and earlier, but that's not the point...)
And this guy's saying we should go back to that?
Sorry, but Sun knows a lot more about massively parallel architecture than Intel does. Intel lives in a world where 8 CPUs is a lot.
I've had no such issue with the .22 kernel, so likely this is being hammered out as time goes on.
Liberte, Egalite, Fraternite (TM)
Most marketing people are not technical professionals. That's not the case at Apple. Phil Schiller doesn't mention it very often, but he started his career writing code.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
He's an expert at (amonst other things) providing automatically optimised applicaton parallelisation in general, which as such is not available (I can know, I used to be a reaesrch team leader in a company that was working on it as well, but from a power management angle).
What he said about OS-level schedulers is that he ideally wants them to behave in a specific way, without even saying that specifically Linux needs to be changed (again: there are hundreds of OS-es out there). He has not said that Linux is "behind" or "retarted" anything. He has not said that Intel needs volunteers to do his dirty work for free. The only person who used the word Linux during that interview was the interviewer (as usual, the /. title gets it completely wrong and the /. summary is a composition of two quotes originating from the guy's anwers to two different questions). Read and understand the bloody article before posting nonsense!
In any case, if the thing has been available for many years and people are to be ridiculed for seemingly not knowing that, they what the hell are Ingo and Con fighting about? They must be non-experts doing useless work. Both of them.
Linux user since early January 1992.
By the way, NetBSD still uses an N:M model (FreeBSD tried it, but abandoned it due to code complexity issues).
I am TheRaven on Soylent News