Intel Chief Evangelist Comments on Linux Scheduler

ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"

Comments

the real world, too.

[User 956]

If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads

Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.

I don't understand

[Junta]

I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'" Isn't this *exactly* what numactl gives you (hint, you don't have to run numa for numactly to allow processes to be bound to specific sets of processors).

Re:I don't understand

[Rogerborg]

Christ, yes. Propagandist is more accurate.

Back in the Day, Microsoft's European DirectX 'evangelist' paid a visit to our game dev studio to try to sell us on the benefits of Direct X Retarded Mode. We had our engine running with D3DRM and Glide (I said, back in the Day), and the Glide framerate was easily twice that of D3DRM. He told us that that was impossible. We said "Look at the screen.". He literally refused to look. He just insisted that we were wrong, that there was no reason that D3DRM should be slower than Glide, and that we must therefore be doing something wrong. Did I mention that he wouldn't look at the screen?

Needless to say, we didn't even offer him lunch.

So do it

[QuantumG]

I hear Intel has some engineers who you could get to implement this. If you don't completely botch it, you might even get it into Linus' tree.

It's Free Software, you can add whatever you want.

Re:So do it

[teknognome]

It's elitist and arrogant when directed at a regular user, but when it's directed at some official person of a company that does contribute to Linux projects, it's a bit different.

Re:So do it

[Anonymous Coward]

Way ahead of you.

$ sudo apt-get install schedutils
...
Unpacking schedutils (from .../schedutils_1.5.0-1_amd64.deb) ...
Setting up schedutils (1.5.0-1) ...

$ man taskset | cat
Reformatting taskset(1), please wait...
TASKSET(1) Linux User's Manual TASKSET(1)

NAME
taskset - retrieve or set a processes's CPU affinity

SYNOPSIS
taskset [options] [mask | list ] [pid | command [arg]...]

DESCRIPTION
taskset is used to set or retrieve the CPU affinity of a running pro-
cess given its PID or to launch a new COMMAND with a given CPU affin-
ity. CPU affinity is a scheduler property that "bonds" a process to a
given set of CPUs on the system. The Linux scheduler will honor the
given CPU affinity and the process will not run on any other CPUs.
Note that the Linux scheduler also supports natural CPU affinity: the
scheduler attempts to keep processes on the same CPU as long as practi-
cal for performance reasons. Therefore, forcing a specific CPU affin-
ity is useful only in certain applications.

The CPU affinity is represented as a bitmask, with the lowest order bit
corresponding to the first logical CPU and the highest order bit corre-
sponding to the last logical CPU. Not all CPUs may exist on a given
system but a mask may specify more CPUs than are present. A retrieved
mask will reflect only the bits that correspond to CPUs physically on
the system. If an invalid mask is given (i.e., one that corresponds to
no valid CPUs on the current system) an error is returned. The masks
are typically given in hexadecimal. For example,

0x00000001
is processor #0

0x00000003
is processors #0 and #1

0xFFFFFFFF
is all processors (#0 through #31)

When taskset returns, it is guaranteed that the given program has been
scheduled to a legal CPU.

OPTIONS
-p, --pid
operate on an existing PID and not launch a new task

-c, --cpu-list
specifiy a numerical list of processors instead of a bitmask.
The list may contain multiple items, separated by comma, and
ranges. For example, 0,5,7,9-11.

-h, --help
display usage information and exit

-V, --version
output version information and exit

USAGE
The default behavior is to run a new command with a given affinity
mask:
taskset [mask] -- [command] [arguments]

You can also retrieve the CPU affinity of an existing task:
taskset -p [pid]

Or set it:
taskset -p [mask] [pid]

PERMISSIONS

Oh, man

[Anonymous Coward]

I scheduled an interview with linux a while back. It was all like, "LINE UP", and I was like, "WUT?", and it was like, "What's your priority?", but I was all like, "interupt, bitch", and it was like, "cool, SWAP", and I was all like, "I'm in ur processor, executin' my code".

Intel's Chief Evangelist

[MightyMartian]

And I'm gonna tell ya... I'm gonna tell all the people here-a, that the Lord has come... Amen! Yes indeed, and the Glorious Pentium IV has arrive-ed! Praise be to Pentium! And lo, the sheep of the field will line down-a with the process, and the thread-a will be managed by the application!

Oh, don't be fooled by the Devil... No-a! AMD is the sign of the Beast-a! And he shall be cast out of heaven! Raise-a your hands to the heavens and press CTRL-ALT-DELETE!

Re:Hello??

[phoenixwade]

The word Linux doesn't even appear anywhere in that entire article. Yeah, it does, almost to the end of the article... on Page - 2. But that is irrelevant. Yet another summary that has little or nothing to do with the article, Which actually focuses on Intels Threaded Building Blocks. I really hate misleading summaries. I know that some have issues with the signal to noise ratio of "news" to marketing or old news, but really, a good summary means you could, at least, skip over the stuff you didn't want to read....

Slashdot would be kicked out of Journalism School

[headkase]

Not only is it obvious that the submitter didn't read the article but by posting it its obvious that the 'editor' didn't either! Jebus! It's about a library template that Intel provides to developers! Linux gets a passing mention on the second page but other than that coincidence the writeup has nothing to do with the real article! And then there are people tagging the article as 'clueless' - I hope they meant the submitter because if they meant the article then ironically they would have been showing their own 'cluelessness'.
This is the first time I've felt a need to complain about /.'s integrity btw even though I've seen more articles fly-by that shouldn't have over the years.

Re:Puh-leeeeze!

[jcr]

First of all, any job with the word 'evangelist' directly implies 'highly paid fanboy.'

I worked with the Evangelists in Apple Developer Relations, and my direct personal experience tells me that you're full of shit.

Apple's Technology Evangelists are some of the most talented technical marketing professionals I've ever met, and now that I'm an outside developer, they do a great job for me pulling the levers and turning the gears that I need to get my products out the door, connecting our company with customer prospects, and finding and introducing us to other companies that we can collaborate with. I'd hire any of those guys in a heartbeat for either a development or a marketing position, if I could get them out of Apple.

-jcr

Available through POSIX pthread and RT extension

[EMB Numbers]

I think that what Intel's Evangelist wants is already available through standard POSIX pthreads and the POSIX-RT extensions.

man pset_create, pset_assign, or pset_bind.

I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html

"Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."

its called "scheduler activations"

[paulbd]

what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.

the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.

there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.

Re:its called "scheduler activations"

[anticypher]

it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues

I haven't worked in that field in some number of decades, but I'm going to have nightmares because of that phrase. You heartless Bastard, at least warn us that the path to madness lies within your post.

the AC
sits here rocking back and forth, afraid to google what Tom Anderson's later works covered, but knowing I'm about to lose my weekend to this

Re:Puh-leeeeze!

[macshit]

Apple's Technology Evangelists are some of the most talented technical marketing professionals I've ever met

Wait a minute, is that a compliment or an insult?!?

Re:You think maybe they have other things to do?

[xenocide2]

Either way is ok, there's nothing wrong with saying "We are all about DIY, if you aren't willing to do it, don't expect other people to), you just have to recognise that is a viewpoint inconsistent with "We believe this is something everyone could and should use." There's nothing necessarily incompatible with those viewpoints. The point isn't necessarily "write it yourself," but to take ownership of the need and get it done, perhaps by paying others. You can then go about writing to others and say "You should use this stuff, and we can help make it better for your needs." Its a silly belief that other people are going to fix your problem for free if you sit there wishing loud enough. Open source projects are truly successful not when you take responsibility for listening to what people want, but allowing the public to take ownership in the software that could address their needs.

Re:You think maybe they have other things to do?

[xenocide2]

I should point out that SGI has done exactly this: on behalf of their customers they go about fixing scalability problems in Linux. The results are quite interesting -- eight thousand CPU computers and the like. To give credit to the Evanglist, perhaps he was hoping coworkers and superiors at Intel would recognize the need and step up.

Re:The name doesn't need to be obvious

[ricegf]

Is the name 'ls' obvious? Is 'cat' obvious?

Well, sure. 'ls' gives you a catalog of files on the disk, while 'cat' lists one or more of the files' contents. But Windows' 'dir' is much harder to remember - it just means "Drrr, I wonder what's in this folder?"