Slashdot Mirror
Intel Chief Evangelist Comments on Linux Scheduler
ByeByeWintel writes "James Reinders is Intel's Chief Evangelist for Intel's Software Development Products. In a
recent interview on Devx.com he stated: 'If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling. And demand that processes manage their scheduling of threads ... There is a lot of opportunity for operating systems to offer these types of control in the 'running of applications' interfaces. I'd like an OS to let me specify the 'world' my application runs in (which processors, how many, etc.) These interfaces are available in Windows at run time (the task manager will let you adjust where a running task can go).'"
From TFA: Q: Could you introduce yourself?
James Reinders: I'm Intel's Chief Evangelist [blah blah blah]. Another way to look at it is I'm an engineer who joined Intel in 1989 'cause I thought it would be a cool place to work for a few years. [snip]
Wha wha wha what??
First of all, any job with the word 'evangelist' directly implies 'highly paid fanboy.'
Second, people who use slang contractions in text (i.e. " 'cause") should be shot on site.
If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads
Yeah, a lot of us feel the same way about the fancy-dressing guys that work over in the sales office.
The theory of relativity doesn't work right in Arkansas.
XML is like violence. If it doesn't solve the problem, use more.
I hear Intel has some engineers who you could get to implement this. If you don't completely botch it, you might even get it into Linus' tree.
It's Free Software, you can add whatever you want.
How we know is more important than what we know.
James doesn't seem to acknowledge that such features already exist in Linux as well. Additionally Linux has __FAR__ better support of NUMA system management. Many Opteron-based systems are NUMA as well as many high end Intel-based servers. Is he just oblivious?
Resume your regularly scheduled Con vs Ingo flamewar.
I scheduled an interview with linux a while back. It was all like, "LINE UP", and I was like, "WUT?", and it was like, "What's your priority?", but I was all like, "interupt, bitch", and it was like, "cool, SWAP", and I was all like, "I'm in ur processor, executin' my code".
Right after people who can't tell the difference between "on sight" and "on site".
And I'm gonna tell ya... I'm gonna tell all the people here-a, that the Lord has come... Amen! Yes indeed, and the Glorious Pentium IV has arrive-ed! Praise be to Pentium! And lo, the sheep of the field will line down-a with the process, and the thread-a will be managed by the application!
Oh, don't be fooled by the Devil... No-a! AMD is the sign of the Beast-a! And he shall be cast out of heaven! Raise-a your hands to the heavens and press CTRL-ALT-DELETE!
The world's burning. Moped Jesus spotted on I50. Details at 11.
The word Linux doesn't even appear anywhere in that entire article.
It was a two page article, you fool!
from page two of TFA:
How does TBB interact with the OS scheduler? I guess you read the recent discussion about two Linux schedulers...
James Reinders: TBB sits on top of the threading interfaces offered by operating systems. An enhancement we are working on, is to tackle the problem of interaction with the OS by providing 'affinity' requests to lock threads to particular processors. This seems like an obvious optimization, but once you try it you find it is anything but obvious sometimes. There was a paper a couple years ago which showed that using affinity made runtimes more predictable but raised the average runtime because it stole control for the OS--and that the OS was making optimization which the program did not. So leaving it to the OS the runtimes varied more widely, worse and best cases were more extreme, average was better--and overall it looked less predictable than using affinity. This whole area needs a lot more investigation.
Nope, you complainers got it backwards. This is a tech expert in his area, saying that he would want OS developrs - which he is not - do change their ways. Being the expert that he is, he is very much entitled to say that. What's more, he would still be entitled to say that even if he were in charge of the OS. Note that he doesn't say: "go out and do it for me because I'm lazy and don't want to pay". For all we know, he might be busy pulling "evangelical" levers to get it done at Intel and we are just observing one of those levers in action. Besides, he said "OS" not "Linux". It may come as a surprise to some of you, but Linux is not the only OS in the world.
Linux user since early January 1992.
Try Robert Love's "run" command... Processes can be started or moved to any CPU or collection of CPUs.
Not only is it obvious that the submitter didn't read the article but by posting it its obvious that the 'editor' didn't either! Jebus! It's about a library template that Intel provides to developers! Linux gets a passing mention on the second page but other than that coincidence the writeup has nothing to do with the real article! And then there are people tagging the article as 'clueless' - I hope they meant the submitter because if they meant the article then ironically they would have been showing their own 'cluelessness'. /.'s integrity btw even though I've seen more articles fly-by that shouldn't have over the years.
This is the first time I've felt a need to complain about
Shh.
Is the name 'ls' obvious? Is 'cat' obvious?
The names of tools don't need to be obvious, they just need to be memorable.
Because you see we have this fantastic brain thing that happens to be absolutely marvelous at associative searching of an immense word space, so any fairly pronounceable word becomes recallable after you've used it once or twice.
Most of what that guy wrote is total bollocks, and that includes the technical part about scheduling. I'll leave that for another post though.
I think that what Intel's Evangelist wants is already available through standard POSIX pthreads and the POSIX-RT extensions.
man pset_create, pset_assign, or pset_bind.
I just checked and it seems that Solaris, HP-UX support this POSIX feature. Maybe it's only Linux that is non-standard: I found ythis quote: http://developers.sun.com/solaris/articles/solaris_linux_app.html
"Both Linux and the Solaris OS support the notion of binding a process or thread to a processor. Linux allows binding to a set of processors for non-exclusive use of those processors. The Solaris OS allows binding to a set of processors for exclusive use, (that is, CPU fencing), but does not allow binding to a group for non-exclusive use (except via Solaris Zones?). Linux does not have a mechanism for CPU fencing, though implementations can be found on the web (see, for example, the CPUSETS for Linux page on the bullopensource.org site). The Linux system calls that are processor affinity based are sched_setaffinity(2) and sched_getaffinity(2)."
I mean it seems that Intel does one or two other things like, oh, I dunno develop processors, motherboards, NICs, video cards, RAID controllers, drivers for those, compilers, profilers, and a few other things. Perhaps they really don't have the time or interest to devote to that project.
Sorry, but the "Do it yourself," attitude is just bad. Even assuming it is directed at the extremely small segment of the population that has the level of programming knowledge required (usually it isn't) that then assumes that they have nothing better to do with their time. Sorry, not how it works.
Part of having a successful product is listening to what people want and working to implement that. Now maybe you think that the devs shouldn't care, that it should just be whatever it is and there shouldn't be any thought given to making it what people want, unless said person is willing to do that. Fair enough, that's a valid stance. However if you take that stance, then do go evangelising Linux as a replacement for Windows or an everyman's OS. Realise that if that is how things are to be done, it is a sysadmin/geek OS and will never be otherwise.
Either way is ok, there's nothing wrong with saying "We are all about DIY, if you aren't willing to do it, don't expect other people to), you just have to recognise that is a viewpoint inconsistent with "We believe this is something everyone could and should use."
what is being discussed is called "scheduler activations" within the CS community (or was). its an old idea. i did some work on a real-world (hah) implemention back in the early 1990's when i worked at UWashington. google it. Solaris actually added this design at least 10 years ago (plus or minus 2 years). its a very cool OS design, but can also be hard to get the implementation right; it also requires both kernel and userspace implementations.
the basic idea is that the kernel doesn't try to decide which threads within a task/process should run. as long as the process is scheduled to have access to a CPU, whenever its about to block (e.g. on disk i/o) or to be granted a processor from another task, the kernel tells the user space scheduler what is going on. scheduling is then done in user space, where maximal knowledge about the applications internal design and thread priorities can be easily accessed.
there are several papers on this design, ranging from Tom Anderson's "original" through reports on various implementation efforts. it was certainly fun trying to write a user space context switch routine that has to be reentrant itself, not to mention trying to deal with priority inversion issues. i think sun simply worked around the latter problem with some design assumptions/limitations, but i don't know for sure.
I did my hard-time at Intel designing shit (thats a techinical term) - this guys just another over-paid suit who's over stayed his welcome - taking up space - money - and oxygen from the folks who actually do shit (another technical term). I did some good shit (more techincal terms) at Intel but the best day was when I ejected out of my cube and moved on ..... that is one fucked place.
Its not the years, its the mileage
Intel is not offering to fund such a venture. They expect these volunteers to be at their beck and call whilst claiming to know best. Unless Intel is going to hire a cadre of developers or otherwise pay a bounty for this, demanding a bunch of volunteers to do their dirty work is pretty elitists.
Surely I'm not the only one who didn't see any mention of linux in the summary, and clicked the article and did a search for linux/nix only to find nothing?
Would a virtualization layer fulfill this? VMware's ESX is essentially an OS in itself so it has schedulers that schedule VMs, which is a lot like processes. Inside the VM, the guest OS then schedule its own processes. So maybe the answer is to write and ship application inside virtual appliances. This is just me pondering.
EvilCON - Made Famous by
I'm working on a Slashdot "killer" which will be out by end of October. Posting as anon for obvious reasons (although editors can probably see my IP and correlate to my account, oh well).
Beside better editors and articles, what else would you like to see changed in Slashdot? Would those changes make you move to another site?
I'm really willing to give this a shot. Slashdot isn't that horrible, just the noise is way too much, and it takes way too much time to find the good stuff hidden in there. It doesn't have to be like that.
Please let me know, thanks!
It's easy to be an Apple Evangelists. Look what Apple has done with Intel chips. Cool notebooks, great workstations and even the X-serve trumps similar servers using the same chips from Intel. They design everything better! Look at the similar servers: IBM servers are what you would expect IBM servers to be, solid, expensive and well serviced. Dell and HP servers are almost what IBM's are, but less expensive. Apple makes better servers, but the OS keeps their market numbers low, even though you can run other operating systems, directly or as virtual machines. Sun's Sunfire servers may be a better buy, they are well designed, solid, well serviced, less expensive and I think they are faster at this point in time, but they are not using the Intel chips. Does AMD even have Evangelist's?
If I could get ONE wish fulfilled would be for OS scheduling to focus on processes, and not threads, for scheduling.
One wish, and THAT'S what he'd choose? That's effed-up. Evangelize a new set of priorities! Infinite wishes, dumb-ass!
Rumor mill: Intel is in the process of creating the Chief Evangelists Department. This innovative organization will hire all participants in this excellent debate for the positions of Chief Evangelist. According to the rumor all posters will be contacted by email.
Ok, this is a knee jerk response from someone not reading the article yet, so forgive me in that regard. I was to understand from someone else that every thread in Linux is actually a separate process and therefore carries with it a performance hit for multi-threaded applications VS say BSD systems. Thats second hand information from a Sun developer who thought it was a terrible idea.
I've actually thought about something like that, and mostly given up hope. Eventually, every worthwhile tech discussion site becomes flooded with noise.
The problem is, when a site becomes popular among a group of people who aren't morons, the second-rate hacks find out about it and start going there to fit in, because it's the new "cool" place to be. So, for a few months, there will be intelligent conversation, and then all the tards move in, and you eventually end up with crap. It's happened to Slashdot, Digg, Reddit, and most of the others.
That's not realistic for most people or companies.
For people it is twofold. The first is not knowing even how to go about it. You really think a non-technical user knows how to go about having software or features written? Hardly. They wouldn't know who to hire. The second problem is one of money. I'd love to have a nice OSS audio/video editor written. Basically I'd like to have most of Sony's Vegas and some of Cakewalk's Sonar. That's be just great, I'd even be willing to pay. The problem is I wouldn't be willing to pay enough. I'd be willing and able to pay, at most, the cost of both of those packages. Unfortunately, I know the scale of such a project and I realise you are talking probably 6-7 figures for development assuming you want it done well (which I would). I'm afraid I can't swing that. Also I'd only want to pay on contingency, as in I have ot like the results. I can't afford to pony up my cash for something that might not work out, not when there are alternatives out there that I know work.
As such it is extremely unfeasible to expect people to pay for development of OSS. They haven't the knowledge or the money to do so.
So what about business? Well there the money is available, if it needs to be (at least for large enough businesses) and the knowledge should be as businesses should be able to research things they need. However there you run in to lack of reason. So let's say I need a package like this for my company. I decide to have one made and pay a lot of cash for it. Why should I let any of you have it then? Either it is something that helps me do business better, in which case I should keep it to myself to get an edge over competitors, or it is something that's marketable, so I should sell it. In either case, having spent the money and thus getting to dictate the terms, there's no real reason to open it to the world.
That's the problem. It is just unreasonable to expect people to want to pay for things like this especially since one of the big talking points I see for free software is how it's cheaper.
You just shouldn't tell people to "DYI" or "Buy DYI." Those that can and want to will. I mean I've yet to meet someone that said "What, you mean because I have the source I can use my programming skills to modify it? Shit I never knew." Those that want to, do, hence we have the OSS we do. However, if you want it as the mainstream, dominant stuff, those that don't have to be respected.
I don't want to write a video editor, I can't I don't have the skill and I'm not interested in spending the time. However I want a good one. I'm willing to pay only as much as I have to (or I suppose I should say my employer is) which is about $500 in this case. If that isn't sufficient funds, and if you aren't will to have them ex post facto, then I'm afraid the commercial solution is more attractive.
That's just how it goes with most people. You tell them you'll give them a perfect Office replacement for free, they are interested. You tell them they can have Office for free if they are willing to put in thousands of hours of work modifying OO code to make it that way, or paying big dollars to a team to do it, they will just go buy Office.
However you then lose the right to bitch if someone DOESN'T choose your chosen product. I don't at all have a problem with the people who see Linux as the wild west of sorts, where it's a "Do what you want, do it yourself," kind of world. The problem I have are with the people who think that but then except that everyone should come play and be happy doing it. These are the people that will try to get me to switch to Linux, and then when presented with the things I need that I can't have that get angry with me and tell me "Well then write that yourself!" No, screw you, I've other things to do with my time even if I was a programmer (which I'm not).
All I'm saying is if you want Linux to really be a Windows killer, if you want it in every home, then part of what you have to do is drop the DIY attitude. That doesn't mean doing everything anyone asks for, it means two things:
1) Listening to what people want, and making an effort to give it to them. Worry less about what you want, more about what they want. Doesn't mean do whatever someone says, just consider what they say, try and meet their needs as best you can.
2) Not ever telling someone to DIY. Don't have the attitude and give off the impression that they should do it themselves. Because what they probably will do themselves is use something else.
So in this case I'm not saying that this should be done for Intel, however what shouldn't be done is just telling Intel to do it. It should be taken under consideration. If Intel wants to do it, they'll do it, if not someone else should consider it. Considering doesn't mean committing, just consider. If you decide not, that's fine. However saying "DIY" is just abrasive. It says you don't care at all and aren't going to listen.
People's (and companies who are made of people) feelings are important, though some geeks don't seem to wish to acknowledge that. Telling someone to do it themselves is often something that they will take as an affront and a dismissal.
> And this, ladies and gentleman, is why all Linux users are perceived as elitist, arrogant bastards.
You are right. The way free software works is that if you want something done, you either do it yourself, or you pay somebody to do it for you.
Somehow, this has become at odds with mainstream society. People have come to expect that if you want something done, you whine about it for long enough, and someone else will do it for you.
It is a society of builders and whiners.
I know which side I'm on.
You won't be the first to try. You'll might end up about like Technocrat, with most stories getting only a handful of comments.
P.S. I'll take the overs on the 'out by the end of October' part.
most talented technical marketing professional = highly paid fanboy (or girl)
How would you be marketing a stuff you are not a fan of?
And don't tell me Apple doesn't pay well its most talented technical professionals!
Patents Drive Free Software as Hurricanes Drive Construction Industry
He's not talking about binding. Or at least not only. He's talking about the OS scheduling processes and the application (i.e. user space) scheduling the threads within them. Not a new idea either, but then again he has not said the idea was new, he has only said he would want people to use it.
Linux user since early January 1992.
Clearly this is proof of the dominance of Linux, and how Windows Vista is going to fail!
People read reddit for the stories.
People read slashdot for comments.
People read digg for amusing pictures and pop culture things.
Editors are people too.Unlike userdriven selection of stories,it depends on one man choosing whatever on his mind at the time.
If I had one wish that I could wish this holiday season, it would be for all the children of the world to join hands and sing together in the spirit of harmony and peace.
If I had two wishes that I could wish for this holiday season, the first would be for all the children of the world to join hands and sing in the spirit of harmony and peace.. and the second would be for $30 million a month to be given to me, tax-free in a Swiss bank account.
You know, if I had three wishes that I could make this holiday season, first, of course, would be for all the children to get together and sing . . . the second would be for the $30 million every month to me . . . and the third would be for all encompassing power over every living being thing in the entire universe.
And if I had four wishes that I could make this holiday season, first would be the crap about the kids . . . second would be for the $30 million . . . the third would be for all the power.. and the fourth would be to set aside one month each year for an extended 31-day orgasm, to be brought about slowly by Rosanna Arquette and that model Paulina somebody, I can't think of her name, of course my lovely wife could come, too. She's behind me 100% on this, I guarantee you.
Wait a minute, maybe that sex thing should be the first wish! So, if I made that the first wish, because, you know, it could all go boom tomorrow, and then what have you got? No, no . . . the kids singing would be great, that would be nice. No, no, who am I kidding! I mean, they're not gonna be able to get all those kids together! I mean, the logistics of the thing is impossible! It's more trouble than it's worth!
So, we reorganize: here we go. First, the sex - we go with that; second, the money. No! We go with the power second, then the money, and then the kids. Oh, wait, oh geez! I forgot about revenge against my enemies! Okay . . . revenge against all my enemies, they should die like pigs in Hell! That would be the fourth wish! And of course, my fifth wish would be for all the children of the world to join hands and sing in the spirit of peace and harmony.
Thank you, everybody.
Build a man a fire, he's warm for one night. Set him on fire, and he's warm for the rest of his life.
doesn't this pretty much describe windows fibers?
DirectX retained mode is a scene graph API,
Glide is a low-level API that is equivalent to Direct3D Immediate mode,
not retained mode. You can't compare Retained mode vs your
own scene graph and Retained Mode, which is made for people who do not have a scene graph already.
Solaris geek time. Processor sets are a bit brute force, unless you have a very specialised workload. Placing distinct application sets into distinct projects, and then allocating those projects cpu shares for the fair shares scheduler to work with is a more general solution. Sometimes you do it to lump aspects of the same workload together, so that two workloads can co-exist reasonably. Sometimes you do it to keep aspects of the same workload together, so that one part can't swamp another. We did some judicious tuning of an old E450 that provides Cyrus mail for ~600 people, and found that keeping imapd and lmtpd in distinct projects, with nscd and bind in a third, allowed load averages of 150+ (no typo) and 95% CPU utilisation to nonetheless deliver decent response.
Exactly the point and apparently, according to the poster, the Microsoft drone kept pretending that :
Thus proving that market drones are hopeless.
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
Not many, but in numa, there is benefit for allocations and execution occur on the same memory 'node'. Of course, the OS scheduler should know what to deal with this...
XML is like violence. If it doesn't solve the problem, use more.
New here?
Read my blog.
If I could get ONE wish fulfilled
1. Download Ubuntu
2. Visit the KernelCustomBuild page on the Ubuntu wiki.
3. Write your dream scheduler.
I believe there are two more steps that involve uncertainty and money, but I can't remember what they are now.
Your wish has been granted.
Where I typed "inter" please read "intra". Quite important difference. Stupid me...
Linux user since early January 1992.