Multicore Requires OS Rework, Windows Expert Says

alphadogg writes "With chip makers continuing to increase the number of cores they include on each new generation of their processors, perhaps it's time to rethink the basic architecture of today's operating systems, suggested Dave Probert, a kernel architect within the Windows core operating systems division at Microsoft. The current approach to harnessing the power of multicore processors is complicated and not entirely successful, he argued. The key may not be in throwing more energy into refining techniques such as parallel programming, but rather rethinking the basic abstractions that make up the operating systems model. Today's computers don't get enough performance out of their multicore chips, Probert said. 'Why should you ever, with all this parallel hardware, ever be waiting for your computer?' he asked. Probert made his presentation at the University of Illinois at Urbana-Champaign's Universal Parallel Computing Research Center."

Comments

This is new?!

[DavidRawling]

Oh please, this has been coming for years now. Why has it taken so long for the OS designers to get with the program? We've had multi-CPU servers for literally decades.

Re:This is new?!

[PhunkySchtuff]

Since when have OS designers optimised their code to milk every cycle from the available CPUs? They haven't, they just wait for hardware to get faster to keep up with the code.

Re:This is new?!

[Jeremi]

Why has it taken so long for the OS designers to get with the program?

Coming up with a new OS paradigm is hard, but doable.

Coming up with a viable new OS that uses that paradigm is much harder; because even once the new OS is working perfectly, you still have to somehow make it compatible with the zillions of existing applications that people depend on. If you can't do that, your shiny new OS will be viewed as an interesting experiment for the propeller-head set, but it won't ever get the critical mass of users necessary to build up its own application base.

So far, I think Apple has had the most successful transition strategy: Come up with the great new OS, bundle the old OS with it, inside an emulator/sandbox, and after a few years, quietly deprecate (and then drop) the old OS. Repeat as necessary.

Re:This is new?!

[Cryacin]

For that matter, since when have software vendors been willing to pay architects/designers/engineers etc to optimise their software to milk every cycle from the available CPUs and provide useful output with the minimum of effort? They don't, they just wait for hardware to get faster to keep up with code.

The only company that I have personally been exposed to that gives half a hoot about efficient performance is Google. It annoys me beyond belief that other companies think it's acceptable to make the user wait for minutes whilst the system recalculates data derived from a large data set, and doing those calculations multiple times just because a binding gets invoked.

Re:This is new?!

[jc42]

Since when have OS designers optimised their code to milk every cycle from the available CPUs?

This isn't just an OS-level problem. It's a failure among programmers of all sorts.

I've been involved in software development since the late 1970s, and for the start I've heard the argument "We don't have to worry about code speed or size, because today's machines are so fast and have so much memory. This was just as common back when machines were 1,000 times slower and had 10,000 times less memory than today.

It's the reason for Henry Petroski's famous remark that "The most amazing achievement of the computer software industry is its continuing cancellation of the steady and staggering gains made by the computer hardware industry."

Programmers respond to faster cpu speed and more memory by making their software use more cpu cycles and more memory. They always have, and there's no sign that this is going to change. Being efficient is hard, and you don't get rewarded for it, because managers can't measure it. So it's better to add flashy eye candy and more features, which people can see.

If we want efficient code, we have to figure out ways to reward the programmers that write it. I don't see any sign that people anywhere are interested in doing this. Anyone have suggestions for how it might be done?

Re:This is new?!

[fuzzyfuzzyfungus]

I doubt that it's just google. I suspect the following:

There are(in broad strokes, and excluding the embedded market), two basic axes on which you have to place a company or a company's software offering in order to predict its attitude with respect to efficiency.

One is problem scale. If a program is a once-off, or an obscure niche thing, or just isn't expected to have to cope with very large data sets, putting a lot of effort into making it efficient will likely not be a priority. If the program is extremely widely distributed, or is expected to cope with massive datasets, efficiency is much more likely to be considered important(if widely distributed, cost of efficient engineering per unit falls dramatically, if expeced to cope with massive datasets, amount of hardware cost and energy cost avoided becomes significant. Tuning a process that eats 50% of a desktop CPU into one that eats 40% probably isn't worth it. Tuning a process that runs on 50,000 servers into one that runs on 40,000 easily could be).

The second is location: If a company is running their software on their own hardware, and selling access to whatever service it provides(search engine, webmail, whatever), their software's efficiency or inefficiency imposes a direct cost on them. Their customers are paying so much per mailbox, or so much per search query, they have an incentive to use as little computer power as possible to deliver that product. If a company is selling boxed software, to be run on customer machines, their efficiency incentives are indirect. This doesn't mean "nonexistent"(a game that only runs on $2,000 enthusiast boxes is going to lose money, nobody would release such a thing. Among enthusiasts, browser JS benchmarks are a point of contention); but it generally does mean "secondary to other considerations". Customers, as a rule, are more likely to use slow software with the features they want, or slow software that released first and they became accustomed to, than fast software that is missing features or requires substantial adjustment on their part. Shockingly enough, software developers act on this fact.

On these axes, you would strongly suspect that Google would be efficiency oriented. Their software runs on a grand scale, and most of it runs on their own servers, with the rest competing against various desktop incumbents, or not actually all that dramatically efficient(Nothing wrong with Google Earth or Sketchup; but nothing especially heroic, either). However, you would expect roughly the same of any entity similarly placed on those axes.

Re:This is new?!

[Brian Gordon]

Maybe it's not a question of whether the code is efficient. Maybe it's a question of how much you're asking the code to do. It's no surprise that hardware struggles to make gains against performance demands when software developers are adding on nonsense like compositing window managers and sidebar widgets. I'm enjoying Moore's law without any cancellation.. just run a sane environment. Qt or GTK, not both, if youre running an X desktop. Nothing other than IM in the system tray. No "upgrade fever" that makes people itch for Windows Media Player 14 when older versions work fine and mplayer and winamp work better.

Re:This is new?!

[Mr. Freeman]

Because Google ain't crunching data sets on fucking mobile phones. They're optimizing their servers and the applications that run on those servers because Google is so damn big that a fraction of a percent increase in efficiency translates into huge amounts of money saved through less wasted CPU time. Mobile phones aren't a part of google.

If you phone runs a little less efficient then no one gives a damn. They want to make their phones easy to program for, which generally conflicts with efficiency.

Re:This is new?!

[jc42]

Hey, if you liked programming for a one-byte machine, maybe you should join the quantum computer research effort. They're just now looking forward to the creation of their first 8-bit "computer" in the very near future. ;-)

Of course, you can do a bit more computing with 8 Q-bits than you can with 8 of the more mundane bits that the rest of us are using.

Re:This is new?!

[IamTheRealMike]

Why Java for Android? This is a good question. There are several reasons (that the Android team have discussed).

One is that ARM native code is bigger, size-wise, than Dalvik VM bytecode. So it takes up more memory. Unlike the iPhone, Android was designed from the start to multi-task between lots of different (user installed) apps. It's quite feasible to rapidly switch between apps with no delay on Android, and that means keeping multiple running programs in RAM simultaneously. So trading off some CPU time for memory is potentially a good design. Now that said, Java has some design issues that make it more profligate with heap memory than it maybe needs to be (eg utf16 for strings) so I don't have a good feel for whether the savings are cancelled out or not, but it's a justification given by the Android team.

Another is that Java is dramatically easier to program than a C-like language. I mean, incredibly monstrously easier. One problem with languages like C++ or Objective-C is that lots of people think they understand them but very few programmers really do. Case in point - I have an Apple-mad friend who ironically programs C# servers on Windows for his day job. But he figured he'd learn iPad development. I warned him that unmanaged development was a PITA but he wasn't convinced, so I showed him a page that discussed reference counting in ObjC (retain/release). He read it and said "well that seems simple enough" - doh. Another one bites the dust. I walked him through cycle leaks, ref leaks on error paths (no smart pointers in objc!), and some basic thread safety issues. By the end he realized that what looked simple really wasn't at all.

By going with Java, Android devs skip that pain. I'm fluent in C++ and Java, and have used both regularly in the past year. Java is reliably easier to write correct code in. I don't think it's unreasonable to base your OS on it. Microsoft has moved a lot of Windows development to .NET over the last few years for the same reasons.

Fortunately, being based on Java doesn't mean Android is inherently inefficient. Large parts of the runtime are written in C++, and you can write parts of your own app in native code too (eg for 3D graphics). You need to use Java to use most of the OS APIs but you really shouldn't be experiencing perf problems with things like gui layout - if you are, that's a hint you need to simplify your app rather than try to micro-optimize.

waiting

[mirix]

'Why should you ever, with all this parallel hardware, ever be waiting for your computer?'

Because I/O is always going to be slow.

Re:waiting

[DavidRawling]

Well, with the rise of the SSD, that's no longer as much of a problem. Case in point - I built a system on the weekend with a 40GB Intel SSD. Pretty much the cheapest "known-good" SSD I could get my hands on (ie TRIM support, good controller) at AUD $172, roughly the price of a 1.5TB spinning rust store - and the system only needs 22GB including apps.

Windows boots from end of POST in about 5 seconds. 5 seconds is not even enough for the TV to turn on (it's a Media Center box). Logon is instant. App start is nigh-on instant (I've never seen Explorer appear seemingly before the Win+E key is released). This is the fastest box I've ever seen, and it's the most basic "value" processor Intel offer - the i3-530, on a cheap Asrock board with cheap RAM (true, there's a slightly cheaper "bargain basement" CPU in the G6950 or something). The whole PC cost AUD800 from a reputable supplier, and I could have bought for $650 if I'd wanted to wait in line for an hour or get abused at the cheaper places.

Now, Intel are aiming to saturate SATA-3 (600MBps) with the next generation(s) of SSD, or so I'm told. Based on what I've seen - it's achievable, at reasonable cost, and it's not only true for sequential read access. So if the IO bottleneck disappears - because the SSD can do 30K, 50K, 100K IO operations per second? Yeah, I think it's reasonable to ask why we wait for the computer.

Not that I think a redesign is necessary for the current architectures - Windows, BSD, Linux all scale nicely to at least 8 or 16 logical CPUs in the server world, so the 4, 6 or 8 on the desktop isn't a huge problem. But in 5 years when we have 32 CPUs on the desktop? Maybe. Or maybe we'll just be using the same apps that only need 1 CPU most of the time, and using the other 20 CPUs for real-time stuff (Real voice control? Motion control and recognition?)

The problem isnt even that simple

[indrora]

The problem is that most (if not all) peripheral hardware is not parallel in many senses. Hardware in today's computers is serial: You access one device, then another, then another. There are some cases (such as a few good emulators) which use muti-threaded emulation (sound in one thread, graphics in another) but fundamentally the biggest performance kill is the final IRQs that get called to process data. The structure of modern day computers must change to take advantage of multicore systems.

Grand Central?

[volfreak]

Isn't this the reason for Apple to have rolled out GrandCentral in Snow Leopard? If so, it seems it's not THAT hard to do - at least not that hard for a non-Windows OS.

Re:Grand Central?

[jasmusic]

I'm thinking you don't have much experience with .NET. During my projects it has always run comparable to native compiled code when I write my code with the mindset of a C++ programmer and not a VB one.

Re:Because

[Anonymous Coward]

Why should you ever, with all this parallel hardware, ever be waiting for your computer?' he asked.

Because it might be waiting for I/O.

That's no reason for the entire GUI to freeze on Windows when you insert a CD.

Re:Current architecture flawed but workable BUT...

[Threni]

Windows explorer sucks. It always just abandons copies after a fail - even if you're moving thousands of files over a network. Yes, you're left wondering which files did/didn't make it. It's actually easier to sometimes copy all the files you want to shift locally, then move the copy, so that you can resume after a fail. It's laughable you have to do this, however.

But it's not a concurrency issue, and neither, really, are the first 2 problems you mention. They're also down to Windows Explorer sucking.

Re:I hate to say it, but...

[GIL_Dude]

Are you running a 9 year old version of OSX too, or are you comparing a two generation old Windows version to a nice new Mac version? It really sounds like you are comparing apples (snicker) to oranges. After all, both Vista and Windows 7 have no problem running for a long, long time between reboots and don't get slow during that time.

Re:Current architecture flawed but workable BUT...

[Kenz0r]

I wish I could mod you higher than +5, you just summed up some of the things that bother me most about the OS that is somehow still the most popular desktop OS in the world.

To anyone using Windows (XP, Vista or 7) right now, go ahead and open up an Explorer window, and type in ftp:// followed by any url.
Even when it's a name that obviously won't resolve, or an ip of your very own local network of a machine that just doesn't exist, this'll hang your Explorer window for a couple of solid seconds. If you're a truly patient person, try doing that with a name that does resolve, like ftp://microsoft.com . Better yet, try stopping it.... say goodbye to your explorer.exe .

This is one of the worst user experiences possible, all for a mundane task like using ftp. And this has been present in Windows for what, a decade?

Re:Luckily OSX is Already Has MultiCore Tech

[shutdown -p now]

The trick with GCD is that it is somewhat more high-level than a simple thread pool - it operates in terms of tasks, not threads. The difference is that tasks have explicit dependencies on other tasks - this lets scheduler be smarter about allocating cores.

Microsoft's slowness and Windows 2005

[gig]

I love how Microsoft can come along in 2010 and with a straight face say it's about time they took multiprocessing seriously. Or say it's about time we started putting HTML5 features into our browser. And we're finally going to support the ISO audio video standard from 2002. And by the way, it's about time we let you know that our answer to the 2007 iPhone will be shipping in 2011. And look how great it is that we just got 10% of our platform modernized off the 2001 XP version! And our office suite is just about ready to discover that the World Wide Web exists. It's like they are in a time warp.

I know they have product managers instead of product designers, and so have to crib design from the rest of the industry, necessitating them to be years behind, but on engineering stuff like multiprocessing, you expect them to at least have read the memo from Intel in 2005 about single cores not scaling and how the future was going to be 128 core chips before you know it.

I guess when you recognize that Windows Vista was really Windows 2003 and Windows 7 is really Windows 2005 then it makes some sense. It really is time for them to start taking multiprocessing seriously.

I am so glad I stopped using their products in 1999.

It's not even about multiple cores

[macraig]

What's wrong with at least some operating systems doesn't even have anything to do with multiple cores per se. They're simply designing the OS and its UI incorrectly, assigning the wrong priorities to events. No event should EVER supersede the ability of a user to interact and intercede with the operating system (and applications). Nothing should EVER happen to prevent a user being able to move the mouse, access the start menu, etc., yet this still happens in both Windows and Linux distributions. That's a fucked-up set of priorities, when the user sitting in front of the damned box - who probably paid for it - gets second billing when it comes to CPU cycles.

It doesn't matter if there's one CPU core or a hundred. It's the fundamental design priorities that are screwed up. Hell should freeze over before a user is denied the ability to interact, intercede, or override, regardless how many cores are present. Apparently hell has already frozen over and I just didn't get the memo?

Re:Duh

[keeboo]

Why should you ever, with all this parallel hardware, ever be waiting for your computer?'

For a lot of problems, for the same reason that some guy who just married 8 brides will still have to wait for his baby.

Of course, he'll be able to get 8 babies at once, assuming none of the processes crash during the computation.

That improves bandwidth, but not latency: almost 1 baby/month, but 9 months of latency.
The guy could try interleaving the pregnancies, in order to get the illusion of lower latency.