More Interest In Parallel Programming Outside the US?

simoniker writes "In a new weblog post on Dobbs Code Talk, Intel's James Reinders discusses the growth of concurrency in programming, suggesting that '...programming for multi-core is catching the imagination of programmers more in Japan, China, Russia, and India than in Europe and the United States.' He also comments: 'We see a significantly HIGHER interest in jumping on a parallelism from programmers with under 15 years experience, verses programmers with more than 15 years.' Any anecdotal evidence for or against from this community?"

Comments

Duh?

[gustolove]

Old programmers don't want to learn new things -- trust the tried and true.

Young bucks want to be on the cutting edge to get the jobs that the old people already have.

----

Oh, and the people see the benefit in the other countries more than those in the U.S.? Probably not, we're just lazy American's though.

Re:Duh?

[foobsr]

the latest thing

1960: E. V. Yevreinov at the Institute of Mathematics in Novosibirsk (IMN) begins work on tightly-coupled, coarse-grain parallel architectures with programmable interconnects. ( c.f. )

An extra core or so on a desktop is nice, beyond that they really won't be anywhere near the speedup its hyped.

And of course any virtual reality scenario will not profit from extra power.

CC.

Re:Duh?

[tedgyz]

I should have said: Most problems do not EASILY parallelize to large scales.

In regards to your comments about mediocre programmers...

You do not recognize the fact that most programmers are mediocre. You can scream at them that it is easy, but they will still end up staring at you like deer in the headlights.

Sorry - we are entering the "Model T" mass production era of software.

Re:Duh?

[smallfries]

What do you mean by "parallelize well"? Normally people would use that phrase to imply fine-grained parallelism in the problem but I suspect that you are using it differently. For example when you say that compilation parallelizes well are you talking about coarse-grain parallelism across compilation-units? This is not really a single instance of a problem being computed in parallel, but rather many independent instances of a problem. If you are willing to use many independent instances of a problem then the coarse-grain parallelism will always scale "well" - i.e linearly in the number of cores. But this does not provide a speedup in many real-world applications.

In your compilation example, it is easy to get speedups at the compilation level using something like make -jN. But this assumes that each unit is independent. If you want to apply advanced global optimisations then this is not always the case, and then you hit the harder problem of parallelizing the compilation algorithms rather than just running multiple instances. It's not impossible but I'm not aware of any commercial compilers that do this.

Re:Duh?

[CastrTroy]

Most people who do anything are mediocre. Otherwise, mediocre would be redefined. It's like saying, half the people in the class scored below average. The fact that half the people scored below some value determines what the value of average is.

Re:Duh?

[JAlexoi]

Most people who do anything are mediocre. Otherwise, mediocre would be redefined. It's like saying, half the people in the class scored below average. The fact that half the people scored below some value determines what the value of average is. Do you have any idea what is statistics? Because you post suggests that you don't.
So in a set 0,10,10,10,10 average is 8, but surprisingly enough only 1 is below average!

Re:Duh?

[Inoshiro]

"And of course any virtual reality scenario will not profit from extra power."

It's more a matter of what kind of speedup you see, and what algorithm you start with.

If your algorithm is serial, and there is no parallelism to speed it up, you're not going to see any speed increase. This applies to a lot of things, such as access to the player-character status variables (all cores use the same memory address, so they need a mutex or other synchronization anyway). Any AI-NPC interactions would likely need to be coordinated. You could divide the large stuff across some collection of cores (physics core, AI core, book-keeping core, input core), but not all of those will break down further (so an 80 core machine is a limited speedup over a 4 core machine -- which is, itself, a limited speedup over a basic 2-core machine for most people).

The easy things to breakup are embarrassingly parallel problems (like certain graphics activities), and they are being broken up across GPUs already. Algorithms, even if they are entirely easy to parallelize, are still linear. To be 10 times faster, you need 10 processors (this is why GPUs have a lot of simple graphics pipelines and shader processors -- they're throwing more hardware at the problem). If a problem is simply too big (you need to be 1000 times faster, or exponential and beyond algorithms), more hardware isn't going to help.

People seem to think that parallel programming will make the impossible into something possible. That's not true. Proving NP = P and coming up with efficient algorithms for certain things will. More hardware won't -- it'll just make things that were O(n) or O(log n) [for a sufficiently large n, like the number of pixels in a 1920x1200 monitor] possible. It won't affect O(n^2) and above.

Re:Duh?

[Anonymous Coward]

The problem not that. Actually O(n) is the same as O(n/p)

The problem is that K*O(n) does not become K/p*O(n) but can decrease much more slowly. 10x the cores doesn't give you 10x the performance increase. Adding a core doesn't help at all if you depend on intermediate results in your calculation.

For sorting you can get very close, but many problems are harder (if you can't partition the processing easily).

Re:Duh?

[RandCraw]

Examples of problems that do not parallelize:

1) Problems that contain little data

2) Problems that require sequential processing

3) Problems that are often interrupted, that cannot predict future actions or execution paths (e.g. pipelined)

What fraction of computing tasks match these 3 constraints? At least 90% of the work done on desktops.

Speeding up the other 10% will be done by specialized hardware like multicore video or DSP chips. The real potential for parallelism in everyday computing is negligible. I've been parallel programming for years, and all forms of parallelism are next to useless unless, like Google, you have lots of data.

BTW, Google has no use for multicores either. All of their parallelism is embarassingly parallel, which is better served by shared-nothing architectures like many, many, many cluster nodes that are cheap, cheap, cheap.

        Randy

Re:Duh?

[smallfries]

I'm fairly sure that you have missed the point.

We both agree about how parallelism impacts jobs in the real world. We also agree that if we can speed a job up then we don't care how it is achieved. The point that I made that you keep skipping over is that when there is no fine-grained parallelism available the key question becomes do I care about speeding up multiple copies of a job, or do I need a single job to run faster.

The OP's choice of compilation was odd, which is what I remarked on. If I'm compiling lots of things in one project then it will go faster. But compilation itself is not easy to parallelise, and as I pointed out commercial compilers don't currently speed up single compilation units over multiple cores.

Your false dichotomy is that a single unit must be simple (hello world) and anything more complex can be split into multiple units. Not only is this not true - but I gave a concrete example right at the beginning of where this would be a problem. This would be the point that has whistled over your head on multiple occasions.

When the compiler needs to perform global analysis (ie if you are doing aggressive global optimisations across the whole program - not local optimisations within a single unit) there is no obvious way to speed it up. There may well be fine-grained parallelism in there but nobody has exploited it yet. There is no coarse-grain parallelism because you only have a single task - compile the whole program.

Applying these sort of aggressive global optimisations has been the focus of the compiler community for decades. Now that multicores are becoming common it will be another interesting problem to parallelise.

throwing more CPUs at the problem

[Anonymous Coward]

On the surface, it seems that Japanese engineers have a history of this. Fujitsu had their dual-6809 home computer, arcade games commonly had two and sometimes three Z80s, 680x0, or whatever. Sega, in their mad rush to beat the specs of the upcoming Playstation, stuffed four off-the-shelf processors plus a few custom chips into the Saturn.

Parallel tools are still pretty weak

[Goalie_Ca]

I'm writing a scientific library that is supposed to scale for many many cores. Using a mutex lock is not an option. Unfortunately right now I am spending all my time trying to figure out how to get compare and swap working on all the different platforms. I am saddened to see the lack of support since this is such a fundamental operation. Also, the whole 32 vs 64-bit thing adds more pain because of pointer size.

Re:Oversimplified

[Nursie]

"E.g., let's say we write a massively multi-threaded shooter game. Each player is a separate thread,"

Well there's your first mistake.
That's a recipe for disaster and built in limits to the number of players.

Ideally you seperate up your server app into multiple discrete jobs and process them with a thread pool. I don't know how well that maps to gaming but many server apps work very well with that paradigm.

Re:Oversimplified

[Coryoth]

E.g., let's say we write a massively multi-threaded shooter game....Debugging it gets even funnier, since some race conditions can happen once a year on one computer configuration, but every 5 minutes on some hapless user's. Most will not even happen while you're single-stepping through the program Well that just says that you're doing it wrong. Sure, a massively concurrent system done in the manner you describe would be incredibly tricky to make work correctly. Of course that's not the only way to do it. With a little bit of thought and analysis going in you can write massively concurrent multi player games that just work right first time. That's a system that had literally thousands of concurrent processes all interacting, with no deadlock or livelock, and no complex debugging to ensure that was the case. Just because you can't imagine how it could be done doesn't mean it can't be.