More Interest In Parallel Programming Outside the US?

simoniker writes "In a new weblog post on Dobbs Code Talk, Intel's James Reinders discusses the growth of concurrency in programming, suggesting that '...programming for multi-core is catching the imagination of programmers more in Japan, China, Russia, and India than in Europe and the United States.' He also comments: 'We see a significantly HIGHER interest in jumping on a parallelism from programmers with under 15 years experience, verses programmers with more than 15 years.' Any anecdotal evidence for or against from this community?"

Comments

What are the applications?

[BadAnalogyGuy]

Which is more efficacious? To take up as much simultaneous processor time as possible in order to finish faster, or to leave extra cores open for other processes to run simultaneously.

Given that the management of threads isn't exactly the easiest thing in the world (not the hardest either, mind you), perhaps it would be more beneficial to let the OS determine which threads to run rather than trying to optimize a single app.

For example, many webservers use a semaphore rather than multiple threads to handle request dispatch. The result is that there is less overhead creating and cleaning up threads.

Re:What are the applications?

[SanityInAnarchy]

It depends on the application. Some applications simply benefit from running in realtime. And some applications don't really scale well by breaking them up into individual processes. Some applications want to use as much CPU as you can throw at them -- a web app, for instance, had better be able to handle another few (dozen?) app servers if you get Slashdotted.

Also: Management of threads is mostly hard because we're mostly still using such low-level tools to do it. Semaphores, locks, and threads are the equivalent of GOTOs, labels, and pointers. While you can build a higher-level system (message-passing, map/reduce, coroutines) out of semaphores, locks, and threads, I'd argue that's like writing everything in GOTOs instead of doing a proper for loop -- or using a for loop instead of a proper Ruby-style "each" iterator. At the risk of abusing my analogy, yes, sometimes you do want to get under the hood (pointers, threads), but in general, it's much safer, saner, and not that much less efficient to use modern parallel programming concepts.

There's still some real problems with pretty much any threaded system, but I suspect that a lot of the bad rap threads get is from insufficient threading tools.

Oh, and about that webserver -- using more than one process is pretty much like using more than one thread, and I'd say weighs about the same in this discussion. Webservers, so far, are able to get away with using one thread (or process, doesn't matter) per request (and queuing up requests if there's too many), so that's a bit less of a problem than, say, raytracing, compiling, encoding/decoding video, etc.

Re:What are the applications?

[BadAnalogyGuy]

Regarding the webserver, I am arguing the opposite, though. The single-threaded, event-driven architecture allows the server itself to remain relatively lightweight. Some webservers certainly do use threads and optimize that further by using a threadpool, but threads are still more cumbersome for an application like that which has mandatory blocking while the hardware is busy doing stuff (tx or rx).

The rest of your post, I am in agreement. I used to work on a very large high profile application framework. I heard all the time how the framework was huge and bloated and slow. But what I found 9 times out of 10 was that when an application which eschewed the framework grew to a large enough size, most of the core logic of 'my' application framework was duplicated in the resulting application. In other words, they were re-engineering stuff that had already been solved and were worse off because their code was less tested and less optimized than the existing framework.

In the same way, we need a good "framework" (maybe a language, maybe a library, maybe a new paradigm) which takes advantage of and makes beautifully clear the paradigm. I don't know if it's necessary to hide the parallelization from programmers (Erlang) or to expose just the bare minimum (fork(), etc). There needs to be a middle ground which can be taken advantage of whether the programmer knows he is using it or not, but also is readily available for the times when he absolutely needs the capability.

Experince

[forgoil]

One reason could be that software engineers with more experience simply already know about these things, and have faced off against the many problems with concurrency. Threads can be hell to deal with for instance. So because of things they don't show any interest.

That being said I think that if you want to actually make use of many cores you really do have to switch to a language that can give you usage of many threads for free. Writing it manually usually ends up with complications. I find Erlang to be pretty nifty when it comes to these things for instance.

Old dogs and new tricks

[Opportunist]

Whether it's a good idea or not, you will have a VERY hard time convincing an old dog programmer that he should jump on the train. Think back to the time when relational models entered the database world.

Ok, few here are old enough to be able to. But take object oriented programming. I'm fairly sure a few will remember the pre-OO days. "What is that good for?" was the most neutral question you might have heard. "Bunch o' bloated bollocks for kids that can't code cleanly" is maybe more like the average comment from an old programmer.

Well, just like in those two cases, what I see is times when it's useful and times when you're better off with the old ways. If you NEED all the processing power a machine can offer you, in time critical applications that need as much horsepower as you can muster (i.e. games), you will pretty much have to parallelize as much as you can. Although until we have rock solid compilers that can make use of multiple cores without causing unwanted side effects (and we're far from that), you might want to stay with serial computing for tasks that needn't be finished DAMN RIGHT NOW, but have to be DAMN RIGHT, no matter what.

Re:Duh?

[AuMatar]

Young bucks jump on the latest thing without thinking (or the experience to back their thoughts) of whether or not its the best way to go.

The experienced programmers know that most parallelisable problems are already being solved by breaking it across machines, and the rest won't be helped by 15 bazillion cores. An extra core or so on a desktop is nice, beyond that they really won't be anywhere near the speedup its hyped.

Re:Duh?

[dltaylor]

Spoken like a completely ignorant child. How the hell do you think (if you even can) we older guys got into this business? We were tinkering with new things, using them when appropriate, before many of you were born, and the joy of new ideas hasn't worn off. The only difference is we don't do it quite so impulsively, just because it seems new.

For one thing, multiple homogeneous cores is NOT new (hetero- either, for that matter), just fitting them into the same die. I've used quad 68040 systems, where, due to the ability of the CPUs to exchange data between their copy-back caches, some frequently-used data items were NEVER written to memory, and on System V you could couple processing as tightly or loosely as you wanted. There are some problem sets that take advantage of in-"job" multi-processing better than others, just as some problem sets will take of advantage of multiple cores by doing completely different tasks simultaneously. Simple (very) example: copying all of the files between volumes (not a block-for-block clone); if I have two cores, I can can either have a multi-threaded equivalent of "cp" which walks the directory tree of the source and dispatches the create/copy jobs in parallel, each core dropping into the kernel as needed, or I can start a "cpio -o" on one core and pipe it to a "cpio -i" on the other, with a decent block size on the pipe. More cores means more dispatch threads in the first case, and background horsepower handling the low-level disk I/O in the other. In my experience, the "cpio" case works better than the multi-threaded "cp" (due, AFAICT, to the locks on the destination directories).

Re:Duh?

[bit01]

I work in parallel programming too.

Most problems do not parallelize to large scales.

I'm getting tired of this nonsense being propagated. Almost all real world problems parallelize just fine, and to a scale sufficient to solve the problem with linear speedup. It's only when people look at a narrow class of toy problems and artificial restrictions that parallelism "doesn't apply". e.g. Look at google; it searches the entire web in milliseconds using a large array of boxes. Even machine instructions are being processed in parallel these days (out of order execution etc.).

Name a single real world problem that doesn't parallelize. I've asked this question on slashdot on several occasions and I've never received a positive reply. Real world problems like search, FEA, neural nets, compilation, database queries and weather simulation all parallelize well. Problems like orbital mechanics don't parallelize as easily but then they don't need parallelism to achieve bounded answers in faster than real time.

Note: I'm not talking about some problems being intrinsically hard (NP complete etc.), many programmers seem to conflate "problem is hard" with "problem cannot be parallelized". Some mediocre programmers also seem to regard parallel programming as voodoo and are oblivious to the fact that they are typically programming a box with dozens of processors in it (keyboard, disk, graphics, printer, monitor etc.). Some mediocre programmers also claim that because a serial programming language cannot be automatically parallelized that means parallelism is hard. Until we can program in a natural language that just means they're not using a parallel programming language appropriate for their target.

---

Advertising pays for nothing. Who do you think pays marketer's salaries? You do via higher cost products.

Re:Duh?

[Gr8Apes]

I should have said: Most problems do not EASILY parallelize to large scales. ... You do not recognize the fact that most programmers are mediocre. You can scream at them that it is easy, but they will still end up staring at you like deer in the headlights.

Sorry - we are entering the "Model T" mass production era of software. First, no one said parallelism is easy (in this thread anyways). I don't care that most "programmers" are mediocre and will never be able to understand parallelism, much like I don't care that most native English speaking Americans will never be able to understand nor speak Mandarin Chinese. They have about the same relevance to parallel programming as we're not talking about the masses being able to do parallel programming, speak Mandarin, or make Model Ts, for that matter.

Speaking of your "Model T" mass production comment, I'd quite disagree, I'd say we've entered a split environment of Yugo production (that'd be your mediocre programmers) and the perhaps 10% of the programmers that can actually code and are capable of understanding higher level concepts. I'm being gracious with that 10%, my personal experience has shown that fewer than 3% of programmers are actually capable of coherent coding. That most likely has to do with how programmers are taught or the lack thereof, but that's another discussion entirely.