Slashdot Mirror
More Interest In Parallel Programming Outside the US?
simoniker writes "In a new weblog post on Dobbs Code Talk, Intel's James Reinders discusses the growth of concurrency in programming, suggesting that '...programming for multi-core is catching the imagination of programmers more in Japan, China, Russia, and India than in Europe and the United States.' He also comments: 'We see a significantly HIGHER interest in jumping on a parallelism from programmers with under 15 years experience, verses programmers with more than 15 years.' Any anecdotal evidence for or against from this community?"
Instead of addressing the root problems with concurrency, we are just going see super-high-level languages that have no bearing or relationship to the actual hardware the underlying machine.
Which is more efficacious? To take up as much simultaneous processor time as possible in order to finish faster, or to leave extra cores open for other processes to run simultaneously.
Given that the management of threads isn't exactly the easiest thing in the world (not the hardest either, mind you), perhaps it would be more beneficial to let the OS determine which threads to run rather than trying to optimize a single app.
For example, many webservers use a semaphore rather than multiple threads to handle request dispatch. The result is that there is less overhead creating and cleaning up threads.
One reason could be that software engineers with more experience simply already know about these things, and have faced off against the many problems with concurrency. Threads can be hell to deal with for instance. So because of things they don't show any interest.
That being said I think that if you want to actually make use of many cores you really do have to switch to a language that can give you usage of many threads for free. Writing it manually usually ends up with complications. I find Erlang to be pretty nifty when it comes to these things for instance.
Whether it's a good idea or not, you will have a VERY hard time convincing an old dog programmer that he should jump on the train. Think back to the time when relational models entered the database world.
Ok, few here are old enough to be able to. But take object oriented programming. I'm fairly sure a few will remember the pre-OO days. "What is that good for?" was the most neutral question you might have heard. "Bunch o' bloated bollocks for kids that can't code cleanly" is maybe more like the average comment from an old programmer.
Well, just like in those two cases, what I see is times when it's useful and times when you're better off with the old ways. If you NEED all the processing power a machine can offer you, in time critical applications that need as much horsepower as you can muster (i.e. games), you will pretty much have to parallelize as much as you can. Although until we have rock solid compilers that can make use of multiple cores without causing unwanted side effects (and we're far from that), you might want to stay with serial computing for tasks that needn't be finished DAMN RIGHT NOW, but have to be DAMN RIGHT, no matter what.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Young bucks jump on the latest thing without thinking (or the experience to back their thoughts) of whether or not its the best way to go.
The experienced programmers know that most parallelisable problems are already being solved by breaking it across machines, and the rest won't be helped by 15 bazillion cores. An extra core or so on a desktop is nice, beyond that they really won't be anywhere near the speedup its hyped.
I still have more fans than freaks. WTF is wrong with you people?
The free lunch (in performance) used to get wasted on slack programmers ignoring performance. Faced with not slacking off further or mountain of bugs from multithreading or learning some new functional language made by academia, the average programmer will choose not slacking off.
Spoken like a completely ignorant child. How the hell do you think (if you even can) we older guys got into this business? We were tinkering with new things, using them when appropriate, before many of you were born, and the joy of new ideas hasn't worn off. The only difference is we don't do it quite so impulsively, just because it seems new.
For one thing, multiple homogeneous cores is NOT new (hetero- either, for that matter), just fitting them into the same die. I've used quad 68040 systems, where, due to the ability of the CPUs to exchange data between their copy-back caches, some frequently-used data items were NEVER written to memory, and on System V you could couple processing as tightly or loosely as you wanted. There are some problem sets that take advantage of in-"job" multi-processing better than others, just as some problem sets will take of advantage of multiple cores by doing completely different tasks simultaneously. Simple (very) example: copying all of the files between volumes (not a block-for-block clone); if I have two cores, I can can either have a multi-threaded equivalent of "cp" which walks the directory tree of the source and dispatches the create/copy jobs in parallel, each core dropping into the kernel as needed, or I can start a "cpio -o" on one core and pipe it to a "cpio -i" on the other, with a decent block size on the pipe. More cores means more dispatch threads in the first case, and background horsepower handling the low-level disk I/O in the other. In my experience, the "cpio" case works better than the multi-threaded "cp" (due, AFAICT, to the locks on the destination directories).
As you say atomic pointer swaps are often not an option. And if you have to rule out mutex locks for timing or some other reason, I think you may be down to functional programming. If you eliminate objects with state and just pass everything by value (make copies on the calling thread to hand to the called), it can solve the problem, but it can cost a load in memory access times. Wiki the buzzphrase "referential transparency"
refactor the law, its bloated, confusing and unmaintainable.
The experienced programmers know that most parallelisable problems are already being solved by breaking it across machines, and the rest won't be helped by 15 bazillion cores. An extra core or so on a desktop is nice, beyond that they really won't be anywhere near the speedup its hyped. Mod this guy. Short and to the point.
I worked extensively with parallel programming since the early 90's. There is no silver bullet. Most problems do not parallelize to large scales. There are always special problems that DO parallelize well, like image and video processing. So, if you are a watching 20 video streams, your Intel Infinicore (TM) chip will be worth the $$$.
"No matter where you go, there you are." -- Buckaroo Banzai
It all depends from where you stand. If your bread and butter is business applications, then indeed it's rare to having to deal with threads. But for almost everything else, threading is a necessity. I am a programmer in the field of defense applications, and I never ever worked in a single-threaded application. And if you look around, almost all applications have some sort of parallelism (examples: Word when re-paginating long documents, Firefox with multiple tabs, Excel running long computations, Torrent clients downloading multiple torrents, etc).
I work in parallel programming too.
Most problems do not parallelize to large scales.
I'm getting tired of this nonsense being propagated. Almost all real world problems parallelize just fine, and to a scale sufficient to solve the problem with linear speedup. It's only when people look at a narrow class of toy problems and artificial restrictions that parallelism "doesn't apply". e.g. Look at google; it searches the entire web in milliseconds using a large array of boxes. Even machine instructions are being processed in parallel these days (out of order execution etc.).
Name a single real world problem that doesn't parallelize. I've asked this question on slashdot on several occasions and I've never received a positive reply. Real world problems like search, FEA, neural nets, compilation, database queries and weather simulation all parallelize well. Problems like orbital mechanics don't parallelize as easily but then they don't need parallelism to achieve bounded answers in faster than real time.
Note: I'm not talking about some problems being intrinsically hard (NP complete etc.), many programmers seem to conflate "problem is hard" with "problem cannot be parallelized". Some mediocre programmers also seem to regard parallel programming as voodoo and are oblivious to the fact that they are typically programming a box with dozens of processors in it (keyboard, disk, graphics, printer, monitor etc.). Some mediocre programmers also claim that because a serial programming language cannot be automatically parallelized that means parallelism is hard. Until we can program in a natural language that just means they're not using a parallel programming language appropriate for their target.
---
Advertising pays for nothing. Who do you think pays marketer's salaries? You do via higher cost products.
Parallel programming is simply harder than typical sequential programming. Not only does the design take more time and effort, but the debugging is VERY much harder. tools for parallel programming are poor but debugging tools are basically pathetic. Worse, today's project and development methodologies don't focus on getting something up and hacking, not on careful upfront design that is needed to really parallelize things. We get most of our parallelism from the web server being multi-threaded and the database handling concurrency.
As many have said, large scale parallel systems are not new. Just because we need a solution to the problem doesn't mean that it will appear any time soon. Some problems are very difficult and involve not only new technologies and programing models but major re-educational efforts. There are many topics in physics and mathematics that only a small number of people have the intellectual skill and predilection to handle. Of all the college graduates, what percent complete calculus, let alone take advanced calculus? Pretty small number.
My prediction is that the broad base of programmers will have the tools and be able to do some basic parallelism. A small number of programmers will do the heavy duty parallel programming and this will be focus on very high value problems.
BTW, this Intel guy, while addressing a real issue, seemed to be doing marketing for his toolkit/approach. Sounds like a guy trying to secure his budget and grow it for next year.
There are two problems: thread sychronization, and race conditions.
Race conditions
A modern CPU architecture uses multiple levels of cache, which aggrivate the race condition scenario. For a programmer to code multi-threaded code, and "not-worry", then the architecture must always read and write every value to memory. This worst case scenario is only needed in a tiny fraction of cases. So the compiler can do much better by working with the memory model of the architecture, instead of assuming that no memory model is in place.
Any significant improvement in speed will require solving the race condition problem. I believe research into transactional memory may be the way to go - but it is still half the speed of normal memory access.
The synchronization problem
This requires that the programmer reason about multiple threads (at least two). It doesn't really matter what buildling blocks you're using, you simply must be aware that two pieces of code are effectively executing at the same time - or at least their cpu slices are interwoven. This type of reasoning significantly raises the bar for writing bug-free code, because a whole new class of subtle problems arise. The mind must wrap itself around something that is simply more complex, and we have trouble enough with single threaded programs.
I like the flex/javascript solution. Just one thread - with asynchronous-like behaviour through events. It means there are no race conditions, and you can get enough async behaviour to get the job done. I hope, in the future, that both these technologies allow you to create a seperate "process" (thread but with different memory context) that executes asychronously and returns the results on the main event queue. It's a bit clunky, but at the same time it's much more idiot proof. And I'm speaking as someone who's done a reasonable amount of parallel programming.
Like all pain, suffering is a signal that something isn't right
Sorry - we are entering the "Model T" mass production era of software. First, no one said parallelism is easy (in this thread anyways). I don't care that most "programmers" are mediocre and will never be able to understand parallelism, much like I don't care that most native English speaking Americans will never be able to understand nor speak Mandarin Chinese. They have about the same relevance to parallel programming as we're not talking about the masses being able to do parallel programming, speak Mandarin, or make Model Ts, for that matter.
Speaking of your "Model T" mass production comment, I'd quite disagree, I'd say we've entered a split environment of Yugo production (that'd be your mediocre programmers) and the perhaps 10% of the programmers that can actually code and are capable of understanding higher level concepts. I'm being gracious with that 10%, my personal experience has shown that fewer than 3% of programmers are actually capable of coherent coding. That most likely has to do with how programmers are taught or the lack thereof, but that's another discussion entirely.
The cesspool just got a check and balance.
Wow, you totally missed the point of what I was saying didn't you. Fine-grained / coarse grained does matter in real life.
If a problem doesn't exhibit fine-grained parallelism then running multiple copies is the *best* you can do. In some situations that is enough (i.e. a large project with lots of separate compilation units). In some situations it isn't enough, i.e. where you can't split your compilation into separate units because you're trying to run global optimisations across the whole lot.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php