Slashdot Mirror
Choice Overload In Parallel Programming
scott3778 writes to recommend a post by Timothy Mattson over at Intel's Research Blog. He argues, convincingly, that the most important paper for programming language designers to read today is one written by two social psychology professors in 2000. This is the well-known academic study, "When Choice is Demotivating: Can One Desire too Much of a Good Thing?" "And then we show them the parallel programming environments they can work with: MPI, OpenMP, Ct, HPF, TBB, Erlang, Shmemm, Portals, ZPL, BSP, CHARM++, Cilk, Co-array Fortran, PVM, Pthreads, windows threads, Tstreams, GA, Java, UPC, Titanium, Parlog, NESL,Split-C... and the list goes on and on. If we aren't careful, the result could very well be a 'choice overload' experience with software vendors running away in frustration."
Quoth the blogger: "With hundreds of languages and API's out there, is anyone really dumb enough to think "yet another one" will fix our parallel programming problems?"
Yet Intel touts its Threading Building Blocks library as just such a fix to many parallel programming problems. Now, TBB is a very nice product, and in many ways it is superior to a lot of existing libraries, APIs, and languages, but one gets the sense that maybe the left hand doesn't know what the right hand is doing at Intel.
I might also draw an analogy to the open source world, where there are often dozens of solutions to both simple/mundane problems (text editors, media players, command line shells, etc) and more complex ones (window managers, Linux distributions, etc). I wonder if the free and open source software world wouldn't also benefit from a "culling of the herd," so to speak.
While many of these technologies overlap, I think that each serves a niche that is important. I think it would be very difficult to design a language that is easy to use which combines the whole superset of what all of them cover. IBM is attempting to try and create a langugae that covers a lot of it with its experimental X10 languagehttp://domino.research.ibm.com/comm/research_projects.nsf/pages/x10.index.html. While it works, it can be very painful to program in. In the case of parallel and concurrent programming, I think choice is good.
It's designed for shared memory boxes. This is great if you own an E25K, not so great if you've chained together a couple thousand itaniums.
the more accurate the calculations became, the more the concepts tended to vanish into thin air. R. S. Mulliken
I've been gradually trying to learn more about functional programming, partly because I think fp techniques and ways of thinking come in handy even if you're programming in a procedurally oriented language, and partly because fp seems like a paradigm that is likely to get more and more useful as we get machines with more and more cores. Okay, fp!=parallel, but, e.g., one of the big selling points of Erlang is supposed to be that it lends itself to completely transparent use of parallel processors.
The choice overload does seem like kind of an issue to me. For as long as I continue to keep programming comfortably in the procedural languages I'm comfy with (e.g., perl), I'm never going to really wrap my mind around the radically different ways of thinking that you get in a more fp world. I'm been thinking for a long time that it would be fun to do a coding project in ocaml ... or haskell ... or lisp ... or erlang ... or -- you get the idea.
The trouble is, it's really not clear what to hitch my wagon to. Ocaml seems to have a very high quality implementation, but its garbage collector isn't multithreaded, the only book you can buy is in French (it's nice that you can download the English version for free, but I'd prefer to buy something bound), and the availability of libraries (and documentation for them) isn't quite as wonderful as I've gotten used to with perl. Lisp could be cool, but I hate the fact that it's not standardized, and I'm not convinced that eschewing arbitrary syntax really carries more pros than cons. Haskell? Maybe, but it sounds like putting on a hair shirt. The list goes on. I really feel like a deer in the headlights.
Find free books.
Just use RubyMPI when I release it next week :)
The power of MPI wrapped in the beauty of Ruby.
http://www.public.iastate.edu/~crb002/
bash-2.04$
bash-2.04$yes "Don't you hate dialup connections?"| write USERNAME
Someone told me once there was a study where people also are more likely to choose something out of a group if it is clearly superior to another item in the group.
:-)
I found this true for myself once, I was looking to buy a DVD and had the usual overload of choices. I noticed that there was a copy of "LXG movie only" for $30, but also a copy of "LXG with special features" for $20. This triggered me buying the LXG with special features. In hindsight, it was the fact that I "knew" I was getting a "bargain" that tipped me over the edge. No doubt it was a deliberate marketing ploy.
A bit sexist to say this, but women seem to be especially targetted by "discount marketing" of this sort. Mainly with shoes
is not an excess of choice, is an excess of improvisation.
Long story short... now that hardware speed is not easily doubled every few years, the industry has found a 'simple' way to keep pushing the weel, duplicate cores!. Well, it turns out that after decades of ignoring the parallel programming demand from academics now they are trying to push the 'somewath parallel' mess thay are producing.
The problem is 'duplicated cores' != 'parallel programming', that's the problem.
What's in a sig?
No, a red fuzzy sphere is NOTHING like a 5-dimensional hypersphere. Even a sphere of 4 spatial dimensions would blow your fucking mind. If we lived in a universe with 4 dimensions of space, then planets would be 4D spheres with a 3D surface. You would be able to walk around, and "turn" up and down without ever leaving the ground, just like you can turn left and right in our universe. It would be impossible to tie a knot. Imagine, if you will, people living on the surface of a circular planet in a 2D universe, then imagine how incredibly different it would be from our 3D universe. If you aren't utterly shocked by thinking about what it would be like to live in 4 spatial dimensions, let alone 5, then you haven't been thinking hard enough.
I agree and disagree here. The way I see it, we'll always need human intelligence to explicitly annotate the source to indicate where nondeterministic behavior is permissible (or rather, where it's NOT allowed). But on the other hand I think compilers of the future will all use runtime analysis to refine JIT compilation, and they'll be able to spot very obscure parallelism opportunities that might only arise due to the actual data used in the evaluation of the program.
I say we go with Fortran, because it supports array programming natively without any of this bolt-on stuff like OpenMP.
Escher was the first MC and Giger invented the HR department.
I always find it amusing (in a sad kind of way) how people talk about Herb Sutter's "call to action" over this. It's not that I've got anything against Herb himself: he's a decent writer, an excellent speaker, and a guy who can use the word "expert" legitimately in areas like C++. But it's also not like he's the first guy to notice that modern desktop computer architectures have been heading for parallelisation rather than increased speed for several years now.
Despite being right in the thick of this culture shift myself — I'm sure I'm not the only one here who has been talking about this for a while, and is just seeing management catch up — I don't think this is going to be that big of a deal for most people. The harsh reality, for the buzzword-wielding consultants rubbing their hands with glee at a new programming approach they can hype up, is that most people just don't need all this.
Your average desktop PC is more than powerful enough for most things that most people do with it: Internet communications, writing documents, working with databases, shop floor software, and the like. As long as the operating system is reasonably smart about scheduling, the guys writing these common types of applications don't really have to know anything about multithreading, locking, message passing, and all that jazz. Similarly, your average mobile device has more than enough juice to dial another phone, write a quick e-mail, or capture a digital photo.
At the other end of the spectrum, serious servers (database, communications, whatever) have been dealing with parallel processing of many requests since forever. High-end systems doing serious maths (the guys modelling weather systems, say) have also been using massive parallelisation on their supercomputers for zillions of years now.
There is a gap between these different areas, which we might traditionally have called the "workstation" market: the guys doing moderate number crunching for CAD, scientific visualisation, simulations, and the like. Many modern games also fall into this classification. This market is ripe for a parallel processing revolution, because historically it hasn't followed this approach very much because the hardware wouldn't really take advantage of it, yet the extra power is genuinely useful. But I don't think this represents some huge proportion of the software development industry as a whole. The guys working in these areas tend to be pretty smart, and will no doubt adopt useful practices and conventions fairly quickly now that the hardware has reached the point that they are useful.
As to what those conventions are, I just don't buy the whole "choice overload" theory. There are relatively few basic models for parallel processing: for example, you can have no shared state and communicate only through message passing, or you can have shared state. In the latter case, you then have the question of how to make sure that the sharing is safe, which leads to lock-based or lock-free approaches. Funky toys like transactional memory run at a slightly higher level than this, but they are ultimately constructed from the same building blocks, and again there are only a small number of approaches at this level to consider.
I'm not familiar with all of those libraries mentioned in the story, but I'll bet that those three classifications (no shared state, shared state with explicit locking, shared state without explicit locks) probably cover the models used by most if not all of them. If you understand the trade-offs in those, you can produce a sensible design, and then the toolkit or framework you use to code it up is mostly just an implementation detail. Given that the trade-offs are pretty obvious and will often steer projects clearly in one direction, I don't think there's really that much to choose at all.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.