The fact that writing C and Fortran code using a message passing library constitutes a high level programming environment is a complete indictment of the sad state of parallel programming today.
Seriously, do you want to be programming complex parallel algorithms on HPC machines using Soviet Era technology? I've tried that and it made me want to jump out a window. It's about as easy to program in this type of an environment as it is to program an FPGA (hint: it's a pain in the ass).
The entire reason why CUDA works and is powerful is exactly because it is limited. Nvidia knows that there is no silver bullet. They're not claiming that this is one (David Kirk has said so himself at conferences). CUDA is a fairly elegant way of mapping embarrassingly data parallel programs to a large array of single precision FP units. If your problem fits into the model, the performance you get via CUDA will smoke just about anything else (except maybe an FPGA in some scenarios).
Your notion about particular models making some parts of parallel programming easy while other parts are hard is what people really need to learn to accept about parallel programming. If you're expecting a single model to make everything easy for you, trust me, stop programming right now.
You need to pick the programming model that matches the parallelism in your application- there will never be one solution. When sitting down to write code, you have to ask yourself: what is the right model for this algorithm? Is it:
Data parallel (SIMD, Vector)
Message Passing
Actors
Dataflow
Transactional
Streaming (pipe and filter)
Sparse Graph
Etc...
There are many models out there, and many languages + hardware substrates for these models that will give you orders of magnitude speedup for parallel programs. They key is to just to sit down, think about the problem, and pick the right one (or combinations).
The real research focus in parallel programming should be to make a taxonomy of models and start coming up with a unified infrastructure to support intelligent selection of models, mixing and matching, and compilation.
One of my buddies is on one of these teams (which will remain nameless). According to him, one of the major problems last year was the enormous amount of time spent on sensor and hardware development vs. the incredibly small amount of time spent on software development and testing. When you have an autonomous vehicle with great realtime terrain mapping capabilities, but with no collision detection code (!), you get a hunk of metal with great vision that likes to run into the same bush over and over again. Other teams did have collision detection in, which would back the vehicle up when a potential collision was detected. However, the vehicle would pick up too many potential collisions (lots of bushes and rocks in the desert) and continously back up! Just goes to show you that developing the algorithms for these things is pretty important. Let's hope they get it right this year... although the chances are pretty slim.
I was up all night in the computer lab at school writing a program, and wanted to delete all the.o files manually. But instead of typing "rm -rf *.o" I did the ol' "rm -rf *" mistake. However, since rm was aliased to "rm -i", I had to confirm every file I deleted. In my stupor I said yes to delete every.c,.h,.o and anything else in the directory. It was at that point that I decided to take up drinking coffee.
Slashdot Mirror
The fact that writing C and Fortran code using a message passing library constitutes a high level programming environment is a complete indictment of the sad state of parallel programming today. Seriously, do you want to be programming complex parallel algorithms on HPC machines using Soviet Era technology? I've tried that and it made me want to jump out a window. It's about as easy to program in this type of an environment as it is to program an FPGA (hint: it's a pain in the ass).
The entire reason why CUDA works and is powerful is exactly because it is limited. Nvidia knows that there is no silver bullet. They're not claiming that this is one (David Kirk has said so himself at conferences). CUDA is a fairly elegant way of mapping embarrassingly data parallel programs to a large array of single precision FP units. If your problem fits into the model, the performance you get via CUDA will smoke just about anything else (except maybe an FPGA in some scenarios).
Your notion about particular models making some parts of parallel programming easy while other parts are hard is what people really need to learn to accept about parallel programming. If you're expecting a single model to make everything easy for you, trust me, stop programming right now.
You need to pick the programming model that matches the parallelism in your application- there will never be one solution. When sitting down to write code, you have to ask yourself: what is the right model for this algorithm? Is it:
Data parallel (SIMD, Vector)
Message Passing
Actors
Dataflow
Transactional
Streaming (pipe and filter)
Sparse Graph
Etc...
There are many models out there, and many languages + hardware substrates for these models that will give you orders of magnitude speedup for parallel programs. They key is to just to sit down, think about the problem, and pick the right one (or combinations).
The real research focus in parallel programming should be to make a taxonomy of models and start coming up with a unified infrastructure to support intelligent selection of models, mixing and matching, and compilation.
This is absolutely correct. As someone in computer science at Caltech, this is *not* surprising.
One of my buddies is on one of these teams (which will remain nameless). According to him, one of the major problems last year was the enormous amount of time spent on sensor and hardware development vs. the incredibly small amount of time spent on software development and testing. When you have an autonomous vehicle with great realtime terrain mapping capabilities, but with no collision detection code (!), you get a hunk of metal with great vision that likes to run into the same bush over and over again. Other teams did have collision detection in, which would back the vehicle up when a potential collision was detected. However, the vehicle would pick up too many potential collisions (lots of bushes and rocks in the desert) and continously back up! Just goes to show you that developing the algorithms for these things is pretty important. Let's hope they get it right this year... although the chances are pretty slim.
I was up all night in the computer lab at school writing a program, and wanted to delete all the .o files manually. But instead of typing "rm -rf *.o" I did the ol' "rm -rf *" mistake. However, since rm was aliased to "rm -i", I had to confirm every file I deleted. In my stupor I said yes to delete every .c, .h, .o and anything else in the directory. It was at that point that I decided to take up drinking coffee.