Slashdot Mirror
New Languages Vs. Old For Parallel Programming
joabj writes "Getting the most from multicore processors is becoming an increasingly difficult task for programmers. DARPA has commissioned a number of new programming languages, notably X10 and Chapel, written especially for developing programs that can be run across multiple processors, though others see them as too much of a departure to ever gain widespread usage among coders."
A lot of problems are I/O driven -- I would like to see more database client libraries allow a full async approach that lets us not block the threads we are trying to do concurrent work on.
How blinkered are you?
There exist whole classes of software that have been doing parallel execution, be it through threads, processes or messaging, for decades.
Look at any/all server software, for god's sake, look at apache, or any database, or any transaction engine.
If you're talking about desktop apps then make it clear. The thing with most of those is that the machines far exceed their requirements with a single core, most of the time. But stuff like video encoding has been threaded for a while too.
True enough, but the class of applications for which parallel processing is useful is growing rapidly as programmers learn to think in those terms. Any program with a "for" or "while" loop in which the results of one iteration do not depend on the results of the previous iteration, as well as a fair number of such loops in which the results do have such a dependency, is a candidate for parallelization -- and that means most of the programs which most programmers will ever write. We just need the languages not to make coding this way too painful.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
Erlang is an older established language designed for parallel processing.
Erlang was first developed in 1986, making it about a decade older than Java or Ruby. It is younger than Perl or C, and just a tad older than Python. It is a mature language with a large support community, especially in industrial applications. It is time tested and proven.
It is also Open source and offers many options for commercial support.
Before anyone at DARPA thinks that they can design a better language for concurrent parallel programming then I think they should be forced to spend 1 year learning Ada, and a second year working in Ada. If they survive they will most likely be cured of the thought that the Defense department can design good programming languages
It's not creating threads that's hard - it's getting them to communicate with each other, without ever getting into a situation where thread a is waiting for thread b and thread b is waiting for thread a that's hard.
Check out Clojure. The only programming language around that really addresses the issue of programming in a multi-core environment. It's also quite a sweet language besides that.
One. Newline characters are for wimps.
Most computing intensive problems that a user will encounter at home are easily parallelizable, i.e. video encoding, gaming, photoshop filters, webbrowsing and so on. The amount of times where I maxed out a single CPU and the given problem would not have been to some large degree parallelizable are close to zero.
The trouble is that they are only "easy" parallelizable in concept, implementing parallelization on an exciting serial codebase is where it gets messy.
I've been very disappointed in parallel programming support. The C/C++ community has a major blind spot in this area - they think parallelism is an operating system feature, not a language issue. As a result, C and C++ provide no assistance in keeping track of what locks what. Hence race conditions. In Java, the problem was at least thought about, but "synchronized" didn't work out as well as expected. Microsoft Research people have done some good work in this area, and some of it made it into C#, but they have too much legacy to deal with.
At the OS level, in most operating systems, the message passing primitives suck. The usual approach in the UNIX/Linux world is to put marshalling on top of byte streams on top of sockets. Stuff like XML and CORBA, with huge overhead. The situation sucks so bad that people think JSON is a step forward.
What you usually want is a subroutine call; what the OS usually gives you is an I/O operation. There are better and faster message passing primitives (see MsgSend/MsgReceive in QNX), but they've never achieved any traction in the UNIX/Linux world. Nobody uses System V IPC, a mediocre idea from the 1980s. For that matter, there are still applications being written using lock files.
Erlang is one of the few parallel languages actually used to implement large industrial applications.
And how many of those cores are above 2% utilization for 90% of the day? Parallelization on the desktop is a solution is search of a problem- we have in a single core dozens of times what the average user needs. My email, web browsing, word processor, etc aren't cpu limited. They're network limited, and after that they're user limited (a human can only read so many slashdot stories a minute). There's no point in anything other than servers having 4 or 8 cores. But if Intel doesn't fool people into thinking they need new computers their revenue will go down, so 16 core desktops next year it is.
I still have more fans than freaks. WTF is wrong with you people?
I have not read the article (par for the course here) but I think there is probably some confusion among the commenters regarding the difference between multi-threading programs and parallel algorithms. Database servers, asynchronous I/O, background tasks and web servers are all examples of multi-threaded applications, where each thread can run independently of every other thread with locks protecting access to shared objects. This is different from (and probably simpler than) parallel programs. Map-reduce is a great example of a parallel distributed algorithm, but it is only one parallel computing model: Multiple Instruction / Multiple Data (MIMD). Single Instruction / Multiple Data (SIMD) algorithms implemented on super-computers like Cray (more of a vector machine, but it's close enough to SIMD) and MasPar systems require different and far more complex algorithms. In addition, purpose-built supercomputers may have additional restrictions on their memory accesses, such as whether multiple CPUs can concurrently read or write from memory.
Of course, the Cray and Maspar systems are purpose-built machines, and, much like special-build processors have fallen in performance to general purpose CPUs, Cray and Maspar systems have fallen into disuse and virtual obscurity; therefore, one might argue that SIMD-type systems and their associated algorithms should be discounted. But, there is a large class of problems -- particularly sorting algorithms -- well suited to SIMD algorithms, so perhaps we shouldn't be so quick to dismiss them.
There is a book called An Introduction to Parallel Algorithms by Joseph JaJa (http://www.amazon.com/Introduction-Parallel-Algorithms-Joseph-JaJa/dp/0201548569) that shows some of the complexities of developing truly parallel algorithms.
(Disclaimer: I own a copy of that book but otherwise have no financial interests in it.)
Sorry but, IMO, the cell is a perfect example of how not to design a multicore processor. Heterogenous processors introduce nothing new to the table of solutions that was not already there. We had systems with CPUs and GPUs before the Cell (or Intel's Larrabee and AMD's Fusion) showed up. Everybody knows that they're a pain in the ass to program. Neither CUDA nor OpenCL nor Microsoft's much ballyhooed TBB (threaded building blocks) will change that fact.
My point is that one does not design a parallel processor and then come up with a programming model to exploit it. It should be the other way around. The programming model should come first. One should design a model that makes parallel programming easy and the resulting apps rock-solid. Only then, after you have perfected your model, should you even consider designing a processor to support the model.
IOW, everybody's doing it wrong, and by everybody, I mean the all big players in the multicore hardware/software industry: Intel, Microsoft, IBM, Sun-Oracle, AMD, ARM, Apple, FreeScale, etc. The major computer science centers who are getting a lot of research money form the industry are not helping either since they have to kowtow to the likes of Intel and AMD whose main interest is to safeguard their installed base and preserve continuity.
It makes no difference. When the pain becomes unbearable (it's all about money), it will suddenly dawn on everybody that what is needed is to break away from the past.
Whoever told you that is mistaken.
The easiest way to take advantage of a multiprocessing environment is to use techniques that will be familiar to any high level programmer. For example, you don't write for loops, you call functions written in a low level language to do things like that for you. Those low level functions can be easily parallelized, giving all your code a boost.