Multi-Threaded Programming Without the Pain

holden karau writes "Gigahertz are out and cores are in. Programmers must begin to develop applications that take full advantage of the increasing number of cores present in modern computers. However, multi-threaded development has been notoriously hard to do. Researcher Stefanus Du Toit discusses and demonstrates RapidMind, a software system he co-authored, that takes the pain out of multi-threaded programming in C++. For his demo he created a program on the PlayStation 3 representing thousands of chickens, each independently tracked by a single processing core. The talk itself is interesting but the demo is golden."

Comments

Which Comes First?

[jforest1]

The multi-threaded chicken or the multi-threaded egg?

--josh

Re:Which Comes First?

[zoefff]

nope, the post

Re:Which Comes First?

[Kjella]

Since the egg thread is always converted to a chicken thread while the chicken thread spawns new subthreads, the egg came first. QED.

Re:Which Comes First?

[cronot]

the fork()

Re:Which Comes First?

[Timesprout]

Since the egg thread is always converted to a chicken thread

Someone forgot about the omlette thread.

Deadlocked!

[Trimbo2]

Deadlock detected!

Huh?

[dreamchaser]

I didn't know the PS3 had thousands of cores ;)

I think what he meant was 'each tracked in a separate thread'...obviously each core is still handling many threads. I haven't watched the presentation and don't plan on it until later today, too much to do and I'd rather read something about it. It just sounds like it provides an efficient high level way to write a multi threaded app. Evolutionary but not revolutionary?

Re:Huh?

[Gr8Apes]

Even so, this is a "bad" implementation. There's absolutely no reason for there to be 1 thread per chicken. That's inherently not scalable. What you really want are an optimum number of threads for the number of cores in a pool that handle work units (chickens). This will scale much higher than the 1 thread per object model discussed in this topic.

Oh, and there's no such thing as "easy" multi-threading. Hell, the average programmer can't even grasp OO, so what makes them think they can grasp threading which has many many more aspects to it?

Re:Huh?

[dreamchaser]

Having written more than my share of threaded apps I agree 100%. I still haven't looked into this more, but it's probably a C++ class library that abstracts the creation and management of threads. Too many threads thrashes the processor nicely in many cases, so unless they have some magic behind the scenes managing the number of threads vs. cores then this is just a hyped up multi threading library.

Fsck the chickens...show me what this does with a real game or a real world app that lends itself to highly parallel operations, then demo it on a quad quad core Xenon.

Re:Huh?

[Gr8Apes]

The chicken scenario described removed any curiosity I had about looking into the library further. Why? Because it's very similar to the Java 101 bouncing ball thread demo (one thread per ball) which is used to show why 1 thread per ball doesn't scale to first time would be multi-threaded programmers.

Re:Huh?

[grumbel]

### That's inherently not scalable.

Not scalable? I beg to differ. Thousands threads for sure scale are a lot better then when you just have two or four or whatever, since with thousands you don't really have an upper limit of how many CPU you want to throw at the problem. The real issue with threads is that OS threads are extremely slow, so you can't have thousands threads or your machine would go to a crawl. Threads also are painful to work with since the languages just aren't up to the task.

However for both these issues there exist solutions, namely Erlang, using user-level threads there is no upper limits and you really can have each chicken have its own thread without a problem and the language is also build from the base up to work nicely with threads.

Now I haven't yet seen the talk, bittorrent still busy downloading it, but I seriously doubt that it will just be yet-another-simple-wrapper class.

Re:Huh?

[Gr8Apes]

First, last time I ran the ball test just to see how processors had improved in their capabilities to run code, I got to over 2K threads in a single JVM before significant degradation occurred and then it occurred rapidly.

Using the threadpool concept, however, you can tune the size of the threadpool via performance metrics from the threads in the threadpool for the optimum size of threadpool, after which you can place however many objects on the pool you'd like. Generally, this is based on the work the thread has to do. If there is no I/O blocking, I've found that 2-3 threads per CPU with moderate CPU time work units will load it to 100% (read moderate CPU time work units as work units that take on the order of 100-1000 ms to complete). If you start adding in any type of I/O blocking, including large amounts of memory access, then that number goes up. A DB retriever system wound up running 64 threads for my particular work load due primarily to the lag involved in the synchronous calls made to the DB. I could have tuned that further using future tasks and reducing the number of threads (a Doug Lea addition to the JDK 1.5 and also available in his previous concurrency library) but my particular case didn't have any negative effects by running 64 threads, so we left it at that. This particular DB access module ran across 64 systems (64*64 threads) serving roughly 35K concurrent customers.

I haven't run Erlang, so can't comment. I have heard nice things about it though, and I'm curious about it. One day I'll have enough time to play with it.

Re:Huh?

[gbjbaanb]

Well, given that the chicken demo did scale, aren't you curious as to what they are doing differently? Not using Java? :-)

Re:Huh?

[joto]

Thousands threads for sure scale are a lot better then when you just have two or four or whatever, since with thousands you don't really have an upper limit of how many CPU you want to throw at the problem.

Yes, the upper limit is thousand(s)! Go directly to jail. Do not pass Go. Do not collect $200.

Seriously, with companies already offering 4 cores per CPU, and promising to offer 16 cores in the near future, and Moores law being as it is, you don't exactly have to be a visionary to predict that the future might bring a shitload(TM) of processor cores to somewhere in your vicinity. Note that a shitload(TM) is more than a few measly thousands. Oh, and before you start telling me that nobody needs a shitload(TM) of processor cores, remember that nobody needed more than 640K RAM either.

The real issue with threads is that OS threads are extremely slow, so you can't have thousands threads or your machine would go to a crawl.

OS threads aren't necessarily slow (I assume you mean switching between them). If this is at all true, it is an artificial limitation of current hardware/software combos that can be easily fixed (at least the fix is much easier than the work involved in creating shitload-core CPUs). Note that the cost of OS-threads, user-space threads, and OS processes vary wildly among different systems already. But shared memory really needs to go. It just doesn't scale.

However for both these issues there exist solutions, namely Erlang, using user-level threads

Last time I checked, Erlang used only user-space threads, meaning that even if you had a shitload(TM) of cores, a given Erlang program would only use one of them. Erlang focuses on modelling, not performance. I suspect there to be good ideas in Erlang, but it's not going to be the system programming language of the next century.

Re:Huh?

[Procyon101]

You are using JVM threads. Most massively scalable threaded languages, like Erlang, use green threads. A green thread acts like a thread from the standpoint of the programmer, but carries little or no context switch cost (because it's not really a thread). The underlying platform then load balances these green threads across the actual hardware in an optimal pool of true threads.

What makes these programming languages easy to grasp the massive concurrency of is one of 2 things:

1) In Erlang and Termite (A scheme dialect) there is no mutable state, and no globals. Every function is in essence a "service" that simply gets messages and then responds with replies. There is no need to think about locking in such a system and very easy message passing idioms to do what you would normally do with mutable object orientation.

2) In languages like Haskell, there is no concept of a "thread" at all... not even a single thread. There is no concept of "ordering". Things are defined as they are in mathematics.. as relationships between functions and variables. There is no mutable state allowed. This strictness allows the compiler to make very deep conclusions as to what can be parallelized. The compiler can then load balance under the covers across any number of procs without exposing any issues of concurrency to the user at all.

So yes, in Java (and OO in general), concurrency is very, very difficult. In other paradigms though it can be trivial, or even transparent.

Re:Huh?

[shutdown -p now]

Have a read please.

• Software transactional memory
• Erlang
• Fortress

As it goes, when it comes to multithreading, the model used by C++, Java and similar languages is rapidly becoming outdated.

Re:Huh?

[shutdown -p now]

A C++ library could simply implement Erlang semantics on top of C++.

Bah humbug

[kahei]

Multithreaded development is commonplace in applications that need it. The places it's not common in are:

-- old-style Unix development, because of the 'lightweight process model'. It's a unix-ism that's on the way out but until it disappears we will have some things like Ruby that don't 'get it'.

-- places that have absolutely no need for it, which certainly includes the chicken demo. One core per chicken?? Seems more like the guy just discovered threads but hasn't quite grasped what they're for.

Re:Bah humbug

[ari_j]

You are, of course, correct. The other thing that people need to keep in mind is that there is rarely only a single process running on a given machine. For applications where it makes sense, such as video rendering on a machine doing nothing else, multithreading can increase overall performance. For applications where it doesn't or where there are other things running on the same machine, you normally end up with worse overall performance by trying to get your naturally single-threaded program to run on multiple cores at once when the extra cores would be better dedicated to running things other than your program.

Multithreading is a tool. Just like more traditional tools, like the hammer, this one is useful for certain applications. But multithreading is not the only tool at your disposal - people need to stop looking at everything as if it were a nail.

Re:Bah humbug

[aldheorte]

"old-style Unix development, because of the 'lightweight process model'. It's a unix-ism that's on the way out but until it disappears we will have some things like Ruby that don't 'get it'."

I'm not sure I follow you there. Lightweight process models are perfect for multi-cores. The more the merrier. Given the andundance of high-quality networking and commodity machines, heavyweight programs outside of very niche areas that use internal threads are less suitable for distributed computing than lighteight process models that can call across the network or the OS to other lightweight processes. A heavywight process can only scale to the number of cores avaiable on the machine it is running on, whereas a flock of lightweight processes can scale to the locally available cores and onto to other machines in a distributed fashion without a major bump in the road between local and remote. Any machine that has multi-cores today could easily run, say, one Ruby process per core with negligible overhead.

Re:Bah humbug

[kcbrown]

-- old-style Unix development, because of the 'lightweight process model'. It's a unix-ism that's on the way out but until it disappears we will have some things like Ruby that don't 'get it'.

And it's silly for it to be "on the way out".

Anyone remember the Amiga? It had a preemptive multitasking OS that lacked hardware memory protection because the hardware it was running on couldn't support it. And while the OS itself was very fast and efficient, the overall system was relatively crash-prone, because any memory-related programming error in any running application had a decent chance of taking down the system.

Fast forward to today. Every computer sold has hardware memory protection built-in. Anyone who doesn't know why that's a good thing needs to spend time on an Amiga.

And yet, despite that, threads are all the rage. Why? Because people have this idiotic belief that they're somehow "more efficient" than processes. Such people probably program about as well as they think, which is to say not very well. Threads are indeed more efficient at context switching than processes, but the real question is: does that really matter? In the vast majority of cases, it doesn't, because in the vast majority of cases multiple threads are being used to make the user interface responsive. There's no way a human being can tell the difference between a millisecond-level context switch time and a microsecond-level one.

On top of that, processes bring one critical advantage to the table that threads don't: memory protection. And for the same reason memory protection is important at the OS and hardware level, so too is it important at the process and thread level: it allows clean, protected separation of concern and greater overall application stability.

The vast, vast majority of applications that are multithreaded don't actually need the slight additional context switch performance advantage that threads bring to the table, but they very much need the memory protection facilities that processes bring to the table. Which is another way of saying that if your application needs concurrency, you're a fool if you blindly use threads instead of processes.

Even Windows supports fork() these days, with the POSIX subsystem (available, as far as I know, on any Windows 2000 and later system), so creating a clone of your current process is dirt simple even under Windows. End result: application authors have no good reason to use threads over processes unless they've actually done the math and can prove that their application really needs the slight performance advantage of threads more than the significant reliability advantage of processes.

As to the other reason for using threads, the sharing of memory, there's this really cool new technology out these days. Maybe you've heard of it. It's called "shared memory". It's only been available for 20 years or so. No wonder most people haven't heard of it. Being forced to explicitly declare what's shared and what isn't is a good thing, because it makes you program easier to maintain, easier to debug, and more reliable -- all at the same time.

The bottom line is this: if you need concurrency in your application, you should be using processes, not threads. If you insist on using threads, you'd better have a damned good reason for it, because the reliability implications of threads are hugely negative while the performance implications are modest at best.

Re:Bah humbug

[odourpreventer]

Please correct me if I'm wrong, but it seems to me this discussion has gone into apples and oranges mode. Threads, as far as I'm aware, are supposed to be used for single, explicit tasks and always under supervision by a parent thread. I've used multi-threading with excellent results, but then I've taken pains to ensure that the threads don't have any privileges whatsoever. Processes, on the other hand, are more like stand-alone programs working in the same context.

Re:Bah humbug

[tuskentower]

Huh? Threads and processes are two different animals.
Since you need a reason, here's one, its called concurrency. With processes I have to consume finite system resources to handle concurrency issues or role my own, which is called reinventing the wheel (aka waste of time). Thread libraries will do this for me.

Re:Bah humbug

[gbjbaanb]

Unfortunately you say processes have their own memory protection which is better than threads that have to do their own synchronisation when accessing shared memory, but then go on about process-based shared memory needing its own additional protection.

If you need concurrency in your apps, there isn't that much between threads and processes. However, if you need interprocess-communication then you are far better off with threads, they are significantly faster wrt locking than processes as all process-based locks must be done at the OS level, using shared (and finite) system resources. Threads can just use a critical section and have done with it, almost no overhead.

Threads are not more efficient at context switching than processes, the same procedure happens whether a thread is switched, or a process is (in fact, a process is really an app with 1 thread). However, as threads can share memory more efficiently, locking is often not needed as much so they appear to be more efficient.

The best argument for threads v processes is Apache. Personally, I agree with the Apache group that Apache 2 with its thread-based model is better. They should know.

Re:Bah humbug

[Bluesman]

Nope, still the same. The OS has to flush the TLB when it switches processes, which is the cache for virtual memory address lookups.

This and the reduced startup time are the most compelling reasons to use threads instead of processes on a single core.

However, on a large number of cores, things aren't so clear-cut, since if you have as many cores as active processes, you're not doing the context switching as much, and the benefit of using threading to reduce cache flushes isn't so clear. You'd still benefit from the quick startup of threads, so for things like a highly concurrent web server that creates a thread per user, threads may still be a better solution.

Interestingly, the much maligned cooperative threads (user-space) are the fastest of all since the programmer can control when the context switch happens. However, if there's blocking or an infinite loop, the whole application will hang. You have to use asynchronous I/O and make sure no thread runs for too long.

Like most things, it's a trade off between protection from various mistakes and errors vs. speed and control. Processes give you the most protection with the greatest amount of overhead, while user level threads give you the best performance, but only if you design everything correctly.

Re:Bah humbug

[Foolhardy]

A critical section makes use of a synchronization mechanism like a semaphore at entry and exit time to ensure that only one thread of execution (whether that be a process or a thread) is running in the critical section at any one time.

On Windows at least, a critical section requires no kernel object, and only a few instructions with no syscall to acquire and release as long as there is no contention on the object. If, while entering the section marked as already owned, a kernel notification event is created for waiters to sleep on. A kernel mutex OTOH, always requires a kernel object and a syscall for both acquire and release. Syscalls are quite expensive, making critical sections much faster in most cases. A design involving a large number of small lockable objects with rare contention would benefit from being able to use them in particular. I know that Solaris also has lightweight mutexes that can't be shared between processes, and I assume they avoid syscalls in most cases as well.

Re:Bah humbug

[dkf]

On a single core without hyperthreading, your best bet (if you can) is to write very efficient single-threaded code, using non-blocking I/O as much as possible. Some language runtimes require the use of lots of threads even on single-core systems, but that's horrible.

Once you've got multiple cores, getting multiple threads of execution (either in multiple processes or in multiple threads) makes a lot of sense. I believe hyperthreading benefits particularly such code that has multiple threads executing in the same bit of code since the parallelism there is within a memory management domain, so OpenMP is better there than pthreads, and pthreads is (probably) better than processes. On the other hand, if you're potentially working across a cluster (cue the beowulf jokes!) your code had better be written with processes (and probably MPI) in mind. Of course if you're going that way, you also ought to spend on getting a good interconnect network...

All in all, getting proper high performance is tricky. The best guide to making things go faster is to try to reduce the amount of shared state between threads-of-execution. Reducing shared state also helps to make the code easier to debug. (Alas, dealing with the bits of state that must be shared is what makes life hard.)

RapidMind = vendor lock-in

[Anonymous Coward]

Both, RapidMind and Peakstream are proprietary commercial solutions and those companies are trying to lock users into their particular framework. What we really need is the equivalent as true open-source solution, perhaps as a gcc extension. Does anyone know if there is progress being made on this?

Re:RapidMind = vendor lock-in

[Anonymous Coward]

OpenMP is implemented into GCC 4.2 (I think, I've never used it in GCC).

Re:RapidMind = vendor lock-in

[acidrain]

Does anyone know if there is progress being made on this?

The GPUs will ship with C compilers soon enough. They are already supporting limited forms of C. Actually we will see hybrid CPUs (the cell being a first example) which are capable of massive amounts of parallel math operations stacked in along side some of your CPU cores in time. As the number of cores grows, room is made for specialized processors where that makes sense in the market.

Re:"hundreds of cores"?

[dreamchaser]

If by 'on the horizon' they mean 'possibly in the next ten years', then sure. I can see that happening. Quad cores are already here. If they double the number of cores every 18 months that means in 7.5 years we'll have 128 cores. I'm just throwing that out as an example, but it's certainly possible even if all the cores are not on the same package. Take 8 physical CPU's with 16 cores each for example.

Just rampant speculation, but it is certainly possible.

Functional programming

[Cthefuture]

Also note that certain programming languages can make multithreaded programming a lot easier. Nothing against C++ (one of my favorite languages) but no matter what you do it's relatively hard to use in multithreaded applications compared to a functional language. We are already seeing more functional features put into existing languages.

The main problem I see is that there is lack of focus in the functional arena. Many current functional languages are designed to use a VM with bytecode (Erlang for example) and don't support native threads easily (often requiring multiple VM instances and slow[er] message passing). The languages that do support native compiling almost always have other problems like horrible syntax (O'Caml, Lisp) or just general lack of refinement. Arguably Haskell comes the closest but suffers from a complicated and large backend support requirement like Java.

Without native thread support it's hard to take advantage of multiple processor cores. Too bad we don't see more mature native compiled functional languages out there.

Re:Functional programming

[kcbrown]

Without native thread support it's hard to take advantage of multiple processor cores. Too bad we don't see more mature native compiled functional languages out there.

What?

Sorry, that's bullshit. If you want to take advantage of multiple processor cores, use multiple processes! Even Windows has fork() these days, thanks to its POSIX subsystem, so creating a clone of your process is very easy.

You should use threads over processes only if you can prove that the context switch savings really does cause a big performance improvement for your application. If you really think about it, you'll find it's very rare indeed that the context switch overhead difference really matters, even on an OS like Windows where it's relatively high.

Re:Functional programming

[Lazerf4rt]

I recently shipped an Xbox 360 game and am about to ship a PS3 game, and having done a lot of system-level programming and optimization for both, I can tell you don't know what you're talking about. You're probably a smart guy and a good programmer but you're obviously speaking out of academic experience without having much real-world experience.

The key to performance and stability does not lie in the discovery of high-level tools that abstract away all the hardware details for you. And it definitely doesn't lie in a functional language. They key is knowing your hardware and designing your software for it, right down to the low level. You have to create and manage your tasks/jobs/threads/fibers, each to do a specific thing, and you manage their lifetimes and the flow of data between them. If want need more performance, you need clever ways of pipelining your data.

Anyway, just thought I'd share that. If you make a career in programming, you'll eventually learn that having a low-level understanding of each platform, and just using existing tools, is far more productive than trying to research and develop new programming tools. I'm downloading TFA's video right now, look forward to hearing whatever it is they have to say.

Re:Functional programming

[Communomancer]

The main problem I see is that there is lack of focus in the functional arena.

Whoa whoa whoa! You may not like Erlang's implementation, but you can hardly attribute it to a lack of focus. The whole language was built with concurrency in mind. Heck, the concurrency even has built-in network awareness. And Erlang's been multi-core since last May.

Erlang goes multi-core

Yeah, that doesn't say anything about your VM worries. I don't have those, though. Seamless multi-threading and a language paradigms designed for concurrency more than make up for the VM performance hit, imo. When I have to write non-trivial concurrent systems, I reach for the language that already has the plumbing excellently implemented. I'm sure it's better done than anything I could implement myself, and since the system is concurrent, cheap hardware is easily added to improve performance.

Man, this is the second time this week I've had to stick up for Erlang around here.

Re:Functional programming

[The_Wilschon]

That approach works just fine if you know exactly what hardware your code is going to be running on, and you know that it will never have to run on any other hardware, and you know that you won't have to ever work on it again once it is released.

In the Real World (ie not game consoles), programs must be portable. They must be maintainable. They must be writeable in a short time. Your approach completely ignores these requirements which enormously outweigh the tiny performance gains that you can get by tweaking for the hardware.

High level languages remove the portability problem entirely, by shifting it to the shoulders of the language implementors. If I implement a language on one platform, and you implement the same language on a completely different platform, any program written in that language will now run just fine on both platforms. Sure, unless your compiler or interpreter is incredibly smart, programs written in that language won't run as fast as endlessly hand-tweaked assembly programs written for that platform. But if your compiler or interpreter is even just passingly not bloody stupid, then programs in that language will run pretty close to as fast.

Higher level languages allow the programmer to express more directly exactly what he means. Suppose I have a very very low level language that manipulates bits (ie logic gates). I want an addition routine for arbitrary length sequences of bits. Implementing this in logic gates is not only very difficult and very very time consuming, but the end result doesn't look like addition at a glance. Now suppose I use a high level language. I can just write (a + b). It is clear to anyone who made it through elementary school that this code adds two things, a and b. Of course, I could abuse this, and actually make the language so that (a + b) deletes all files on the hard disk, but a realistic high level language is essentially self documenting. This means that 5 years down the road, when I've forgotten why I even wrote this bit of code, I can pull it up in a text editor and remember that it adds two numbers very easily. Suppose I then need it to add three numbers. Well, now that's a very easy change to make. The code is immensely more maintainable, not only by the original author, but by anyone, than code in a low level language.

Many of the extremely high level languages that are disparagingly referred to as "research languages" have an extremely high information density. That is, I can express in 1 line of code what it might take you perhaps as many as hundreds of lines of code to express in a low level, close-to-the-metal language. This also contributes to the maintainability, because there is a lot less room for mistakes in one line of code than in 20 lines of code. More importantly, this very high expressiveness means that I can write code in a high level language much more quickly. When you're trying to compete with other companies, or trying to finish a PhD thesis in less than 10 years, or pretty much anything at all, being able to code 20 times faster is a very important thing.

If you make a career outside of the very sheltered world of Xbox and PS3 programming, you'll see that endless performance tweaking in very low level languages is not only useless, but it is wastefully stupid.

Re:Functional programming

[shutdown -p now]

Well, speaking of game development, I hope you'll excuse me if I will hold the opinion of that guy Tim Sweeney, you know, the one behind Unreal, higher than yours? 'Cause he seems to disagree with you pretty strongly on many things, threading issues among them. Tools (which languages are) are key to solving this problem, and a lot of it does come from academia, just as all things heavily used in the industry today (like OOP) did.

What?!

[eldavojohn]

Programmers must begin to develop applications that take full advantage of the increasing number of cores present in modern computers.

I'm a developer. I may not be the greatest one but I enjoy it. This declaration baffles me.

You choose to go with a multi-threaded application when it is necessary. Anyone who just starts adding threads because they feel they need to utilize the number of cores is a complete idiot in my book. Hell, why don't we just put spin locks in there so your CPU usage shoots up and it looks like I'm using it to its full potential?

My point is that there have been a few applications I've written that require a multi-threaded solution. Perhaps this API would have made my life easier but I doubt it as I had to pretty much structure by hand each thread. There are frameworks, graphical libraries and that also use multi-threading that the scheduler has taken care of in the past. Hurray for multi-core if you use those.

A good programmer keeps things as simple as possible. They will be easier to maintain in the future. I'm afraid that this is unneeded layer of abstraction or some nut case trying to "utilize cores" for the sake of it. No one has only one application running at one time. The OS is usually running, you have a network process, etc. If I write my application to use one core, I'm giving the user more options to do with the other cores whatever he wants. Let the scheduler work with the futuristic hardware and sort that crap out.

Also, not everyone is multi-core already. Take use into consideration please!

Re:What?!

[acidrain]

why don't we just put spin locks in there so your CPU usage shoots up and it looks like I'm using it to its full potential?

I heard stories of this being done by games companies when their publishers complained they weren't using the VU1 on the PS2 enough. That was the VU which was really hard to utilize because had no access to the rendering hardware. And yes, publishers ran the diagnostic tools available when you submitted builds.

Re:What?!

[zx75]

I think you've missed the mark a little.

I believe what he is saying is that if your an application developer who is pushing the limits of what a single core is capable of in terms of performance, then you are going to see decreasing rate of improvement and then stagnation because the focus of hardware development is shifting away from more power in a single core to more power because there are more cores.
At some point you will hit a wall, and for single-threaded applications you're going to reach a point where there isn't any more power to be had.

Therefore if you want to tap that extra power that a multi-core processor has, you will by definition *need* to start multi-threaded programming. This isn't about you people who are happy with the speed and power that you already have, research is pointless if you already have everything you could possibly need. This is for the people who push the edge, at some point if you need more you will need to learn to multi-thread correctly.

And a simpler way to do it, is gold in my books.

*From a former University classmate of Stephanos*

what a joke

[acidrain]

From the site:

• 1. Replace types: The developer replaces numerical types representing floating point numbers and integers with the equivalent RapidMind platform types.
• 2. Capture computations: While the user's application is running, sequences of numerical operations invoked by the user's application can be captured, recorded, and dynamically compiled to a program object by the RapidMind platform.
• 3. Stream execution: The RapidMind platform runtime is used for managed parallel execution of program objects on the target hardware platform, which can be a GPU, the Cell processor, or a multicore CPU.

Man thats some funny stuff. Wow that cracked me up. A *games* company using a tool that has this level of indirection?!? I sure hope these guys got a lot of money from their sucker VC to roll in.

Look guys. There is no multi-processing silver bullet. It isn't even such a hard problem, *if you stop trying to solve it at such a low level*. Break your application into separate pieces that, *don't need to communicate very often.* Then this is the same kind of problem scalable websites like Google, MySpace, Hotmail and so on, have already, just without having to factor in the reliability issues. Finer grained multi-threading just leads to deadlocks and is really hard to debug. If you *really must* render the same sphere on 100 processors at the same time, then you need the speed of a custom coded solution. But you don't so let it go. The main loop of your program will be just fine as a single threaded implementation, 1 processor will do, and farm the 10% code / 90 % heavy lifting out in big clean chunks to other processors. If you find yourself writing some bizzare multi-threaded message passing system so that you can have 100s of threads all modifying the same live object model at the same time -- you are fucked, just forget about it 'cause you will never be able to debug that one killer bug that you know is going to get you right as you go to ship.

Re:"hundreds of cores"?

[Bastard of Subhumani]

that means in 7.5 years we'll have 128 cores.

What a pity 640 isn't a round power of 2. That ought to be enough for anybody.

Yep, concurrency is a problem, not a solution!

[argent]

100% agree. Concurrency is a problem, not a solution, and it needs to be abstracted out early if you need it at all.

Re:Yep, concurrency is a problem, not a solution!

[grumbel]

Concurrency is a problem, but its one that you *can't* avoid. Everything in todays CPU development points very strongly into a multi-core direction, in a few years you can't buy single cores any more and in a few years down the road again something like eight cores might be the norm. So how exactly do you want to write programs then? Single thread, using only 12.5% of the available computing power? I don't think so. Now it is of course hard to write multi-threading in C++, but other languages such as Erlang or function languages in general have a lot less issues with it, sooner or later we actually might need a big switch in the programming languages, since C++ simply seems to be no longer up for the task.

Square peg, round hole.

[Ihlosi]

Programmers must begin to develop applications that take full advantage of the increasing number of cores present in modern computers.

No. Whether something can be done effectively on multiple cores doesn't depend on the programmer, but on the type of processing. Some things have to be done in a certain order, and there's nothing even the best programmer in the world can change about that, period. If you try hacking something together that uses multiple threads for this type of processing, you'll just end up making things slower and messier.

On the other hand, there are other types of processing that just lend themselves fantastically to being done multithreaded.

Re:C++ can't be made safe

[Gr8Apes]

In Java, this is allowed because of performance issues. You can make it almost 100% thread safe (note I said almost) by synchronizing every method, but there's still some gotchas in the JDK.

Multi-threaded programming is a skill that comes with a level of understanding, much like students of mathematics must reach a level of understanding to comprehend Algebra, Calculus, Differential Equations, and Partial Differential Equations (yeah, that last one's a bear, especially when you apply it to various physical models) respectively. It's why you can be a wiz at adding or subtracting, but utterly fail at algebra, or a wiz at algebra, but never be able to differentiate or integrate a function, and so on.

Writing code is easy. Any moron can do it, as the Java and PHP hordes have shown. Writing good code is harder, designing good OO code is even harder, and designing and writing good multi-threaded code is yet a step beyond that. It's why there are so few well-written multi-threaded apps, and most of those are in server land.

Toy Supercomputer

[Doc Ruby]

The problem with programming the PS3 is that once the complexity of its parallel processors is handled, the CPU is so fast that it consumes and produces data much faster than the IO available. The Cell is a basically 204GFLOPS/32bit machine (plus the Power RISC, basically a Mac G5), with an internal 1.6Tbps bus. But even its builtin gigabit ethernet is puny compared to that kind of dataflow. It's not clear whether the USB slots are 1, 2 or 4 buses at 480Mbps each, but even 2Gbps more isn't so much. Maybe another gig-e can plug into its CompactFlash slot, bringing the total up to 4Gbps, but that's still only 0.25% the chip bus. In desperation, perhaps the SATA bus could also be used for another 1.3Gbps. Adding the HDMI output with some fancy codecing (especially on the receiving host) gives 10.2Gbps out, so the other 5.3Gbps can be used for input, but that's still only 5.3Gbps throughput, probably a lot less at under 100% efficiency per channel. The Cell can spin its wheels with 2000 instructions on the data it's got before it gets more. There are lots of "multimedia mixing" and transformation applications that could run multiple cycles in that 2K instructions, which instead need more machines for more IO.

The PS3 doesn't seem to have the PCI-Express bus that would solve all these problems. For some reason Sony left out its old pet, FireWire, which could have added buses at 800Mbps each. There doesn't seem to be any expansion whatsoever, except changing the HD on the single SATA connector. To use what it's got, a huge amount of complex, heterogeneous IO management is necessary to use its power.

It's strange to think that a $600 machine with around 5Gbps throughput and 7Tbps processing is a "toy", but the cropped IO makes the PS3 look that way, relative to its full power. Maybe a HW mod, even at $500 or possibly up to $2000, that adds PCIe for a half-dozen 2x10Gig-E cards, or even InfiniBand, will make this crazy little toy into more than just a development platform for games or prototypes for really expensive Cell machines. Who's got the way out?

Re:Toy Supercomputer

[Doc Ruby]

Because it's a toy supercomputer. If I find a way to expand its IO, I'll have a $600 supercomputer, scalable into a supercomputer cluster.

If I listen to you, Anonymous defeatist Coward, and just cry "waaaahhh, I'm too dumb to hack a toy into a tool", then I'll just have a really cool toy.

Allow me to introduce you to the term hack, which is what Slashdotters used to do before we were mostly posers.

How many cores?

[Aladrin]

"For his demo he created a program on the PlayStation 3 representing thousands of chickens, each independently tracked by a single processing core. "

Wait wait wait... How many cores does a PS3 have? Thousands? I suspect someone has their facts sadly mistaken. I think they meant 'each with its own thread and using multiple cores to processing the threads,' but that isn't nearly as impressive sounding.

Where are the chicks?

[Meneth]

I browsed through the 411 MB ogg file, but could not find any chicks. Where are they?

Relativity

[stratjakt]

However, multi-threaded development has been notoriously hard to do

Only at first, once you wrap your head around it it becomes second nature.

To a newbie, recursion is hard to do. To somebody who's been writing functional FORTRAN for 25 years, object oriented is hard to do.

It's just another way of thinking about problems. The real bitch is having the toolkits and thread safe libraries at your disposal.

Re:Relativity

[Anonymous Coward]

The real bitch is when you have a bug because your bug is not reproductible as easily than in any other programming method.
So no its not only 'another way of thinking'.
And good luck trying to 'extend' multithreaded stuff.
Multithreading should only be used on very special occasions where it is really needed.
That is harly ever in most end users applications.

Active Objects

[lefticus]

I'm not sure what techniques the developer is using as the um, "article" is a little light on details (unless I missed something) But the concept of Active Objects (a trivialized way of using threads) has been around for a while with generic implementations of them becoming more mainstream rapidly. In the past week there as been much discussion about active objects and "futures" on the boost mailing list and it is likely that both will become part of boost shortly. To put it simply, an active object is an object which has its own threaded message queue, so it is asynchronous from the rest of the system and a future is a return value from an asynchronous method call, a "future" value. These techniques are quite reasonable today because of concepts like fibers and the NPTL.

And of course, a shameless plug for my active objects implementation (bsd style license). Actually, that page also does a decent job of demonstrating the concepts.

Re:C++ can't be made safe

[ratboy666]

This stuff is an outgrowth of the Sh work done at Waterloo U. Anyway, the idea is that the declarations in your code are replaced. The new types redefine standard operators to generate code for a parallel machine (say, an nVidia card, or a PS/3 cell).

The code so generated can be run immediately, or deferred (note -- its been a while since I've looked at Sh, so I am being vague).

I didn't think that this was a GENERAL multi-threading solution; more a way to easily generate code for the parallel machines that are coming available.

Re:Don't Bother

[Nimey]

99.99% of the time

...people who use that phrase don't know what they're talking about, and especially don't grok statistics.

Life is Pain

[gillbates]

First of all, I, and many others before me, have been writing multithreaded applications for years in the likes of Linux and UNIX. I have had to maintain multithreaded applications created by others. My collective experience tells me:

It is not trivial.

Let me repeat: It is not a trivial task. Even if you have libraries and an API which abstracts out the ugly stuff, you still have the problem of concurrency, proper locking, deadlocks, etc...

The majority of problems with using multithreaded programming come not from "ugly" parts of the OS/API layer, but from a misunderstanding of the problem. A few problems in computer science - particularly in the physical sciences - do benefit from multithreading. And it is easier to use threads when writing a game than just to execute all of the IO in one big loop (Hello DOS!). But for most applications, using threads is not only unnecessary, but overkill, and introduces the possibility of yet another class of bugs for which the application must be tested. Furthermore, as deadlock and race conditions are often timing related, they are the most difficult type of bug to find and fix. Finding and fixing this class of bugs is still somewhat of a black art in the industry, and is highly dependent on the skill and experience of the programmer.

In short, unless your system/application design cannot do without multithreaded programming, it is best not to use it. Even with a glossy API, you still cannot escape the fact that debugging a multithreaded application is an order of magnitude more difficult than a single threaded one. In any case, you shouldn't be using threads just because you can.

Re:400MB download

[aadvancedGIR]

Me, I was expecting 100 4MB movies files that you would have to play concurrently.

Pthreads?

[gillbates]

Pthreads has been out for a while. It is open source, and runs on Linux, Windows, and Mac(?).

Whether or not you believe concurrency should be an explicit library or a matter of compiler extension is a bit of a religious argument. But pthreads does offer the functionality, and works fairly well.

Comments from the presenter

[sdt]

Good morning slashdot!

As the (slightly terrified to find himself mentioned on slashdot) presenter in the video linked to above I thought I'd respond to a couple of comments in bulk. First off, I'm part of a much bigger team at RapidMind that builds this software to make targeting multicore and stream processors easier -- the system and the "chicken demo" was a group effort, and you can read more about it and the company in general in the article linked to from here, which unfortunately is PDF-only.

For those crying out about multi-threading not being the solution: you're absolutely right! Our platform's approach to programming multi-core processors is to expose a data parallel model. In this model, the programmer explicitly deals with parallel programming (writing algorithms to work well on arbitrarily many cores) but all of the standard multi-threading issues such as deadlocks and race conditions are avoided, and the developer doesn't worry about how many cores there actually are.

And no, the chicken demo didn't run each chicken on an individual core ;). But it did automatically scale to however many cores were available -- 6 SPUs and a PPU on the PS3, and 16 SPUs and 2 PPUs on a Cell Blade (on which we originally showed the simulation at GDC 2006).

If you want to learn more, drop by our website at http://www.rapidmind.net. You can sign up for a free no-strings-attached evaluation version if you want to try it yourself.

Re:Don't Bother

[Dog-Cow]

"Plus your statement is misleading, very few major apps are single threaded, the OS itself has a ton of stuff going on in the background, there are demons/services running all the time."

Plus, you completely and utterly missed the point of the poster you replied to. Most apps (who cares about major?) are single-threaded. The poster's point is that writing a multi-threaded app JUST BECAUSE THERE ARE MORE CPUs/CORES to handle them is pointless and stupid. If the app only requires a single thread, use just one. The other resources will get used by the OS or by other apps (that may, God forbid, *also* be single-threaded). He wasn't talking about dedicating a computing resource to an app. He was saying that an app should only use what it needs, with the understanding that the OS will make good use of any remaining resources for other tasks.

What a lot of multi-thread-happy people seem to miss is that as long as the OS is multi-tasking, the other resources will not go to waste just because the app in the foreground isn't using them.

Or you could just use Ada

[Black Parrot]

which has had easy-to-use multithreading constructs built right into the language for the past 25 years or so.

Re:Or you could just use Ada

[Coryoth]

Unfortunately I think many programmers that read Slashdot are scared off by the clear, readable, maintainable syntax. Typing end is clearly too much work, or something, and as we all know IDEs can't possibly help with that... I would like to see Ada get more use, but unfortunately I doubt it is going to happen.

Re:Don't Bother

[Retric]

You must not work with network apps all that much.

Think of the most basic email app possible. Now when a user presses "send mail" would you create a new fork (), try and micro manage the remote connation in a thread that handles the GUI, or force the user to wait around?

Next think about video where you have a resource intensive task AND you still want a highly responsive GUI.

Granted if all you ever work with is simple biz apps with one user you have a point but I think your 99.99% estimate says more about the work you do than programming in general. Because threads can often simply demanding applications.

CSP Occam and Transputers

[Anonymous Coward]

The communicating Sequential Processes style of programming allows for many lightweight simple threads that communicate over channels rather that the monitor based thread synchronization.

The OCCAM language implemented this style of processing and the Transputer chip implemented a fast context switching hardware that OCCAM could run on.

This was all done back in the 1980s.

I even implemented the original version of the Java Communicating Sequential Processes API which brought CSP style programming to the Java world, although it is based on Java's underlying Thread mechanism so context switching isn't as fast as it could be.

Transactional Memory

[omnirealm]

For those who have not caught wind of this yet, transactional memory is currently the most promising solution to this problem and perhaps the most-covered subject in research conferences on parallel computing today. There have been several proposals for both hardware-based (at the cache level) and software-based architectures. Transactional memory greatly simplifies concurrent programming. When using transactions instead of locks, deadlocks go away completely and there is increased concurrency.

This isn't multithreading in the traditional sense

[igomaniac]

There's a lot of posts saying that multithreading is really hard, which is completely true... But what RapidMind is providing is something else, something more like a SIMD model or vector computations. It solves things like elementwise operations on large arrays in an efficient manner using whatever parallel computing resources are available. It's a language with a semantics that don't require complicated synchronisation because you're bascially telling the compiler which operations are independent and then it can go off and compute it in the most efficient way possible. RapidMind was designed to make GPGPU programming easy, so it's a generalisation of the pixel shader model where you have a lot of 'threads' computing the color of each pixel on the display in parallel. This is an easy problem, because there is basically no communication between threads.

use gcc4.2

[drerwk]

Yes, gcc 4.2 supports OpenMP. As others note, parallel programming is still not trivial. But OpenMP is very nice. I have a write up on building and testing gcc 4.2 on OS X here: http://alphakilo.com/openmp-on-os-x-using-gcc-42/. Serious advantages are that OpenMP can be retrofitted to existing C/C++ and Fortran code. I know that everyone prefers to start from scratch and use Erlang or some other solution, but in a project I am working on, we already have about a million lines of C++. Current OpenMP implementations favor SMP machines, but one can go even further with the Intel OpenMP for clusters solution. I have not tried it myself yet, but I understand that it makes the issue of non-shared memory across the cluster machines transparent. As in all cases YMMV. But, if your code is amenable to parallelizing, OpenMP is a pretty straight forward way to go.

This is about propping up an obsolete technology

[ClosedSource]

"Gigahertz are out and cores are in. Programmers must begin to develop applications that take full advantage of the increasing number of cores present in modern computers."

The marketplace wants and needs new technologies for more powerful processors. Multicore serves the needs of chip makers, not their customers. Making all software multi-threaded is trying to solve the wrong problem. It's going to result in lower-quality software without a significant increase in performance.

Re:"hundreds of cores"?

[Lazerf4rt]

Yeah, they're looking ahead too eagerly. That's what academics do.

Let's not forget that Intel and IBM both recently found a manufacturing process to keep Moore's law going for the next several years. Most people in 2006 thought we hit a wall, and that the multicore revolution was inevitably under way, but that just might not be true anymore. That said, it is always nice to have at least a few cores in available in your system.

At the same time, AMD's Fusion strategy looks pretty interesting. I really wonder what's going to become of that.

Re:C++ can't be made safe

[Coryoth]

Well, if you're going to remove 99+% of the common trouble spots of multi-threaded coding by moving to a messaging paradigm, then yes, it probably is conceptually easier than OO. It can also be significantly slower depending upon the application's design and function and greatly increase its memory footprint. e.g., I don't think a game like Quake would work all that well under this paradigm. I think it is nowhere near as bad as you seem to think - it all depends on how message passing is handled. If you're doing via some slow complex scheme then, sure, it will be slow. But the trick is to think conceptually in terms of message passing - that doesn't mean it actually has to be handled with a big clunky message passing interface internally; just in terms of how you think about it. Take SCOOP for instance. The "message passing" mechanism there is feature calls, the message being parameters passed to the feature/method. The preprocessor and compiler handles all the messy details of locking etc. and in practice it runs about as fast as hand written threading. The difference is that you think in the simpler terms of actors and messages, while the computer (in this case the compiler) handles the grunt work of converting that into efficient code. This is no different than OO, or garbage collection, of course: you simplify what you need to write by writing in a higher level paradigm and leave the hard work of turning that into efficient machine code to the compiler.

You'll also still have the potential of concurrent modifications in this scenario, but at least you won't be working on the same memory storage locations, potentially reading indeterminate/incoherent values. Instead you'll have inconsistent values displayed, depending upon which thread's data you're displaying. Read the page on SCOOP, or this draft paper, to see what it actually does - it is well worth it: it's the best mix of OO and concurrent programming I've ever seen. You won't end up with inconsistent values because everything will block accordingly with the compiler handling all the necessary locking/blocking/waiting and letting you get on with just writing code.

Dining Chickens Problem

[MS-06FZ]

I'm sure the demonstration would've been a lot more difficult if he'd used philosophers instead of chickens. Thing is, chickens can't even hold chopsticks. A chicken just goes straight for the feed, so there's just one resource being acquired. It's still possible for a chicken to starve, but as chickens don't eat that much it's more likely that any shut-out chickens would simply go hungry for a while, and then get to eat before starving.

Multithreaded programming the easy way

[pbrooks100]

LabVIEW. http://www.ni.com/labview.

Download and play with QtConcurrent today

[IceFox]

A project that you can download and play with today is Trolltech's QtConcurrent. Given a task it will automatically manage creating threads and distributing the task among your cores.

From the project page:

The classes and functions available in the Qt Concurrent package allows you to write multi-threaded applications without having to use the basic threading synchronization primitives such as mutexes and wait conditions. This makes it easier to reason about and test parallel programs to make sure that they are correct.
The Qt Concurrent components manage the threads they use automatically. Each application has a global thread counter, which limits the maximum number threads used at the same time. The maximum is scaled according to the number of CPU cores on the system at runtime. This means that programs written with Qt Concurrent today will continue to scale when deployed on many-core systems in the future.

Very cool.

Inmos had the elegant solution

[Tjp($)pjT]

Inmos Transputers C language development had an elegant solution. It should be migrated to mainstream C and C++ in my opinion:

parallel
{/* execute these statements in parallel if possible */
statement1;
statement2;
...
statementn;
}

sequential
{/* execute these statements in order as written */
statement_1;
statement_2;
...
statement_n;
}

Ada 2005

[krischik]

You forgot to mention that Ada 2005 now adds Interfaces to both protected and task objects. See:

http://en.wikibooks.org/wiki/Ada_Programming/Taski ng

Ada's multi-threadeding is not only without the pain but great fun!

Martin

Ada is thread ready since 1983...

[krischik]

...and Ada 2005 even supports Real-Time programming. It is possible - just not with C++.

Find a short intro here:

http://en.wikibooks.org/wiki/Ada_Programming/Taski ng

Martin