Good Books On Programming With Threads?

uneek writes "I have been programming for several years now in a variety of languages, most recently C#, Java, and Python. I have never had to use threads for a solution in the past. Recently I have been incorporating them more in my solutions for clients. I understand the theory behind them. However I am looking for a good book on programming threads from an applied point of view. I am looking for one or more texts that provide thorough coverage and provide meaningful exercises. Anyone have any ideas?"

Comments

PThreads & Java Threads

[eldavojohn]

However I am looking for a good book on programming threads from an applied point of view. I am looking for one or more texts that provide thorough coverage and provide meaningful exercises. Anyone have any ideas?

I went through grad school not too long ago for Computer Science (disclaimer: it was the kind of computer science degree that doesn't focus on hardware so I might not be the best expert on this). Anyway there were two books for the class.

One dealt with coding regular old C on a plain jain Unix machine and method of (I believe there are others) doing multithreaded in that environment is PThreads (or the super short overview). The book we used is the Addison Wesley book (ISBN 0-201-63392-2). It was informative and comprehensive ... wasn't concentrated specifically on applications like you ask but very good reference. Also, I think there are a lot of good books free online in respect to that topic.

As for Java, there was an O'Reilly book (there's probably a new version out for Java 6) that was pretty good. Not as great of a reference but better on applications of threads in Java. Although, as far as introductory material, I personally learned it all from java.sun.com. Although I can't vouch for whether this is an applied approach or not, I would suggest the concurrency tutorial and a good book on Java Patterns or even a design pattern wiki.

I've never done concurrent programming in C# or Python so I do not know first hand what is best. I do know that erlang has been fun to mess around with in my spare time though!

Recently I have been incorporating them more in my solutions for clients.

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

Re:PThreads & Java Threads

[Anonymous Coward]

The Addison-Wesley book mentioned by the parent is "Programming with POSIX Threads" by David R. Butenhof. It's what I used when I needed to get up to speed on p-threads in a hurry - clear and easy to follow. P-threads are what's in Darwin, (and so BSD) Linux, and I'm guessing based on POSIX compliance, just about every commercial flavor of UNIX. (Presmuably, OpenServer uses fraying threads)

Re:PThreads & Java Threads

[Anonymous Coward]

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

Man, I have to disagree with you. That kind of dinosaur thinking will hold back progress. Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential. I'm sorry if its too hard for you to debug but its just the way the cookie crumbles.

Re:PThreads & Java Threads

[ByOhTek]

I used the llnl.gov stuff to teach myself pthreadsy. I'll second that it is DEFINETLY a good resource, although it misses a few points. Those points are fairly obvious after looking through the pthreads include file with your system and the associated man pages for the functions. RWLocks are a good example.

For C# I used the documentation in Visual Studio. Once you find the class reference, it's extremely useful and well done.

For python, I just go to the python web site. They have three different sets of documentation - the tutorial, the language reference, and the lower-level detailed specs (can't remember what they are called). Between the three of them, they make an incredible reference also.

Maybe I'm crazy, but if you want a few tasks to be handled in parallel, how can the scheduler take care of it without threading? Forking?

Re:PThreads & Java Threads

[ByOhTek]

for the morbidly curious, there's even a pthreads library for windows. LGPLed

http://sourceware.org/pthreads-win32/

Re:PThreads & Java Threads

[zolaar]

Erm, the tenets of programming usually involve the general concept of "Eliminate the unnecessary." Therefore, the parent is correct: if multi-threaded processing is unnecessary, avoid it.

What you meant to add to the dicussion is the corollary: If it is unavoidable, use it wisely.

Re:PThreads & Java Threads

[tepples]

Maybe I'm crazy, but if you want a few tasks to be handled in parallel, how can the scheduler take care of it without threading? Forking?

Yes. The current framework of computer science education makes interactions between processes much clearer than interactions between threads. And for years, Apache used processes to handle new requests because it started on platforms with efficient creation of processes.

Re:PThreads & Java Threads

[Metasquares]

I'm still not convinced effective concurrency requires changes to be made in software. I remember AMD working on something that would allow it to be done more or less automatically, even in a single threaded app. I don't know what happened to that or how effective it actually was, but I think the optimal solution is one that is transparent to the programmer, or at least nearly so.

Re:PThreads & Java Threads

[fitten]

I'm pretty sure that stuff was some rumor that came out before Barcelona was released about how the Barcelona core was going to 'destroy' Core2 (basically a load of wild and crazy speculation).

There's already some parallelisation of sequential code in all modern processors (out of order execution) that works well because it has fairly narrow focus on the instruction stream window. Going out further would be a much, much larger problem. Looking for parallelism in larger windows of the instruction stream, to the point of trying to execute whole 'subroutines' in parallel would require vast analysis (and probably not be very viable anyway as most routines are serial with respect to other subroutines.

So far, such parallelism has only been taken advantage of because the programmers put either explicit threading into the source code (pthreads and other threading APIs) or at least put hints into the code (OpenMP). This isn't a new problem... it's been around for many decades now and, so far, there haven't been really any success in automated tools to thread code. Languages like Erlang and functional languages take advantage of the fact that some language semantics allows parallelism.

Re:PThreads & Java Threads

[Anonymous Coward]

> Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential.

That is somewhat questionable.
Especially with simple numerical calculation with problems that do not fit into the cache, the memory interfaces is the _only_ limited thing, so there e.g. NUMA and process-level parallelism are likely to be the better road.
On the other hand, there is the GPU style parallelism: truly massive parallelism where explicit-thread-style as with pthread is simply not feasible without a massive management overhead.
Multi-threaded apps might well end up another niche, at least in the long term (well, explicit parallelism might stay a niche in general, given how many developers still seem to have massive problems with it, at least if someone comes up with a good-enough (i.e. just 10x slower than the optimum instead of 100x) implicit way).

Re:PThreads & Java Threads

[Nova77]

Well, if it's for C++, why not using boost::thread instead?

Re:PThreads & Java Threads

[johanatan]

Mod parent up!

Re:PThreads & Java Threads

[Anonymous Coward]

Because boost is the spawn of satan if you need to debug a program?

Re:PThreads & Java Threads

[immcintosh]

Only if what you're debugging is one of the boost libraries that makes heavy use of templates or preprocessor directives, which as far as I'm aware the thread library is not (very minor use of templates, which shouldn't case a problem). It's not like boost suddenly magically makes your debugger freak out; it's the way compilers handle templates/preprocessor stuff, and not every boost library uses all that.

Re:PThreads & Java Threads

[SQLGuru]

I just got the latest issue of MSDN Magazine. There were quite a few articles related to multi-threaded (I haven't opened the plastic yet, so I'm not sure if they are good or not). http://msdn.microsoft.com/en-us/magazine/default.aspx

Also hints that the next version of Visual Studio will be better tailored towards multi-threaded programming.

Layne

Re:PThreads & Java Threads

[discord5]

Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential.

For each application you name that is benefited by threading, someone else will be able to name one that isn't. Some processes simply are not parallelizable in a meaningful way, where meaningful is defined as in speed of execution not as in the interactive extravaganza of "looky how I can clicky the button while it's still doing hard maths".

There's a good bit of reading about the subject, although much of it is boring and is often difficult to apply to real-world situations. Amdahl's law in many situations can predict if it's worth bothering with multithreading (or other forms of parallelizing) quite easily.

A tool like cat or grep has no benefit of being threaded since it's a simple sequential task. Suppose you were to multithread "cat" into one thread that reads from disk, and another that displays a line of text on the screen. Thread 1 will spend most of its time waiting for I/O, and thread 2 will spend most of its time waiting for thread 1 to pass data. Except now, your multithreaded cat has a somewhat complicated synchronization mechanism on top of it that makes it a bit harder to debug and probably eats some extra cycles as well.

While the previous example is overly simple, there are plenty of tasks that are a lot more complicated but simply have no benefit of being threaded, because they spend more time waiting for I/O than actually calculating or because the algorithm is simply not worth parallelizing because there is no benefit in speed.

Another example would be an application divided in 3 steps. Step A and B can be executed at the same time independently of each other, while step C depends on step A and B. Both step A and B can be written to use two threads, and if they'd use two threads they'd run in half the time of their non-threaded equivalent. On a dual core machine (or 2 CPU machine) running step A multi-threaded and then step B multi-threaded takes 1 hour. In the other case, running step A and at the same time (on the other core/CPU) running step B single threaded also takes 1 hour. At this point you gain nothing by threading. Of course here I assume that I/O by both processes at the same time doesn't create some sort of delay. But if you're working with large enough data sets (more than you can keep in memory) this becomes less and less of an issue since the I/O overhead will already be there anyway.

If you add to that the fact that threading (especially synchronization) is a subject that is not well understood by everyone (in the "find me out of 200 programmers fresh from school, 10 who can write a program that benefits from multi-threading and actually works" sense), threading suddenly becomes less appealing if there aren't any clear benefits for the application you're working on.

The reason I mention that last part is that because so many schools give kids the "make two threads count to 100 then exit" exercise but fail completely at getting the message across of the fact that most of the time the threads actually need to synchronize with each other. They'll give this long lecture about the dining philosophers problem without actually SHOWING them what that means.

In conclusion: it depends on a lot of factors (size of your dataset, how well your algorithm can be split up in parallel tasks, ...) if your process benefits from threading or not, and you should evaluate at design time using Amdahl's law if there's an advantage or not. If your results in a multithreaded environment are only marginally better, the economical factor of cost of development time suddenly weighs in very heavily.

Having said that: if you're a programmer, have fun with threads at least once. Write something silly in your spare time, it can be an amazing amount of fun and often offers an interesting way of approaching future problems.

Re:PThreads & Java Threads

[greenbird]

Erm, the tenets of programming usually involve the general concept of "Eliminate the unnecessary." Therefore, the parent is correct: if multi-threaded processing is unnecessary, avoid it.

Although unnecessary, threading usually simplifies a program rather than adding complexity. The only caveat is that you understand threading. In my experience I've used threading to greatly reduce the size and complexity of solutions that either were or could have been implemented without them.

Re:PThreads & Java Threads

[ELProphet]

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

I'm going to have to disagree with you on this one. Especially in Java client side rich GUI apps, background threads are one of the most useful components to ensure a responsive interface when dealing with asynchronous requests. They really only need two and a half pieces to implement them easily and efficiently. The first component is the request itself, either a subclass of java.lang.Runnable or javax.swing.SwingWorker. The second is a callback handler. The half piece is the shared data structure, and it's only a half piece because you'll want to use the synchronized collections wrapper to get a (you guessed it) synchronized collection.

• http://http//www.google.com/search?q=SwingWorker
• http://http//java.sun.com/javase/6/docs/api/java/util/Collections.html#synchronizedList(java.util.List)

Brushing up on those pieces will give you the background you need to not block the UI whenever something needs to happen. Threads aren't hard, they just take a little thought.

Re:PThreads & Java Threads

[Gr8Apes]

That's overly simplistic. Any interesting application of reasonable complexity can create gains of 100+% in performance with appropriately implemented concurrent design.

That doesn't mean the performance gain is necessary, but it certainly is desired. Think of audio/video encoding, complex modeling, etc.

Re:PThreads & Java Threads

[Jack9]

Most programmers do not think in parallel processing terms, leading to claims that you should avoid it. While Amdahl and Gustafson have attempted to quantify the impact of strictly serial processes, the underlying premise is that parallelism is efficient (limited by serial processes).

Re:PThreads & Java Threads

[hackus]

MMmmmm...

I take a different view.

This is based on the fact that the clock increase dance we have enjoyed in speed is basically over now.

The real increases in computing speed are going to come from the radical and the agressive use of threads everywhere in your programs.

Why?

Well, although speed is not going to increase, the number of cores per clock will increase dramatically over the future.

Speed is going to come from doing everything at the same time on a per clock tick as much as possible.

-Hack

Re:PThreads & Java Threads

[Anonymous Coward]

I would disagree that it should be avoided. It's not that the scheduler can't do that job, but that implies you are using a fork()/wait()/select()/etc strategy to do the job. Which may mean you are implementing much of the work-flow switching code yourself via events, signals, selects.

With a threaded application, you can write more linear code, and let the scheduler/pthread library deal with switching between work items. I.e. for a server, you can accept a connection, and create a thread. Each thread can operate as a linear code flow, and return when done. And with threads, you share memory with all other threads, which can be a big advantage (or nightmware if you don't think it through properly), and something you can't do in a fork()/wait() strategy.

It's not a magic bullet, but threaded code can be wonderful to use in the right circumstances. It's not to be avoided, it's a tool to be applied where the tool makes sense.

Re:PThreads & Java Threads

[Anonymous Coward]

the gain you get by multithreading can be more than overcomed by the loss due to synchronization.
if you work on separate data it is fine, but as soon as you need synchronization...believe me REALLY think about it, it may be worse to multithread.
not to mention the gain is not worth the headache most of the time.
been there, done that.
multithread is nice for gui and asynchronized requests, and then for very specific cases.
and it is not that programmers are lazy or not ready for it, as multiproc exist for a long time.
It is that multithread is complicated, and useless most of the time since all the nice theory is blown cause you hit the bottleneck that is the synchronization very fast, as processors need to share data..

Re:PThreads & Java Threads

[dhasenan]

Note also that explicit parallelism isn't going to get anywhere near optimal, either.

Or rather, there's a PTAS for optimal multithreaded code, and the cost for getting code better than an automated parallelism scheme will quickly approach the cost of creating that automated parallelism scheme.

Re:PThreads & Java Threads

[Jerry Coffin]

People have been working on automatic parallelization for decades. For a few specific areas it works pretty darned well -- for example, Cray had a Fortran compiler that could automatically vectorize some loops well over 20 years ago. It didn't always recognize code that could be executed in parallel (on its own) but it did so often enough to be quite useful -- it provided a noticeable speedup for a fair number of CPU-intensive processes. Around the same time, a number of other vendors had large parallel processors of various other sorts as well, most of them with at least somewhat similar capabilities in their compilers.

These were effective enough to prove the concept and fuel a lot of research. In nearly any given year since, you can almost certainly find at least one researcher who's at least claimed to be on the verge of a breakthrough to generalizing the ability. At the same time, the number of compilers successfully converting typical single-threaded code to take advantage of multiple cores/processors remains essentially constant (i.e. zero).

It's an interesting problem with a lot of approaches that work for certain kinds of situations and look like they might generalize well. It's easy to get something that works well enough that you're sure you must be on the right track -- but so far, none has gone much beyond interesting experiments.

Worse, just threading isn't really enough for a lot of interesting scenarios. Threading more or less assumes something like small-scale symmetric multiprocessing. In particular, it lacks any real provision for things like rings of processors, where communication with one processor might have to go through another, so you might have strong preferences about how threads are scheduled onto processors. Most APIs do include some sort of control over affinity between threads and processors, but this is working at essentially the assembly language level, with little (or no) ability to specify something like: "I don't care which specific processors these two threads run on, but they should run on adjacent processors with fast communication, whereas those two threads communicate only rarely, so greater separation is more permissible."

On the hardware side, the situation is somewhat similar: multicore processors are still really pretty primitive. Despite lots of talk about a much larger number of cores and such, research has already shown that most of what's currently in use won't scale beyond a few hundred processor or so, except for situations where there's essentially no communication between them. Scaling to run a lot of web servers in parallel (for one example) is fairly straightforward. Getting really large matrices multiplied in a hurry by using a lot of processors isn't nearly so straightforward. In particular, coherency is currently maintained primarily by snooping each others memory transactions, with a lot of caching to keep from saturating the memory buses. That's heavily dependent on the caching to even get beyond a half dozen cores or so. For thousands of cores, much more radical architectures are needed.

Re:PThreads & Java Threads

[ShakaUVM]

My Master's Degree was in High Performance Computing from UC San Diego, and I taught parallel processing.

Yes, you're right that most new programmers out of college will screw up (and screw up badly) if they try to write a multithreaded application. Learning to write parallel applications requires more mind-bending mental gymnastics than, say, when you first learned to write recursive applications. That said, once you get a solid understanding of how safe parallel code should look like, and how it should work, it's fairly trivial to write code that works, and doesn't deadlock. From my experience, it takes about 3 to 6 months of pounding on parallel code to reach that state.

While it's not a trivial amount of time, given the importance parallel code has (and will increasingly have in the future), I don't think it's too great a hurdle to ask for people to learn this stuff. All talk about multi-core programming always boils down to "Well, we'll never find enough programmers who are able to write multi-threaded apps." Well... why?

I think it would be in the best interests of Intel and AMD to sponsor online college classes teaching how to do parallel coding. People aren't buying the new chips since code can't take advantage of it -- if they flip it around and make every program able to multithread (that could benefit from multithreading, as you point out, Amdahl's Law) then demand for their chips would surge, and they'd make the money back in billions.

Re:PThreads & Java Threads

[InfiniteLoopCounter]

Although unnecessary, threading usually simplifies a program rather than adding complexity. The only caveat is that you understand threading. In my experience I've used threading to greatly reduce the size and complexity of solutions that either were or could have been implemented without them.

I only partially agree with that assertion.

I an assignment (university level) I was asked to write a multi-threaded statistics program. The IO routine provided was extremely poor for reading in large volumes of data that was expected and it was read in several times for no reason. Eg. Sum read in data slowly and then another method did something by reading in the data and called sum, thus reading the data again! It was about 30 times faster to just sum the data as it was being read from buffers. No parallel programming required.

On the other hand, I have also recently written a hard drive intensive program. The code had to be optimized to not run extremely slowly and the original approach was single threaded. Getting it to run in parallel required splitting up one huge method into several neater and more manageable methods (I had to split single threaded/multiple threaded parts). The overall code turned out much nicer and more efficient in the end.

This is another case of one guy generalizing some statement (threads make your program needlessly complex) and another guy generalizing the opposite (threads make your program simpler). In reality, the best answer is probably case dependent.

Re:PThreads & Java Threads

[ozphx]

Its generally not possible to avoid threading in a .net project of even moderate complexity.

* The UI pumps messages on its own thread. You need to do background processing on another thread.

* Any communication should be done using asynchonous callbacks, or IO completion ports or similar. Otherwise you'll end up with the noob-esq while-sleep pattern.

* ASP.Net pages run slow as hell, because people like to round trip to the database in sequence when databinding lists - this not only makes the pages slow, it limits total throughput. Learn how to use asynchonous pages. Most of the sites I've had the misfortune to work on could be happily served off my laptop instead of the quad-core beast they were struggling with.

To write modern software you need to understand threading.

Re:PThreads & Java Threads

[JonSimons]

Although, as far as introductory material, I personally learned it all from java.sun.com. Although I can't vouch for whether this is an applied approach or not, I would suggest the concurrency tutorial and a good book on Java Patterns or even a design pattern wiki.

I can vouch that this is definitely an effective applied approach. I learned much the same way, developing a solid understanding of the Java threading and memory model, and this has allowed me to pick up other particular thread implementations (pthreads, Linux concurrency primitives, other OS concurrency models, etc.) very easily on the job. Thankfully Java does a great job of formally specifying the threading and memory model, so you have a solid foundation upon which to build.

Plus, there's nothing more fun than being able to do a "kill -3" on your Java process to get a thread-by-thread stackdump of where everything is at. That makes diagnosing some forms of deadlock absolutely trivial. I've done that on the job, too.

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

I agree with avoiding threading if you don't need it, but I implore all programmers to not think of multi-threading as a mysterious force only suited to experts. You can develop a mastery of the subject, and it will help you in many areas.

Re:PThreads & Java Threads

[wrook]

Just thought I'd chime in with an example of where threads are often used, but unnecessary: UI.

You often have a situation where you have a UI, that in addition to collecting your mouse clicks and keystrokes, is monitoring something in the background. Often people use a thread to periodically check the thing and report it's progress to the UI.

This can easily be avoided using the reactor pattern. You simply add a short callback to the main loop of the GUI toolkit -- often it is called "OnIdle()" or some such thing. The callback polls the thing you are interested in and updates the GUI.

I actually use this technique to do real processing too. For instance, if I need to parse a really big file in the background, I will write some code that parses a bit at a time and insert it into the idle loop. It essentially works the same as a really low priority thread -- everything in the UI happens and when it's finished, you get to do your processing. Just be careful to keep your processing short.

The very nice thing about doing things this way is that you don't have to worry so much about concurrency. And it keeps your UI snappy.

Re:PThreads & Java Threads

[Krishnoid]

All talk about multi-core programming always boils down to "Well, we'll never find enough programmers who are able to write multi-threaded apps." Well... why?

1. Is multicore really the same as multithreading? I thought multithreading worked better with smaller compute cores sitting on one real 'core', rather than whole, separate cores, and that multicore was better for multiple concurrent processes, but maybe that leads into ...
2. The lack of having them as first-class citizens in the O/S, at the level of recognition that processes have in the shell, the kernel, and programming/scripting languages -- e.g., listing them separately in /proc and other internal structures I'm not familiar with.
3. The lack of a well-designed, intuitive multi-threaded debugger -- with how difficult multithreaded processes are to write, I can imagine this would be a very difficult task
4. The title of this post which has some very constructive suggestions.

Re:PThreads & Java Threads

[SkunkPussy]

how can synchronisation be worse than multithreaded? at worst it will devolve to the singlethreaded time + the time spent in synchronisation instructions. so yes it could be marginally worse but if there's any benefit at all to be found from parallelisation then it will be better

Re:PThreads & Java Threads

[SkunkPussy]

thats cooperative multitasking. it can work but it assumes you've correctly broken down your background file reading into sufficiently (consistently) small pieces. It also assumes that your file io is non-blocking which may not be true for all kinds of IO.

Re:PThreads & Java Threads

[Anonymous Coward]

Learning to write parallel applications requires more mind-bending mental gymnastics than, say, when you first learned to write recursive applications. That said, once you get a solid understanding of how safe parallel code should look like, and how it should work, it's fairly trivial to write code that works, and doesn't deadlock.

This is a common assumption among people who have "mastered" threading. In reality, deadlocks are always a huge concern if you have any locks at all. There are two rules you should keep in mind when programming with threads:

1> Threading is (very) hard
2> If you think threading isn't hard, you're not doing it correctly

If you intend to create objects with shared state, where operations be "safely" interleaved, then you need to be extremely careful in your design. The "Effective Concurrency" series in Dr. Dobb's Journal has a lot of useful information on how to go about this.

Alternatively, you could study up on functional programming techniques, and look at how techniques like message passing ensure that concurrent operations don't corrupt one another. Erlang is a great example of a language designed to be used in massively parallel applications, and avoids the traditional thread model of shared state, which is the cause of the majority of problems with multi-threaded applications.

Re:PThreads & Java Threads

[ShakaUVM]

>>In reality, deadlocks are always a huge concern if you have any locks at all.

It depends how you structure your code. I wrote HPC code for four years without deadlocks or race conditions. Really, it's not that hard.

Re:PThreads & Java Threads

[ShakaUVM]

>>he lack of a well-designed, intuitive multi-threaded debugger -- with how difficult multithreaded processes are to write, I can imagine this would be a very difficult task

Yeah, the development environment, debugging, and language support for threads is kind of primitive. I've always just debugged my software using printfs - I've used various debuggers in the past, but I'm usually much faster at finding bugs by just printf-ing.

>>Is multicore really the same as multithreading?

Cores function like additional SMP processors. If you can write code for one, you can write code for the other. You have a lot less inter-processor latency than when writing grid or cluster apps. While there are times when you want to multithread on a single processor (UIs are a classic example of this), that's mainly just window dressing.

>>The lack of having them as first-class citizens in the O/S

Which threads are you talking about? Pthreads in Linux are treated like processes.

Re:PThreads & Java Threads

[jgrahn]

Man, I have to disagree with you. That kind of dinosaur thinking will hold back progress. Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential.

It's funny how often the pro-concurrency crowd can come up with no better arguments than:

1. It's the future!
2. All processes must be able to use 100% of my CPU power!

It doesn't match reality. Almost all processes are I/O bound, and will continue to be so unless networks, disks and keyboards see a sudden, major speed increase.

Re:PThreads & Java Threads

[ultranova]

The half piece is the shared data structure, and it's only a half piece because you'll want to use the synchronized collections wrapper to get a (you guessed it) synchronized collection.

What do you need the collection for, exactly speaking ? Simply have a thread pool and queue new tasks to be executed there, and have the task queue the corresponding UI update to the Swing Event Thread upon completion.

For that matter, if you do need a collection, you could simply use an unsynchronized collection and have the UI updates update it as well. Since all such updates happen in the SE thread, no synchronization is necessary, and your UI code can access it at will.

That's what I've done. Works perfectly.

Re:PThreads & Java Threads

[ELProphet]

The collections are to provide a consistent design pattern to use. Strictly speaking, no you don't need the synchronized collection for events that only update the GUI, but it is good to know that a) you will need them if you're doing actual distributed or parallel computing, and b) you don't need to roll your own synchronization checks since JCF already has them.

Using an unsynchronized collection in a multi-threaded environment is always dangerous. Unless you can guarantee that only one thread will EVER read or write from that collection, you have the possibility of race, deadlock, and corruption. The JCF synchronized collections don't incur a serious performance penalty, even in distributed numerical methods applications. Always IMHO better safe than sorry.

Re:PThreads & Java Threads

[starfishsystems]

This model of processing is better expressed with events than threads.

For discussion, see Ousterhout, "Why Threads Are A Bad Idea (for most purposes)"

Re:PThreads & Java Threads

[Anonymous Coward]

What progress? The fact that we now need a minimum of a 3GHz processor with multiple cores and gigs of RAM for what? To create jobs for programmers?

Re:PThreads & Java Threads

[ultranova]

The collections are to provide a consistent design pattern to use.

Which doesn't answer the question of why you need one for in this case.

Strictly speaking, no you don't need the synchronized collection for events that only update the GUI, but it is good to know that a) you will need them if you're doing actual distributed or parallel computing,

Apart from distributed computing being a whole different beast than mere multithreading, and thus outside the scope of this discussion, you are also wrong. Unless you actually coded the server controlling the distributed computing clients as a GUI application - in which case you are, to put it bluntly, an idiot - there's no reason whatsoever why you'd need synchronized collections, or even a multithreaded server for that matter.

I have no idea what you mean with "parallel" computing which wouldn't fit under either distributed computing or multithreading. Is it some method of quantum-computer programming in the multiple words paradigm ?-)

and b) you don't need to roll your own synchronization checks since JCF already has them.

All those synchronization checks do is ensure that the internal state of the collection remains consistent. You still need explicit synchronization if, for example, you want to ensure that a previously non-empty collection doesn't become empty between the check and some operation depending on that emptiness. It would be much better to wrap a non-synchronized collection in a class which implements the desired functionality and provides its own synchronization which makes sense in the specific context of your application.

Nobody mentioned needles

[Corpuscavernosa]

Working with threads! Get it! BA-ZING! Sorry. I'm not a programmer. Clearly.

real world haskell

[j1m 5n0w]

Probably not what you're looking for, but Real World Haskell is soon to be released and has chapters on concurrent and multicore programming and software transactional memory. Even if you're not interested in Haskell per se, STM is kind of an interesting idea.

Re:real world haskell

[SupplyMission]

Even if you're not interested in Haskell per se, STM is kind of an interesting idea.

Interesting, and completely useless for real applications in real companies.

Re:real world haskell

[j1m 5n0w]

Why do you say that? I only learned about STM recently and haven't used it in anything, so it may have defects I'm not aware of.

Re:real world haskell

[burris]

cuz nobody ever got fired for buying IB^H^HMicrosoft and STM isn't available in .NET or J2EE/whatever

Re:real world haskell

[SupplyMission]

I'm talking about using Haskell to do any real work.

But STM actually sounds useful.

Re:real world haskell

[immcintosh]

The problem is that functional languages are naturally FAR more suited to concurrent programming than procedural ones. To the point that I'm not really sure techniques for one are even applicable to the other. So my guess is that'll only be helpful if he happens to be using another functional language, which does not appear to be the case.

Re:real world haskell

[j1m 5n0w]

That was sort of my initial impression: how does anyone get anything done in a language where you can't even write to local variables or call printf most of the time?

After using it for awhile, I'm starting to see that it really can be used for ordinary programming tasks, not just pure computation. For some problems, the Haskell way of doing things can make things hard that would be easy in other languages. On the other hand, some things that would be hard in other languages are quite easy in Haskell.

It's just a matter of using the right tool for the job. However, I think there are a lot of C++ and Java projects that would have been a lot better off if they had been written in Haskell.

Re:real world haskell

[SupplyMission]

I also like the "right tool for the job" attitude. This is partly where my admittedly adverse reaction to using Haskell for actual work.

In my experience, actual work always takes this basic form:

1. Acquire some data
2. Get parameters
3. Process data according to parameters
4. Output the answer

The overall form of most tasks that I do is procedural, so it seems most natural to use a procedural language. Something that gets me point A to point B, start to finish. Aside from Excel spreadsheets, it's hard for me to think of a case where I would find advantages in functional programming compared to Perl, C or C++.

Although, I can see how one could argue that I have procedural programming tunnel vision, and actually giving Haskell a serious try could be a good thing. :-)

Re:real world haskell

[setagllib]

If you can't map functional concepts on to a procedural language, what good are you as a programmer?

Re:real world haskell

[j1m 5n0w]

You've pretty much described the form of many Haskell programs:

1. Acquire data, command-line arguments, etc.. in the IO monad using hGetContents or similar
2. Process data using some pure computation
3. Return the result to the IO monad, and then output it

With laziness, Haskell can defer the IO until it's actually needed, so you can use this approach to process large amounts of data that don't fit in memory all at one time.

I wrote a ray tracer that essentially functions this way: It reads in a text file describing the scene as a lazy string, then it feeds the string into a scene parser that accepts a string and returns a scene, then it invokes a ray intersection test on the scene for each pixel, using OpenGL to output the result.

The nice part of all this is that the scene parser and ray intersection routines are all pure: they do no IO whatsoever, nor do they write to global data structures. Side effects are all contained in the relatively short IO code that reads from the files and handles the OpenGL calls.

Re:real world haskell

[ozphx]

Actually STM is avaliable for .Net

http://www.codeplex.com/NetSTM

Full support for System.Transactions

Language/Environment specific

[MikeRT]

Pthreads, Java threads and .NET threads are implemented differently. If you need a good Java book, just pick up one of the "Core Java" books that covers threading in one of its chapters since Java threads aren't that complicated. That said, with Java applications (the platform I know pretty well), if you're doing "enterprise" development it's best to avoid using them and let the application server do its black magic for you.

Re:Language/Environment specific

[discord5]

if you're doing "enterprise" development it's best to avoid using them and let the application server do its black magic for you

Finally, confirmation!!!! I always suspected all those acronyms to be some form of arcane hex.

Re:Language/Environment specific

[Cyberax]

On the contrary, Pthreads, Java threads and .NET threads are mostly the same thing in different packages.

There are _really_ different ways to implement multithreading: fork-join model, pi-calculus, STM, message-passing model, etc.

Re:Language/Environment specific

[TheRaven64]

There are _really_ different ways to implement multithreading: fork-join model, pi-calculus, STM, message-passing model, etc.

No, there are different ways of implementing concurrency. Threading, in particular, means shared-memory concurrency with a private control stack. Pi-calculus, STM, Linda and CSP are all examples of other models for concurrency, not of multithreading. They differ in many respects (although pi-calculus and CSP have a lot in common), but share one feature - they are all easier to reason about (and therefore to debug) than multithreading. The only valid use for multithreading is to provide an efficient implementation of one of the other models.

Re:Language/Environment specific

[Cyberax]

Yes, thanks for correction.

IMO, 'raw' multithreading should be used about as often as inline assembly. It's too low-level and error-prone.

Re:Language/Environment specific

[Anonymous Coward]

If you need a good Java book, just pick up one of the "Core Java" books that covers threading in one of its chapters since Java threads aren't that complicated.

Complete rubish bollocks. Multithreaded is hard, very hard. And Java doesn't magically solve multithreading issues. Any "entry level" Java devoting one chapter to Java thread would be a very, very bad choice. Pick Java Concurrency in Practice and learn that, well... Multithreaded programming has many gotchas. The very fact that you write "it's best to avoid them" if you're doing "enterprise development" (sic) shows that you don't understand very well how Java thread are working.

Java Concurrency in Practice is now on your "to-read" list and you come back after that and tell us if you think that any Java book containing one chapter on thread is enough.

Free eBook on Threading in C#

[Deffexor]

I'm still getting the hang of Threading in C# myself, but I found this eBook immensely helpful in getting me understand some of the difficult issues such as Thread Safety, Cross-threading issues, Race Conditions, and Event-Delegate pairs.

http://www.albahari.com/threading/

Re:Free eBook on Threading in C#

[Facegarden]

Hey this is pretty sweet, thanks! I just started learning C# and a bit of knowledge about threading would be great. :)
-Taylor

Re:Free eBook on Threading in C#

[Facegarden]

..aaaaaand i just realized that is from a book, and it was pretty helpful, so i bought the book!
-Taylor

For .NET

[Marxist Hacker 42]

The MSDN documents on the System.Threading object tree are pretty good.

Re:For .NET

[johanatan]

Parallel FX and PLINQ (Parallel LINQ) and in general functional programming (F#) with a minimization of mutable state is the future.

Doing this stuff manually with thread objects is so passe.

Concurrent Programming in Java

[progressnerd]

Concurrent Programming in Java is more or less *the* book on good practices for multi-threaded programming for Java, with many lessons that apply to other languages as well.

Re:Concurrent Programming in Java

[PinkPanther]

I second that.

Re:Concurrent Programming in Java

[hexghost]

Yes - this is the definitive guide, must read.

Re:Concurrent Programming in Java

[K.B.Zod]

I recommend Java Concurrency in Practice as well. It's an updated, in-depth look at Java threads. Doug Lea, author of Concurrent Programming in Java, is a co-author of the newer book. A great read.

Re:Concurrent Programming in Java

[Anonymous Coward]

The problem is is that this book is now dated as it was written before the java concurrency library was made part of the basic java environment. Granted, most of the code for the library came from this book, but you have to keep that in mind when reading it.

Re:Concurrent Programming in Java

[Anonymous Coward]

Don't forget Brian Goetz's "Java Concurrency In Practice", which covers the changes they made to the JVM memory model in Java 5. Also check out the Java Theory and Practice section in IBM's developerworks site.

Re:Concurrent Programming in Java

[jonabbey]

Third that.

Re:Concurrent Programming in Java

[Raenex]

"Concurrent Programming in Java" is pretty awful for somebody new to threads and looking for practical advice. The writing is just plain bad, and too encyclopedic. The Goetz book is much better.

Re:Concurrent Programming in Java

[oldhack]

Great book. Puts Java thread in perspective of overall concurrent programming scheme, and fills you in on the relevant details of Java language and library doc/specs.

Re:Concurrent Programming in Java

[Raedwald]

I recommend Java Concurrency in Practice

As do I. The emphasis on documenting synchronization rules is very good, and the suggestion of using Java annotations to do so is cool.

Re:Concurrent Programming in Java

[Anonymous Coward]

Ia have read this book two times, and I am going to read it a third one. I think this book is great, but you need to read it calmly

Free Online Book

[Anonymous Coward]

http://www.albahari.com/threading/

This is by far the best book on C# threading and best of all it is free. It cleared up a lot of the nuances of threading for me, and I think it would help any programmer.

Obligatory serious needle reply

[davidwr]

The Jacquard Loom involves programming of a sort, albeit without branching or computations. In that sense it's more like a translator, translating punches into patterns. Sort of like printf for the clothing industry.

Java Concurrency in Practice

[mckayc]

I highly recommend this book if you are doing threads or any sort of concurrent programming in Java. It's written by the guys who designed Java's concurrency features.

Re:Java Concurrency in Practice

[majorbugger]

Ditto, highly recommended

Re:Java Concurrency in Practice

[JonSimons]

Great book. (sorry I don't have mod points today, else I'd mod this up).

Win32 System Services by Marshall Brain

[wandazulu]

Before Brain was doing the HowStuffWorks podcast, he wrote what I consider to be the best book of how the low-level stuff in Windows works: Win32 System Services. I learned everything about threads in there.

That said, it's specifically for C/C++ and never mentions Java or C#, so the examples probably won't help. What I really liked was how he explained the Dining Philosophers problem, how mutexes, semaphores, etc., work, and even though he talks a lot about specific API calls, I really learned how Windows was working under the hood and helped my code considerably.

I'd suggest at least reading the threads section in the store and see if it helps getting the gist of it.

Two books

[grindcorefan]

First: Programming Erlang: Software for a Concurrent World
by Joe Armstrong
http://www.pragprog.com/titles/jaerlang/programming-erlang

The Erlang programming language is well suited to develop concurrent programs with.

The second book I'd recommend is
Distributed Systems: Principles and Paradigms, 2/E
by Andrew S. Tanenbaum
http://www.pearsonhighered.com/educator/academic/product/0,,0132392275,00%2Ben-USS_01DBC.html

Not specific to any programming language, but a very good introduction to the concepts and methods used developing distributed systems, as all multi-threaded programs are.

not covered in books on threads

[bugi]

The thread model has some fundamental problems, but since they seem here to stay there are some things you should keep in mind, nicely summarized in this article(pdf).

Article also available in html if you click on the first computer.org link from google. Hmm, why does it work from google and not from slashot?

Re:not covered in books on threads

[TheRaven64]

Threads are a very good tool for building tools for building concurrent applications. They are not, themselves, a good tool for building concurrent applications and should not be treated as such. If you are building an application, stay away from using threads directly, and instead use a high-level concurrency API. If you are building a concurrency API, then by all means use threads (I have done, and so did the Erlang guys), but you probably don't want to be doing this just after reading a book on threads. In short, if you are the kind of person who needs a book on threads to understand them, you probably are not the kind of person who can safely use threads. You would be better off picking up a book on operating systems or distributed systems theory and reading the chapters on concurrency. This will give you a deeper understanding of the problems of concurrency and give you a much, much better overview of the tools which can be used to solve it (threads, message passing, transactional memory, and so on), and how they are implemented.

Mod Parent...

[SwordsmanLuke]

+1 Pity laugh

Why threads?

[BobMcD]

I don't know about you, but I always program better with my keyboard.

Two books - work well together...

[Assmasher]

"Multi-Threaded Programming Techniques" and "Win32 Multithreaded Programming" (don't worry that it says win32, the other book handles the non-Windows issues.) Together they make an excellent coverage of the topic. Later, get into the particulars of how threads are implemented on different platforms, it's quite interesting and really reflect fundamental differences in where to stress the performance of a system.

Multithreading Applications in Win32

[GogglesPisano]

Here's one I found useful: Multithreading Applications in Win32 by Jim Beveridge and Robert Wiener. It's a little dated (no coverage of .NET, for example - it's more focused on C/C++), but it still provides a good introduction to threading and synchronization on Windows.

If you can find an inexpensive used copy, it's worth a read.

Python

[Rinisari]

Howsabout books or sites on Python threaded programming? I'm going to be working on a project in a short while which will require the use of GTK and twisted together in a sort of network scanner system with asynchronous results.

Re:Python

[Anonymous Coward]

python threads are crap, if you really need threading you are better off migrating your codebase to a real language.

Re:Python

[AlXtreme]

I'm going to be working on a project in a short while which will require the use of GTK and twisted together in a sort of network scanner system with asynchronous results.

You don't need threads for this problem. The issue is that you have two main loops (MainLoop/MainContext with GTK/GObject and the twisted reactor), you merely have to make them work together in a single event-driven loop.

Use the iterate call in GObject's MainContext to run a single iteration using twisted's task.coiterate method. This way, twisted runs an iteration of GTK for each iteration twisted does. Read up on both and if you don't figure it out google it. You're probably not the first who ran into this problem.

It is possible to use threads for this problem, but it isn't necessary so long as the designers of both libraries have added a way to join together event loops. It's also a bit more messy, IMHO.

Threads, Lesson 1: Only use threads if there's no other way.
Threads, Lesson 2: Remember lesson 1.

Re:Python

[Vornzog]

Howsabout books or sites on Python threaded programming? I'm going to be working on a project in a short while which will require the use of GTK and twisted together in a sort of network scanner system with asynchronous results.

As much as I love Python, it does have some weak points, and threading is one of them. From the python documentation:

The Python interpreter is not fully thread safe. In order to support multi-threaded Python programs, there's a global lock that must be held by the current thread before it can safely access Python objects.

Threading is there, and I'm sure some decent documentation exists somewhere. But the GIL (global interpreter lock) generally means that there are better ways to approach the problem in python, i.e. processes instead of threads.

It's a point of contention in the community, and the GVR-BDFL point of view is that any attempt to remove it makes Python a lot slower, so he won't.

While I don't use twisted, I am given to understand that it does most of its asynchronous stuff using callbacks - you may be able to leave most of the concurrency to it and avoid the process all together...

Re:Python

[C_Kode]

Python IS a real language and a damn good one for what is possible with it. Some tools have weaknesses and threaded programming is one for Python.

I think we would prefer your advice without the biased opinion attached.

Re:Python

[Anonymous Coward]

There is a new multiprocessing module 2.6 (and eventually 3.) that does allow for true multiprocessing. It uses an interface similar to but not exactly like threads, but actually is forking off new processes and using pipes to communicate in the background. It is possible to use shared memory between processes to give you all the benefits (and pitfalls) of threads.
    On systems where spawning processes is cheap (like Linux) I think it could be pretty useful. While certainly not as low-overhead as pthreads, it should finally allow Python to utilize multicore CPUs if you code your program the right way.

Re:Python

[rk]

While I don't use twisted, I am given to understand that it does most of its asynchronous stuff using callbacks - you may be able to leave most of the concurrency to it and avoid the process all together...

This is doable. I have some demo code I wrote to use twisted with pygame as sort of a proof of concept for a network playable game. It's not a real game, it's just flying a spaceship around a grid, and also provides a server you could telnet to and get updates on the position of ships and shots, but it's got enough code to show how to interface two event-driven systems together.

If anyone is that interested to see it, write me at my username + my user id at gmail and I'll send it to you.

Re:Python

[Anonymous Coward]

Howsabout books or sites on Python threaded programming? I'm going to be working on a project in a short while which will require the use of GTK and twisted together in a sort of network scanner system with asynchronous results.

Python for Unix and Linux System Administration has excellent practical coverage of using threads in Python.

http://www.amazon.com/Python-Unix-Linux-System-Administration/dp/0596515820

Concurrency: State Models & Java Programs 2nd

[Anonymous Coward]

We use Concurrency: State Models & Java Programs in the Concurrency course at RIT. It discusses issues with multiple thread and explains how to model them.

Very nice read too.

http://www.doc.ic.ac.uk/~jnm/book/

oldie but a goodie

[fred fleenblat]

Some background in parallelism is helpful for mastering threads.
I learned from this book:

http://www.lindaspaces.com/book/

C-linda never caught on, but it's not hard to read the examples and apply them to pthreads, java, MPI or whatever framework you're using.

If you want a software design book

[Aceticon]

If what you want is to design multi-threaded applications (thus, more than just coding multi-threaded apps), then the book you want is: Concurrent Programming in Java

A little on the stuffy side at times (not quite as easy a read as Design Patterns) it still provides a deep understanding of the trade-offs and techniques used when designing multi-threaded applications. Personally, I found myself again and again using the lessons and many of the patterns of that book when designing new systems (or fixing systems designed by people who were not used to multi-threading).

One of the best value for money books I have for Java.

PS: The concepts and lessons, being at a software design level, are easily portable to other languages.

The Little Book of Semaphores

[Anonymous Coward]

From an applied point of view this may not help you so much, but it does have a good deal of coverage on proper usage. And plenty of examples.
Green Tea Press

Concurrent Systems

[sjuels]

Best book that I have found in > 15 years of programming and teaching threaded programming is "Concurrent Systems" by J. Beacon - she describes the really hard parts very well.

It's not the actual programming that is hard in threaded programming; it's figuring out how to structure your design and understanding what you can get out concurrency and where the pit falls are.

Best,

I'm new at this

[Anonymous Coward]

but I think you should program with computers, not threads.

Use processes whenever possible

[Rezonant]

I used to be a huge proponent of threading applications, but now, whenever possible, I choose multiprocess models instead. The extra protection from separate address spaces is very often worth it from a debugging perspective. And if you're serious about multiprocessing - you're likely gonna spend more time in the debugger than coding.

Of course, threads have legitimate uses too - but always try to see if you can design for separate processes instead, before commiting to a threaded single process model. Google Chrome shows the way.

Re:Use processes whenever possible

[PotatoFarmer]

It really depends on the problem you're trying to solve. Separate processes make sense for something like Chrome where the individual widgets don't really have to interact with each other. On the other hand, if you've got a lot of inter-process communication going on you're going to be better off with threads.

My general rule of thumb: If it needs to talk extensively to something else in the program or needs to coordinate access to an expensive resource, it gets threaded.

Re:Use processes whenever possible

[QuoteMstr]

No. IPC is dirt cheap. Take X11 for example -- that's perfectly usable using plain old named pipes, even without Xshm.

Most of the time, a threaded GUI program will want to use threads in order to perform some operation or another "in the background" while the UI remains responsive. If this operation has well-defined inputs and outputs, why not write it as a separate program? Communication overhead is going to be low.

Re:Use processes whenever possible

[PotatoFarmer]

Communication overhead may be low, but it's also more likely to be tied to the underlying platform. Why rely on an external provider when you get it free in the same process space?
There's also the issue of process management. When the other end of that named pipe breaks, what happens to that separate process? Is it really dead? If it's still alive, how do you kill it cleanly?
I'm not saying separate processes are bad, I'm saying that they're appropriate for certain problems, just like threaded applications are appropriate for other problems. Picking your technology and then trying to mold your solution to fit it is backward.

Re:Use processes whenever possible

[Anonymous Coward]

Yeah but hand-casting a serializer for arbitrary directed graphs isn't. Even the java one tends to run out of stack too often.

Re:Use processes whenever possible

[Jerry Coffin]

No. IPC is dirt cheap. Take X11 for example -- that's perfectly usable using plain old named pipes, even without Xshm.

Something like "Dirt cheap" is nearly meaningless without (a lot of) context. In a typical situation, X11 might be working with 1920x1280x4 bytes of information that updates a maximum of 60 times per second.

Contrast that to a parallel quick sort that executes each recursive call as a separate task. Even on a single processor, the entire sort will almost certainly execute in less time that it takes to create a new process OR a named pipe. Every current form of IPC is unreasonably expensive, and likely to remain that way for a long time.

In fact, you have to be careful to even make threads useful for a situation like this -- even without inter-process overhead, it's easy for the communication overhead to swamp out the gain in processing.

Re:Use processes whenever possible

[QuoteMstr]

Obviously, video will use a lot of bandwidth; that's why we have Xshm. That's all beside the point.

Typical systems, especially those written by people who ask Slashdot about threading books, don't have parts that need to share a lot of information. They might process a lot of information, but it's localized to one segment of the program. (Consider a CD burning application: the GUI doesn't need to know the bytes written to disk, but only how many of them there are.)

For this kind of workload, IPC really is dirt cheap. We're not talking about discrete event simulations; we're talking about programs that spend 99% of the time sleeping.

Really, if you have to ask whether you need threads, the answer is "no". If you still think you do, you actually need to use several communicating processes.

Mod parent up.

[Anonymous Coward]

Thread programming is not hard. Every processor on the market can only do one thing at a time anyway - meaning, the processor itself is pretty much single threaded. Yeah, yeah, multi core and all that stuff, but until there's compiler that takes advantage of it and actually is smart enough to allocate threads to each core, it's a single thread core. So there!

Just imagine a time slice to do one task at a time. And then call the API required. That's all.

Imagine sweeping the floor and brushing your teeth. One sweep floor. One brush stroke. Semaphore to pee...

Re:Mod parent up.

[larry bagina]

Tip: the kernel handles thread management, not the compiler.

On Windows: "Concurrent Programming on Windows" by

[Westley]

Joe Duffy (of the .NET Parallel Extensions team) has an excellent book due to be published very soon:

Concurrent Programming on Windows

You need to know the platform...

[Anonymous Coward]

For Windows, I would recommend the book by Jeffrey Richter. It gives you the API underpinnings that the languages use.

I haven't found a decent book, but...

[EWIPlayer]

Herb Sutter has been doing a lot of work on this stuff over the last 10 years and his blog is full of stuff on what you should do... it's not too nitty gritty in terms of languages and stuff, but it's very informative in terms of understanding the issues and what not. Check out http://herbsutter.wordpress.com/.

Some rules of thumb that I've found useful:

• Hide mutexes and locks at (nearly) all costs. If you have a queue class, for example, that has a locking push() function, and someone needs to lock for a series of pushes, don't expose the lock to let them lock things for the series of pushes, but provide a push function that takes a list of items instead. Keep thinking of ways to hide your locking strategies. If your class is deadlock-free then you can be reasonably sure (I've always said "reasonably" but I've never seen it not work either) that you'll never see a deadlock in real life either. Race conditions are a different story, however.
• Trying to figure out a solution where you never have to think about the concurrency of things is a scary place to go... Have a logical concurrent model instead. For example, if you work with user's and user's get events, rather than just letting them process any number of events in parallel, it may be reasonable to sequence events per-user and let the users run in parallel.
• If you do have to expose locks, use a locking hierarchy. Herb shows this here: http://herbsutter.wordpress.com/2007/12/11/effective-concurrency-use-lock-hierarchies-to-avoid-deadlock/
• Avoid any concept of being impolite among your threads (i.e. forced interrupts or kills). Be polite. Herb has this here: http://herbsutter.wordpress.com/2008/04/10/effective-concurrency-interrupt-politely/
• Locking sucks, but it's necessary. If you think you can get away without having to lock in a dubious situation, you're probably wrong.
• Unit test, unit test, unit test. If your classes hide all of your locks, then unit tests cover a ton of cases.

I believe that following strict OO guidelines is even more important when dealing with concurrency than when dealing with general ideas in software... and let's face it, it's extremely important even when not dealing with concurrency :)

Re:I haven't found a decent book, but...

[TheRaven64]

Locking sucks, but it's necessary. If you think you can get away without having to lock in a dubious situation, you're probably wrong.

There are lots of good, reusable, lockless data structures around if you know where to look. Keir Fraser's PhD thesis contains a really nice lockless ring buffer design (which he implemented for Xen) and several other useful things (including a transactional list and some other shiny stuff). If you have implementations of these in a library somewhere, then you can often get away without locks. There is one rule you should always obey when writing parallel code though:

No data may be aliased and mutable.

As long as you remember that, then it's easy to write concurrent code. In Erlang, for example, this is enforced for you, since the only mutable data structure is the process dictionary, which is not ever shared. This rule actually applies in a lot of serial code too, but in parallel code failure to apply it is the cause of a great many bugs.

Re:I haven't found a decent book, but...

[cfriedt]

• Hide mutexes and locks at (nearly) all costs. If you have a queue class, for example, that has a locking push() function, and someone needs to lock for a series of pushes, don't expose the lock to let them lock things for the series of pushes, but provide a push function that takes a list of items instead. Keep thinking of ways to hide your locking strategies. If your class is deadlock-free then you can be reasonably sure (I've always said "reasonably" but I've never seen it not work either) that you'll never see a deadlock in real life either. Race conditions are a different story, however.

I believe that following strict OO guidelines is even more important when dealing with concurrency than when dealing with general ideas in software... and let's face it, it's extremely important even when not dealing with concurrency :)

I would have to agree that it's best to hide your locking strategies. However, one can easily get lost in mutex-hell by over-doing it and assigning too many mutexes to a single object. Consider how the OO paradigm provides data encapsulation, and apply it to parallel programming. In many cases, all that you would want is a single, per-object lock (very much like the java paradigm). You can actually hide your checks for race conditions inside of your object itself.

Also, keep in mind that polling wastes CPU cycles. Having a well-thought-out 'protocol' between objects will assist you in using notify() and wait() methods efficiently. This was part of the reason for the creation of CORBA, COM and the like.

For your average programmer, the problem of parallel computing has really only started to appear very recently because of the mass-production of multi-core chips. However, electronics and computer engineers have been playing this game for a while. With that in mind, it's often useful to visual object-oriented-code as the behavioral description of a functional block inside of an FPGA or other reprogrammable circuit. That way, you can think of wires (or buffers) as variables, and events being 'rising edges' (look into SystemC). You can even go so far as to have a global event queue. Personally, I like to do parallel, object-oriented programming using typical state machine structures.

Threadless

[Anonymous Coward]

Also make sure to take a good look at programming with THREADLESS concurrency. This is based on a proven model that consists of lightweight processes that share no memory (therefore no deadlocks) and send/receive asynchronous immutable messages.
Erlang is famous for this. Scala (JVM) supports it, and I think there is something for Java (with clumsier syntax due to language limitations).
This model is being touted as an almost-solution to the challenges of multicore programming.

Re:Threadless

[Jerry Coffin]

This is based on a proven model that consists of lightweight processes that share no memory (therefore no deadlocks) and send/receive asynchronous immutable messages.

You can certainly have deadlocks without sharing memory. Just for example, if two processes need to open two files apiece, you can deadlock without sharing a single byte of memory -- in fact, the two processes can easily be on entirely separate machines and still deadlock.

Re:Threadless

[Anonymous Coward]

Er, yeah, I didn't mean "absolutely no reason for deadlocks", but avoiding a major cause for them.

dont

[convolvatron]

please, just dont program in threads. if you do, years from now you will realize that you shouldn't. threads are a terrible mistake

What the hell?

[QuoteMstr]

You don't need a book about threaded programming in Python.

You need two books: one about Python, one about threads. Concepts are universal and can be applied across as many languages as you want. It's like saying you need to re-take Calculus because you just learned French!

CSP (_the_ fundament)

[Anonymous Coward]

Try the CSP theory of Hoare, you will not need any 'applied' book after that :-)

http://en.wikipedia.org/wiki/Communicating_Sequential_Processes

Andrew Birrell's classics

[Anonymous Coward]

Andrew Birrell of DEC and Microsoft has a couple of great papers on threaded programming:

• An Introduction to Programming with Threads
• An Introduction to Programming with C# Threads

Use threads

[Kludge]

Many people are scared of threads, because they try to do fancy things and get burned. The most important thing to remember about threads is KISS.

Using threads can greatly simplify your programming, if you use them well. Keep your threads as modular as possible, like you would your processes. But unlike processes, you get to forget about IPC and all its problems. Woohoo!

Re:Use threads

[goose-incarnated]

Say goodbye to IPC and all its problems, say hello to shared memory space and all its problems ;-)

Re:Use threads

[uneek]

Many people are scared of threads, because they try to do fancy things and get burned. The most important thing to remember about threads is KISS.

Using threads can greatly simplify your programming, if you use them well. Keep your threads as modular as possible, like you would your processes. But unlike processes, you get to forget about IPC and all its problems. Woohoo!

Hi:

Well I am extending / maintaining apps that are already multithreaded. I want to make sure I don't break the code.

I also want to be able to make an expert sounding guess when asked about whether to use multi-threading or not.

The answer is obvious

[paulyt10]

Google. Finds answers much faster than reading.

never used threads?

[Anonymous Coward]

you should get out more

Thread Time

[DrEnter]

Thread Time by Norton and DiPasquale is older, but pretty good.

Use Erlang

[toby]

Programming Erlang

Re:Use Erlang

[Anonymous Coward]

He specifically asked about threads, not superior concurrency models. The OP is looking to make debugging more difficult, and is interested in manually fighting all scalability issues. Why would you bring Erlang into the conversation?

the little book

[scoopr]

I found Little Book of Semaphores a good read.

OMP

[Anonymous Coward]

If you also plan to be programing in C++ you should definitely learn to use Open MP (http://en.wikipedia.org/wiki/OpenMP). As for which books to read I'm not sure but there are a couple on Amazon to choose from.

mods on crack

[Anonymous Coward]

Look, moderators, that's offtopic. Not flamebait, offtopic.

Flamebait: being an asshole/provoking others into being an asshole. eg "Linux threads suck ass."

Troll: posting shit in order to get a response by people who think you're serious. eg - "I've been a lunix users for years. I wish it had thread support. That's why I'm switching to vista. "

Offtopic: anything unrelated to threading.

Avoiding threads.....

[refactored]

1. You can avoid threads all you like... but several libraries / toolkits automagically spin threads for you. eg. If you using java graphical stuff odds on it has it's own thread whirring away doing stuff.
2. Threads have subtle and noxious interactions with processes. Say "man pthread_atfork" sometime to see what I mean.
3. ISRs/Timer/alarm/signal callbacks are effectively another thread context. ie. Most largish systems that claim to be "single threaded" aren't.

Re:Avoiding threads.....

[QuoteMstr]

You're foolish if you do any real work in a signal handler; what you should do instead of send a simple message to the main event loop (traditionally by writing a byte to a pipe) and handle the event that generated the signal in normal program context.

Threads != bad. Badly programmed threads = bad.

[Anonymous Coward]

Those who just say "don't use threads" are just saying "I don't know how to use threads or I haven't been careful when using them, so stay away from them." The same could be said about certain computer languages.

Specifically, you need to be VERY careful of what you do with threads:

* ALWAYS Lock shared resources.
* Be careful of what order you put the locks into, otherwise you can stumble upon deadlocks.

* NEVER, EVER, update the GUI from a secondary thread unless you really know what you're doing (hint: You don't know). Gui programming doesn't mix up well with multithread programming.

* Dealing with detached threads is a headache. Use joinable threads instead.

* If you're developing a high-performance, low-latency app (i.e. multimedia), you may need to research on lock-free and wait-free programming (not for the faint hearted, tho). But if you're just starting with thread programming, I don't recommend it.

* Also, start making simple threaded programs as exercises. Change the order of the locks to see how you can screw up.

For research, before purchasing a book, search Wikipedia for the following fundamental topics:
- mutex
- critical section
- condition variable
- semaphore (these are the four fundamental synchronization primitives)
- read/write lock
- memory barrier
- RAII (How to make your threads exception-safe)
- dining philosophers problem
- Priority inversion (a typical problem with multithreading)
- Deadlocks
- Concurrent programming

Also Google for:
- joinable threads
- detached threads
- thread execution barrier

Good luck!

Re:Threads != bad. Badly programmed threads = bad.

[UnknownSoldier]

> * NEVER, EVER, update the GUI from a secondary thread unless you really know what you're doing (hint: You don't know). Gui programming doesn't mix up well with multithread programming.

You've never used BeOS have you ...

Have one thread for processing AND UI (like Windows) sucks. It makes apps unresponsive.

I can suggest three books...

[Troposphere]

I can suggest three books... But you've got to be able to read them all at the same time ;-)

Take a look at Erlang

[Anonymous Coward]

I don't necessarily mean the language, but the principles therein, they can be used in any language. The main point is that it avoids low-level locking (mutexes) and ensuing contention by never sharing memory between threads ("processes" in the Erlang VM) which would require it. Instead, data is passed as messages between threads, which work like event handlers then. In languages like C, you can also model this non-shared data by simply using read-only shared data, when you make sure that data isn't modified, you don't need to copy it between the threads.

Other than that, in event-based environments (typically window-based programs) you can often replace threads with a timer, which fires an event with the desired frequency. If all you need is to do something repeatedly and sleep in between, that works pretty well and and completely without locking. If you need a worker thread to do intensive computations in the background, that is not the way though.

cheers

Uli
(Ulrich Eckhardt)

For Python and C

[Anonymous Coward]

"The Little Book of Semaphores" by Allen B. Downey is available for free as a PDF from the author. While it doesn't cover platform specifics, it does cover the core concepts of concurrency and synchronization in both Python and C/PThreads.

Java Concurrency in Practice

[WebManWalking]

Java Concurrency in Practice was written by the authors of the java.util.concurrent package. It's an excellent introduction to the problems of concurrency and explains how java.util.concurrent deals with those problems.

Fair warning: It's written using a lot of the Generics syntax, which may steepen the learning curve if you're not used to Generics.

The problem to books with threads

[Anonymous Coward]

Is people don't multitask very well - especially when trying to read from two places at once. So I don't think any book that uses a threaded design is a good book

Here are two good ones:

[Jane Q. Public]

Programming With Card Looms, Jacques de Vaucanson, The King's Press, 1745.

Weaving Technology, Joseph Jacquard, Colonies? What Colonies? Publishing, 1801.

there is only 1 good book for coding with threads.

[Anonymous Coward]

there is only 1 good book for programming with threads. It only has 1 page which reads: "don't use threads. if concurrency is truly needed -- use serialized io model with IPC "

Taming Java Threads

[jed_reynolds]

I took some classes taught by Allen Holub. Very smart guy, and I certainly enjoyed his book.

He provides good solid explanation on functional models for queue design and listener patterns. He also discusses some pitfalls of threads in Java.

http://www.holub.com/training/java.threads.html

http://www.amazon.com/Taming-Java-Threads-Allen-Holub/dp/1893115100

Operating Systems

[Anonymous Coward]

Threads are easy. It is only when you try to share data between them when you get into trouble. This is one of the fundamental concepts in the study of operating systems.

A good OS course would cover concurrency, shared resources, semaphores, monitors, etc.. I'm currently reading Operating Systems Concepts 8th edition in my OS course (http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470128720.html) and it covers all of this in detail. This book also looks at these concepts across different programming languages and operating systems.

I just finished a homework on shared resources and I like the concept of a monitor much more than a semaphore. (although a monitor can be implemented with semaphores) I've also been coding in ada a bit lately and they have this wonderful structure called a protected type which is guaranteed thread safe by the OS. I'm kinda sad that ada is a dying language.

In other words...

[refactored]

...not only is a signal handler another thread context, it's a dodgy thread context in which you may not do "real" work where the definition of "real" is some dodgy illspecified period of time. And then people wonder why systems aren't robust. Sigh!

Dinosaur thinking and Moore's Law...

[refactored]

Moore's law has gifted me with 2**20 increase in compute power since I started programming.

But me old dinosaur brain hasn't been upgraded at all at all.

Me (and my cow-orkers) were crappy at creating defect free concurrency when we started, and I'm afraid this new bunch of cow-orkers aren't much better either.

Python doesn't have threads

[Secret Rabbit]

That might seem wrong given that Python lists threading modules, but just look at Python's GIL to know what I mean. As in, no matter what you do, Python will still be running on one core. So, if you just want a performance boost because of a lot of I/O, then threads can get you there. Unfortunately, if you want to take advantage of a multi-core CPU with Python, Python's threads won't get you there. There has actually been a lot of discussion on this topic, but Guido just refuses to do it. The interpreter has no threads and the lib is not thread safe.

If you want to do multi-processing with Python, look at its subprocess module.

Guido's blog post on the GIL:
http://www.artima.com/weblogs/viewpost.jsp?thread=214235

The FAQ entry on a (fallacious) reason why they won't remove it:
http://www.python.org/doc/faq/library/#can-t-we-get-rid-of-the-global-interpreter-lock

Re:Python doesn't have threads

[Anonymous Coward]

Look up fallacious. You may not agree with the reason, but it's no fallacy.

Re:Python doesn't have threads

[Secret Rabbit]

Actually, the conclusion is not supported by the reasoning. For those that don't like clicking links, Guido's reason is that there exists a patch the removed the GIL and replaced it with fine grain locks. This failed miserably. BUT, when one thinks about it, this implementation would certainly be doomed to fail for obvious reasons.

When one implements fine grain locks, every time something is accessed, it is locked accessed and released. Clearly, this will impact performance on even a single threaded application. Clearly, this impact will be more and more significant with an increasing number of threads. So, the only thing that can be said by Guido's reasoning is that:

That SPECIFIC IMPLEMENTATION failed to remove the GIL.

Now, if one put everything that's currently global into the specific interpreter, there would be no reason for locks and thus the performance wouldn't suffer. Then each thread could run independently without including (many) locks. Lua has the ability to do this. So, don't tell me that this is impossible WHEN ANOTHER LANGUAGE HAS ALREADY DONE IT.

Applied Operating System Concepts

[dropbearsrus]

One I remember from university was the 'dinosaur' book, "Applied Operating Systems Concepts" by Ali Silberschwartz, Peter Galvin, and Greg Gagne.

Although not a book on thread programming per se I remember it had lots of discussion about threads, semaphores, mutexes, shared mem, resource locking etc and I think good understanding of these topics is essential before you dive into trying to write and debug multi-threaded programs!

Especially if you already have significant programming experience I guess you mainly want the theory anyway.

I think it had exercises around these topics using Java as the teaching language to demonstrate the concepts. (If I'm still thinking of the same book)
http://www.amazon.com/Applied-Operating-Concepts-Abraham-Silberschatz/dp/0471365084?tag=particculturf-20

Bible for Java Concurrency

[Anonymous Coward]

is pretty much Java Concurrency in Practice. Great book

cat and grep do benefit from threading

[Anonymous Coward]

Many OS-es have heuristics for having a separate thread (typically a kernel thread) do read-ahead. Having that speeds up (in wall time) cat or grep.

Threading in C#

[MaX_3nTrOpY]

In C# there's: Threading in C#. It's nice to read and has nice code samples to explain the topics. Oh...and the price is also nice!

Re:Threading in C#

[spongman]

check out the latest MSDN magazine: http://msdn.microsoft.com/en-us/magazine/cc992993.aspx it focuses on parallelism, and highlights some of the upcoming features in C++ (lambda expressions) and the .NET framework (PLINK/TPL).

The little book of semaphores

[hkon]

I can recommend the Little Book of Semaphores, which is freely available at http://www.greenteapress.com/semaphores/

If by "an applied point of view" you mean "give me code that I can cut and paste into my app" this might be a bit theoretical for you, but if you make it through this book you will actually understand what's going on and recognize the difficult problems.

Interprocess Communications by W. Richard Stevens

[elwinc]

I learned threads from this book by W. Richard Stevens. I have to warn you that Stevens' books are specific to C, but he has a clear and concise style, and his treatment of the basic issues such as synchronization, locking, shared memory, and semaphores is excellent. If you're coding in C, buy the book; if you're not coding in C you may not need to own it, but you ought to spend a couple hours reading Stevens' discussion of key issues.

Communicating Sequential Processes

[AtrN]

Hoare's classic is freely available. It's worth reading despite workable higher-level techniques being available. I like this bit...

In its full generality, multithreading is an incredibly complex and error-prone technique, not to be recommended in any but the smallest programs. In excuse, we may plead that it was invented before the days of structured programming, when even FORTRAN was still considered to be a high-level programming language!

if you're doing java

[bashar]

http://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601/ref=pd_bbs_4?ie=UTF8&s=books&qid=1223461483&sr=8-4

I misread that

[WetCat]

as Programming with Threats
or was it
Programming with Treats?

recommendation: dave butenhof

[Monkius]

Programming with POSIX threads.

C++ Concurrency in Action

[mavam]

While TBB give you one option to write C++ code in OO style, Boost.Thread and the upcoming new standard provide a similar API. Although not yet available entirely, this book looks promising to me:

http://www.manning.com/williams/

What's inside:
* When and when not to use concurrency
* Concurrency and multi-threading in C++
* Concurrency support in the New Standard
* How to improve performance with concurrency
* How to manage concurrency
* Problems and solutions in sharing data
* Synchronization and why we need it
* Memory model details

Description (from the website):
With the new C++ Standard and Technical Report 2 (TR2), multi-threading is coming to C++ in a big way. There is a new memory model with support for multiple threads, along with a new multi-threading support library featuring low-level atomic operations, as well as basic thread launching and synchronization facilities. TR2 will provide higher-level synchronization facilities that allow for a much greater level of abstraction, and make programming multi-threaded applications simpler and safer.

C++ Concurrency in Action is the first book to show you how to take advantage of the new C++ Standard and TR2 to write robust multi-threaded applications in C++.

[...]

An Introduction to Programming with C# Threads

[CodeBuster]

The following white paper was written by Andrew D. Birrell of Microsoft Research and it is free. If you are looking for an introduction to multithreaded programming in general OR especially if you are using C# then it will probably be worth your while.

An Introduction to Programming with C# Threads (PDF)

Parallel FX, PLINQ, F# (or Haskel, Erlang, et al)

[johanatan]

Parallel FX and PLINQ (Parallel LINQ) and in general functional programming (F#) with a minimization of mutable state is the future.

Doing this stuff manually with threads is passe.

Still can't get progress bar widgets right

[Latent Heat]

It has been how long since Xerox PARC, and people still can't get the implementation of a "progress bar" widgets right.

Look up on Google on how to do a progress bar in Java -- you know, where you do some long process such as an install or a long file save, and you get this bar widget telling you that it is "50% complete."

So the Java noob posts to a forum, "I do a long calculation, and I update the progress bar widget, and nothing happens." The response is "you are blocking the event-dispatch thread (EDT) from updating the GUI, and the proper solution is a 'Swing worker thread class."

So you spin off a Swing worker thread to do the long calculation, and you properly use invokeLater() to synchronize with the EDT to make periodic updates of the progress bar, and the EDT is not blocked by your long calculation. So seriously, what problem does that solve. The progress bar is no longer blocking the EDT at all, but it needs to be blocking the EDT in some sense to prevent certain forms of reentrency. The big deal with GUIs is that they are "non-modal", but there are times when you want some kind of mode and some form of modal behavior. When you are doing the long calc, you may want to update the progress bar, you may even want to enable a Cancel button to bail, but you may want to suspend all other activities of the app until this calculation finishes.

The classic example is that you need to save a long file on exit from an app. So how does a Swing Worker help you at all? The offered solution is to initiate the Swing Worker in response to the Close-and-Save command, and then let the Swing Worker invokeLater() back on to the EDT to post the quit message when finished.

The Windows solution is to use something called a nested event loop. When you are doing the long calc or the long file operation, you run a PeekMessage followed by DispatchMessage loop to keep event processing going. The problem with that is still reentrancy -- if the GUI is active, you could issue another menu command that would mess with your long calc or save. One answer to that is to use the form of the PeekMessage to only process paint events and otherwise lock out the GUI. Another answer is to do this from a modal dialog with a progress meter and a Cancel button. You PeekMessage all events, but since you are modal to that dialog, the only allowed GUI actions are progress bar updates and selecting Cancel, so you don't have the reentrancy problem.

The Visual Basic folks have a DoEvents() function that runs such a nested event loop, but the rap on DoEvents() is that the GUI is still active and you have the reentrancy problem. Most VB developers probably don't think it is a problem because they never test for it by poking at the GUI while the long calc is in progress. But again, the solution is to DoEvents() while presenting a modal dialog with the progress bar and cancel button. Or use a message filter as is not available with VB .NET.

People complain that a nested event loop is evil, or at least that it kills the non-modal aspect of the GUI. That is the whole point, there are times when you want the GUI to go modal on you, and the modal dialogs themselves have nested event loops in them to pull that off. The nested event loop with the appropriate message filters or modal dialog to suppress GUI reentrancy is often the proper way to handle the long calc with progress bar situation given that one may not want to permit other GUI operations until the calc or file save is complete. This business of telling people to use a Swing Worker thread and to Tarzan-swinging-from-the-fines to chain from form close, to long save, to post quit message is a complete kludge.

Windows is Windows you say, and Java is reliant on the Swing Worker to do this. Not! There is an EventQueue class in Swing, and it has exact analogs to the Windows PeekMessage and Dispatch message calls. So you can implement a modal state with nested event loop in Java anyway, and this should be preferred to threads for many cases where such a modal state is required. So why is the EvenQueue class in Swing such a guarded secret that no one recommends its use? It has been 30+ years since PARC, and people still don't know how to write a proper GUI app.

Study Apache Source Code

[jawahar]

I'd advice you to download Apache source code
http://download.nextag.com/apache/httpd/httpd-2.2.9.tar.bz2