Good Books On Programming With Threads?

uneek writes "I have been programming for several years now in a variety of languages, most recently C#, Java, and Python. I have never had to use threads for a solution in the past. Recently I have been incorporating them more in my solutions for clients. I understand the theory behind them. However I am looking for a good book on programming threads from an applied point of view. I am looking for one or more texts that provide thorough coverage and provide meaningful exercises. Anyone have any ideas?"

Comments

PThreads & Java Threads

[eldavojohn]

However I am looking for a good book on programming threads from an applied point of view. I am looking for one or more texts that provide thorough coverage and provide meaningful exercises. Anyone have any ideas?

I went through grad school not too long ago for Computer Science (disclaimer: it was the kind of computer science degree that doesn't focus on hardware so I might not be the best expert on this). Anyway there were two books for the class.

One dealt with coding regular old C on a plain jain Unix machine and method of (I believe there are others) doing multithreaded in that environment is PThreads (or the super short overview). The book we used is the Addison Wesley book (ISBN 0-201-63392-2). It was informative and comprehensive ... wasn't concentrated specifically on applications like you ask but very good reference. Also, I think there are a lot of good books free online in respect to that topic.

As for Java, there was an O'Reilly book (there's probably a new version out for Java 6) that was pretty good. Not as great of a reference but better on applications of threads in Java. Although, as far as introductory material, I personally learned it all from java.sun.com. Although I can't vouch for whether this is an applied approach or not, I would suggest the concurrency tutorial and a good book on Java Patterns or even a design pattern wiki.

I've never done concurrent programming in C# or Python so I do not know first hand what is best. I do know that erlang has been fun to mess around with in my spare time though!

Recently I have been incorporating them more in my solutions for clients.

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

Re:PThreads & Java Threads

[Anonymous Coward]

Most important rule of thumb of multi-threaded programming is to avoid it if possible. Maybe hardware (multi-core) will change that, maybe you feel the scheduler can't do its job as well as you can and maybe you feel it's more intuitive. But, often is the case, that you're just adding more complexity to your code resulting in more difficult bugs and harder maintenance for others. Keep it simple.

Man, I have to disagree with you. That kind of dinosaur thinking will hold back progress. Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential. I'm sorry if its too hard for you to debug but its just the way the cookie crumbles.

Re:PThreads & Java Threads

[zolaar]

Erm, the tenets of programming usually involve the general concept of "Eliminate the unnecessary." Therefore, the parent is correct: if multi-threaded processing is unnecessary, avoid it.

What you meant to add to the dicussion is the corollary: If it is unavoidable, use it wisely.

Re:PThreads & Java Threads

[discord5]

Multi-core is the future and multi-threaded apps are exactly what's needed to fully utilize its potential.

For each application you name that is benefited by threading, someone else will be able to name one that isn't. Some processes simply are not parallelizable in a meaningful way, where meaningful is defined as in speed of execution not as in the interactive extravaganza of "looky how I can clicky the button while it's still doing hard maths".

There's a good bit of reading about the subject, although much of it is boring and is often difficult to apply to real-world situations. Amdahl's law in many situations can predict if it's worth bothering with multithreading (or other forms of parallelizing) quite easily.

A tool like cat or grep has no benefit of being threaded since it's a simple sequential task. Suppose you were to multithread "cat" into one thread that reads from disk, and another that displays a line of text on the screen. Thread 1 will spend most of its time waiting for I/O, and thread 2 will spend most of its time waiting for thread 1 to pass data. Except now, your multithreaded cat has a somewhat complicated synchronization mechanism on top of it that makes it a bit harder to debug and probably eats some extra cycles as well.

While the previous example is overly simple, there are plenty of tasks that are a lot more complicated but simply have no benefit of being threaded, because they spend more time waiting for I/O than actually calculating or because the algorithm is simply not worth parallelizing because there is no benefit in speed.

Another example would be an application divided in 3 steps. Step A and B can be executed at the same time independently of each other, while step C depends on step A and B. Both step A and B can be written to use two threads, and if they'd use two threads they'd run in half the time of their non-threaded equivalent. On a dual core machine (or 2 CPU machine) running step A multi-threaded and then step B multi-threaded takes 1 hour. In the other case, running step A and at the same time (on the other core/CPU) running step B single threaded also takes 1 hour. At this point you gain nothing by threading. Of course here I assume that I/O by both processes at the same time doesn't create some sort of delay. But if you're working with large enough data sets (more than you can keep in memory) this becomes less and less of an issue since the I/O overhead will already be there anyway.

If you add to that the fact that threading (especially synchronization) is a subject that is not well understood by everyone (in the "find me out of 200 programmers fresh from school, 10 who can write a program that benefits from multi-threading and actually works" sense), threading suddenly becomes less appealing if there aren't any clear benefits for the application you're working on.

The reason I mention that last part is that because so many schools give kids the "make two threads count to 100 then exit" exercise but fail completely at getting the message across of the fact that most of the time the threads actually need to synchronize with each other. They'll give this long lecture about the dining philosophers problem without actually SHOWING them what that means.

In conclusion: it depends on a lot of factors (size of your dataset, how well your algorithm can be split up in parallel tasks, ...) if your process benefits from threading or not, and you should evaluate at design time using Amdahl's law if there's an advantage or not. If your results in a multithreaded environment are only marginally better, the economical factor of cost of development time suddenly weighs in very heavily.

Having said that: if you're a programmer, have fun with threads at least once. Write something silly in your spare time, it can be an amazing amount of fun and often offers an interesting way of approaching future problems.

Re:PThreads & Java Threads

[ShakaUVM]

My Master's Degree was in High Performance Computing from UC San Diego, and I taught parallel processing.

Yes, you're right that most new programmers out of college will screw up (and screw up badly) if they try to write a multithreaded application. Learning to write parallel applications requires more mind-bending mental gymnastics than, say, when you first learned to write recursive applications. That said, once you get a solid understanding of how safe parallel code should look like, and how it should work, it's fairly trivial to write code that works, and doesn't deadlock. From my experience, it takes about 3 to 6 months of pounding on parallel code to reach that state.

While it's not a trivial amount of time, given the importance parallel code has (and will increasingly have in the future), I don't think it's too great a hurdle to ask for people to learn this stuff. All talk about multi-core programming always boils down to "Well, we'll never find enough programmers who are able to write multi-threaded apps." Well... why?

I think it would be in the best interests of Intel and AMD to sponsor online college classes teaching how to do parallel coding. People aren't buying the new chips since code can't take advantage of it -- if they flip it around and make every program able to multithread (that could benefit from multithreading, as you point out, Amdahl's Law) then demand for their chips would surge, and they'd make the money back in billions.

Language/Environment specific

[MikeRT]

Pthreads, Java threads and .NET threads are implemented differently. If you need a good Java book, just pick up one of the "Core Java" books that covers threading in one of its chapters since Java threads aren't that complicated. That said, with Java applications (the platform I know pretty well), if you're doing "enterprise" development it's best to avoid using them and let the application server do its black magic for you.

Re:Language/Environment specific

[TheRaven64]

There are _really_ different ways to implement multithreading: fork-join model, pi-calculus, STM, message-passing model, etc.

No, there are different ways of implementing concurrency. Threading, in particular, means shared-memory concurrency with a private control stack. Pi-calculus, STM, Linda and CSP are all examples of other models for concurrency, not of multithreading. They differ in many respects (although pi-calculus and CSP have a lot in common), but share one feature - they are all easier to reason about (and therefore to debug) than multithreading. The only valid use for multithreading is to provide an efficient implementation of one of the other models.

Free eBook on Threading in C#

[Deffexor]

I'm still getting the hang of Threading in C# myself, but I found this eBook immensely helpful in getting me understand some of the difficult issues such as Thread Safety, Cross-threading issues, Race Conditions, and Event-Delegate pairs.

http://www.albahari.com/threading/

Concurrent Programming in Java

[progressnerd]

Concurrent Programming in Java is more or less *the* book on good practices for multi-threaded programming for Java, with many lessons that apply to other languages as well.

Re:Concurrent Programming in Java

[K.B.Zod]

I recommend Java Concurrency in Practice as well. It's an updated, in-depth look at Java threads. Doug Lea, author of Concurrent Programming in Java, is a co-author of the newer book. A great read.

Java Concurrency in Practice

[mckayc]

I highly recommend this book if you are doing threads or any sort of concurrent programming in Java. It's written by the guys who designed Java's concurrency features.

not covered in books on threads

[bugi]

The thread model has some fundamental problems, but since they seem here to stay there are some things you should keep in mind, nicely summarized in this article(pdf).

Article also available in html if you click on the first computer.org link from google. Hmm, why does it work from google and not from slashot?

Re:Use processes whenever possible

[PotatoFarmer]

Communication overhead may be low, but it's also more likely to be tied to the underlying platform. Why rely on an external provider when you get it free in the same process space?
There's also the issue of process management. When the other end of that named pipe breaks, what happens to that separate process? Is it really dead? If it's still alive, how do you kill it cleanly?
I'm not saying separate processes are bad, I'm saying that they're appropriate for certain problems, just like threaded applications are appropriate for other problems. Picking your technology and then trying to mold your solution to fit it is backward.

Avoiding threads.....

[refactored]

1. You can avoid threads all you like... but several libraries / toolkits automagically spin threads for you. eg. If you using java graphical stuff odds on it has it's own thread whirring away doing stuff.
2. Threads have subtle and noxious interactions with processes. Say "man pthread_atfork" sometime to see what I mean.
3. ISRs/Timer/alarm/signal callbacks are effectively another thread context. ie. Most largish systems that claim to be "single threaded" aren't.

I can suggest three books...

[Troposphere]

I can suggest three books... But you've got to be able to read them all at the same time ;-)

Here are two good ones:

[Jane Q. Public]

Programming With Card Looms, Jacques de Vaucanson, The King's Press, 1745.

Weaving Technology, Joseph Jacquard, Colonies? What Colonies? Publishing, 1801.

Python doesn't have threads

[Secret Rabbit]

That might seem wrong given that Python lists threading modules, but just look at Python's GIL to know what I mean. As in, no matter what you do, Python will still be running on one core. So, if you just want a performance boost because of a lot of I/O, then threads can get you there. Unfortunately, if you want to take advantage of a multi-core CPU with Python, Python's threads won't get you there. There has actually been a lot of discussion on this topic, but Guido just refuses to do it. The interpreter has no threads and the lib is not thread safe.

If you want to do multi-processing with Python, look at its subprocess module.

Guido's blog post on the GIL:
http://www.artima.com/weblogs/viewpost.jsp?thread=214235

The FAQ entry on a (fallacious) reason why they won't remove it:
http://www.python.org/doc/faq/library/#can-t-we-get-rid-of-the-global-interpreter-lock