Slashdot Mirror
What Debugger Is Best For Multithreaded Apps?
pollyp asks: "I'm on a team that's working on a multi-threaded (pthreads) C++ application running on GNU/Debian 2.2. One of the challenges of this project is trying to debug the app, or making sense of core files; the feeling is that gdb is just too difficult to work with in a heavily threaded app. We're looking for better tools, and if that means changing languages or going to another UN*X, we're ready to do it. So what is the best, most mature OS/language/tool UN*X environment for doing multi-threaded application development on Intel?"
"One part of the team wants to stay on GNU/Linux but use Java. Java's performance is becoming less and less of an issue, and the GNU/Linux development tools are reasonably attractive (e.g., Visual Age's debugger). The other part of the team wants to stick with C++, but move over to Solaris. But none of us have any practical experience with Solaris, nor do any of us have a sense of how fully baked their Intel development environment is."
As to performance: if you're not writing a nuclear explosion simulator, Java's performance is probably adequate. (Think StarOffice here.)
--
Being well balanced is overrated. -- John Carmack
Are you really changing Language and OS for better multi-thread debug?
This is one of the symptomathics of IT today. You start out analyzing the problem, then you choose an aproprite tool. Some month's later you hit trouble and you consider moving to new language, OS, architecture, whatever, totally discarding the initial analysis that led to the choises you made!
On topic: You should not expect multi-threaded code to be easily debugged, afterall it is hard to write!
You should instead put effort into the really core components that are multi-threading sensitive (synchronization when referencing datastructures and stuff like that, btw. C++ is excellent for that, using Monitors), and make all other code REQUIRE to use the methodology you develop in these components.
Try to collect the multi-thread sensitivity in small, well-defined areas of your program, and try to prove, or at least reason, about the correctness of the code there. One thing you may wan't to do is define (in C++) Locking Iterators, which Lock datastructures they work on, and (using reference counts) automatically unlocks the structure when Iterators fall out of scope.
You may also have some luck in using some of the design patterns that deal with multi-threading.
Morale:You cannot debug away every race-condition, that's bad practice you HAVE to think.
SLOGEN [ http://ungdomshus.nu : Sebastian cover music]
The debugger is dbx, fully supporting multithreaded debugging, and has a full GUI on top of it, as well as full visual integration into Emacs (better than you've ever seen gdb do).
Some people think that Solaris' thread libraries are less than adequate. Personally, I've never used a better multithreading system. The entire "kernel-scheduled LWP managing user-scheduled threads" many-many concept is more efficient and works far better than Linux's one-one kernel-scheduled threads. And all your threads run with the same PID - what a concept!
The obvious disclaimers are that I haven't used the newest Solaris tools, nor was this done on x86, but definitely consider not only the tool quality but also the OS' attitude towards threads. Solaris is better than Linux in that regard, IMO.
Michael J.
Michael J.
Root, God, what is difference?
I can not know much of your situation, but on the surface it looks like a perfect example of the need of a clean rewrite. Scrap all the code you have, and the whole design, and consider all of it at most a preliminary study. But for crying alound, keep the same people and the same tools, unless it is blindingly obvious you are trying to use a totally wrong tool for the job - and C++ for multithreaded programming is not such a bad choice. If need be you may have to add one or two good, experienced people in your team.
Then, start from the beginning. Design the system all again. You know you have problems with thread, so do not overuse them. Consider each thread a black box, and specify its interfaces, just like you are supposed to do with single-threaded systems. Specify the lockings and controls you need for things to work. Then implement bottom up, and test everything as you go.
Spend more time in designing end implementing test suites than on the project itself - this will cost more at the onset, but increases your chances of getting the job done, probably faster than without the investment.Build a good debugging log that can be written from any thread, and where you control bufering. Build test routines to delay things, and pepper your code with them. Routines to exercise each module independently, and various combinations. And what ever else you can think of, that might come in handy. Make most tests automatic, so you can leave them running every night if need be. But most of all, make a clean and understandable design, and do not introduce new languages, tools, methods, or any other reasons for confusion!
In Murphy We Turst
Or you could try ddd
If you need a visual dbugger. It acts like gdb, which I already love, but adds a visual edge that gdb alone doesn't have. When I am really stumped with gdb, I go to ddd.
We've used Metrowerks CodeWarrior to develop and debug very heavily multithreaded applications on MacOS, NT, Solaris, and Linux for about five years now; the CodeWarrior tools seem very much 'up to the job' and thread-aware. As a nice plus, CodeWarrior runs on multiple platforms (including Mac OS X now), which is a nice plus for our development, which is all multiplatform.
We now use a diversity of compilers/IDEs/debuggers, but CodeWarrior is still a favorite, even if it's just because of the "Blood, Sweat, and Code" T-shirts.
To all this I'll add:
Java is a lot more productive than C++.
C/C++ can be lots more runtime efficient.
Aside from that, you're better off in Java.
I miss C++ (and even C) sometimes, but Java is usually a better language overall. Not hard to migrate.
If you go with Java, take a strong look at Doug Lea's stuff.
It's a parallel debugger, which means that it has built-in support for manipulating groups of objects, as if they were one: threads in a proceses, processes in a group, groups in a cluster, and so on. This means you don't need 20 windows to control a 20-thread app; we roll up an aggregated view into one window, commands work on the entire batch.
And yes, we support Linux, as well as almost every other Unix out there. You can snag a free demo license and download the bits from our website. And for those of you who like printf(), you can add them on-the-fly without recompiling. That saves time.
Disclosure: I am a developer on TotalView, but I do "eat my own dog food"- we use it every day on itself.
I can explanate how to administrate your network. You must configurate and segmentate it, so it can computate.
I was expecting someone to make this stupid suggestion. [...] What the hell has this got to do with his suggestion - he's asking about debugging threads. [...] This is a fact that you pulled out of your ass.
When a developer (or team of developers) is spending a lot of time debugging and still not solving the problems, I ususally take this as a sign that there are more fundamental problems than the quality of the debugger. I've been developing serious software for 15 years in Pascal, C, Perl, C++, Objective-C, and Java, and I rarely need to us anything more than the occasional printf.
Admittedly, developing threaded code is hard stuff, and I don't know enough about the original poster's problems to say what is going on. But it is possible that they are in over their heads, in which case it may behoove them to a) admit that they don't know enough and start again from basics, and b) they may wish to choose a language that is more forgiving of inexperienced developers.
In that case, suggesting Java is a good idea. That's not to say that C is bad; if I am doing something speed-critical and am working with a team of crackerjack developers, then C would be my first choice.
But if I'm doing something where maximizing CPU efficiency isn't our #1 issue, or if I'm working with developers who are less than stellar, then I lean towards Java. Why?
- No pointer errors - Experts may not make pointer errors, but average developers sure do. With Java, there are no buffer overruns, no broken pointer arithmetic, no SEGVs. And even better nobody else making some stupid pointer error that hoses one of your data structures, making you spend days looking for a bug that isn't there.
- No malloc errors - Java allows you to pay less attention to memory allocation. This makes classic memory leaks impossible, and subtle leaks harder. You're right that novice Java developers take longer to learn the value of reference handling than C developers, but this is mainly because Java extracts a much smaller penalty for those errors. By your logic, presumably, C++ would be even better if each time the developer left a dangling reference they received a high-voltage shock to the nipples.
- Exception handling - It's been several years since I've used C++, so maybe the exception handling there has improved. But Java's exception handling is a good thing, making it much easier to track down errors when they do happen.
- Multiple VMs - When I am getting some weird-ass error, it's wonderful to be able to try several different VMs on different hardware platforms. That removes all question of OS issues, endian problems, or bugs in the run-time.
So the "maybe you should use Java" is a reasonable answer to the question, especially since the original poster specifically mentioned that they were looking at using java.Ergo, there was no need for you to be an asshole about it.
In the true open source mind, it would be be better to say:
You are changing a debugger because it misses a feature? THEN IMPLEMENT IT
Comments like this make me wonder exactly whether Slashdot is read by programmers or simply people who have heard about programming and think it's cool. Multithreaded applications are hard to design correctly and difficult to debug from an application writer's point of view. Your simplistic statements belie the fact that you must be an inexperienced programmer because in the real world people don't have time to start om mammoth projects simply to help with a medium sized one. Adding threading support to gdb is more difficult than writing a multithreaded application that uses the pthread library unless the application is very complex like a compiler, relational database management system or a web browser.
Your comment is like telling someone to hack garbage collection into C when they complain about memory leaks instead of simply pointing the person to Purify or BoundsChecker.
Grabel's Law
A better way is to avoid bugs in the first place. And for multithreaded programs, a message passing paradigm is generally the best approach in my experience.
I also use assertions and consistency check liberally, so that problems are caught close to where they occur. As a result, the few serious bugs that I have had were usually solvable by inspection of the source code alone.
If you are looking for languages that make avoiding bugs easier, look beyond C++. Java is slightly better than C++ for multithreaded programs because errors tend to be more localized, but Java doesn't have much in the way of useful higher-level abstractions for building multithreaded programs. The currently conceptually best languages for multithreaded programs, in my opinion, are SML/NJ, OCAML, and Erlang.