Debugging

dwheeler writes "It's not often you find a classic, but I think I've found a new classic for software and computer hardware developers. It's David J. Agan's Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems." Read on for the rest. Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems author David J. Agans pages 192 publisher Amacom rating 9 reviewer David A. Wheeler ISBN 0814471684 summary A classic book on debugging principles

Debugging explains the fundamentals of finding and fixing bugs (once a bug has been detected), rather than any particular technology. It's best for developers who are novices or who are only moderately experienced, but even old pros will find helpful reminders of things they know they should do but forget in the rush of the moment. This book will help you fix those inevitable bugs, particularly if you're not a pro at debugging. It's hard to bottle experience; this book does a good job. This is a book I expect to find useful many, many, years from now.

The entire book revolves around the "nine rules." After the typical introduction and list of the rules, there's one chapter for each rule. Each of these chapters describes the rule, explains why it's a rule, and includes several "sub-rules" that explain how to apply the rule. Most importantly, there are lots of "war stories" that are both fun to read and good illustrations of how to put the rule into practice.

Since the whole book revolves around the nine rules, it might help to understand the book by skimming the rules and their sub-rules:

1. Understand the system: Read the manual, read everything in depth, know the fundamentals, know the road map, understand your tools, and look up the details.
2. Make it fail: Do it again, start at the beginning, stimulate the failure, don't simulate the failure, find the uncontrolled condition that makes it intermittent, record everything and find the signature of intermittent bugs, don't trust statistics too much, know that "that" can happen, and never throw away a debugging tool.
3. Quit thinking and look (get data first, don't just do complicated repairs based on guessing): See the failure, see the details, build instrumentation in, add instrumentation on, don't be afraid to dive in, watch out for Heisenberg, and guess only to focus the search.
4. Divide and conquer: Narrow the search with successive approximation, get the range, determine which side of the bug you're on, use easy-to-spot test patterns, start with the bad, fix the bugs you know about, and fix the noise first.
5. Change one thing at a time: Isolate the key factor, grab the brass bar with both hands (understand what's wrong before fixing), change one test at a time, compare it with a good one, and determine what you changed since the last time it worked.
6. Keep an audit trail: Write down what you did in what order and what happened as a result, understand that any detail could be the important one, correlate events, understand that audit trails for design are also good for testing, and write it down!
7. Check the plug: Question your assumptions, start at the beginning, and test the tool.
8. Get a fresh view: Ask for fresh insights, tap expertise, listen to the voice of experience, know that help is all around you, don't be proud, report symptoms (not theories), and realize that you don't have to be sure.
9. If you didn't fix it, it ain't fixed: Check that it's really fixed, check that it's really your fix that fixed it, know that it never just goes away by itself, fix the cause, and fix the process.

This list by itself looks dry, but the detailed explanations and war stories make the entire book come alive. Many of the war stories jump deeply into technical details; some might find the details overwhelming, but I found that they were excellent in helping the principles come alive in a practical way. Many war stories were about obsolete technology, but since the principle is the point that isn't a problem. Not all the war stories are about computing; there's a funny story involving house wiring, for example. But if you don't know anything about computer hardware and software, you won't be able to follow many of the examples.

After detailed explanations of the rules, the rest of the book has a single story showing all the rules in action, a set of "easy exercises for the reader," tips for help desks, and closing remarks.

There are lots of good points here. One that particularly stands out is "quit thinking and look." Too many try to "fix" things based on a guess instead of gathering and observing data to prove or disprove a hypothesis. Another principle that stands out is "if you didn't fix it, it ain't fixed;" there are several vendors I'd like to give that advice to. The whole "stimulate the failure, don't simulate the failure" discussion is not as clearly explained as most of the book, but it's a valid point worth understanding.

I particularly appreciated Agans' discussions on intermittent problems (particularly in "Make it Fail"). Intermittent problems are usually the hardest to deal with, and the author gives straightforward advice on how to deal with them. One odd thing is that although he mentions Heisenberg, he never mentions the term "Heisenbug," a common jargon term in software development (a Heisenbug is a bug that disappears or alters its behavior when one attempts to probe or isolate it). At least a note would've been appropriate.

The back cover includes a number of endorsements, including one from somebody named Rob Malda. But don't worry, the book's good anyway :-).

It's important to note that this is a book on fundamentals, and different than most other books related to debugging. There are many other books on debugging, such as Richard Stallman et al's Debugging with GDB: The GNU Source-Level Debugger. But these other texts usually concentrate primarily on a specific technology and/or on explaining tool commands. A few (like Norman Matloff's guide to faster, less-frustrating debugging ) have a few more general suggestions on debugging, but are nothing like Agans' book. There are many books on testing, like Boris Beizer's Software Testing Techniques, but they tend to emphasize how to create tests to detect bugs, and less on how to fix a bug once it's been detected. Agans' book concentrates on the big picture on debugging; these other books are complementary to it.

Debugging has an accompanying website at debuggingrules.com, where you can find various little extras and links to related information. In particular, the website has an amusing poster of the nine rules you can download and print.

No book's perfect, so here are my gripes and wishes:

1. The sub-rules are really important for understanding the rules, but there's no "master list" in the book or website that shows all the rules and sub-rules on one page. The end of the chapter about a given rule summarizes the sub-rules for that one rule, but it'd sure be easier to have them all in one place. So, print out the list of sub-rules above after you've read the book.
2. The book left me wishing for more detailed suggestions about specific common technology. This is probably unfair, since the author is trying to give timeless advice rather than a "how to use tool X" tutorial. But it'd be very useful to give good general advice, specific suggestions, and examples of what approaches to take for common types of tools (like symbolic debuggers, digital logic probes, etc.), specific widely-used tools (like ddd on gdb), and common problems. Even after the specific tools are gone, such advice can help you use later ones. A little of this is hinted at in the "know your tools" section, but I'd like to have seen much more of it. Vendors often crow about what their tools can do, but rarely explain their weaknesses or how to apply them in a broader context.
3. There's probably a need for another book that takes the same rules, but broadens them to solving arbitrary problems. Frankly, the rules apply to many situations beyond computing, but the war stories are far too technical for the non-computer person to understand.

But as you can tell, I think this is a great book. In some sense, what it says is "obvious," but it's only obvious as all fundamentals are obvious. Many sports teams know the fundamentals, but fail to consistently apply them - and fail because of it. Novices need to learn the fundamentals, and pros need occasional reminders of them; this book is a good way to learn or be reminded of them. Get this book.

If you like this review, feel free to see Wheeler's home page, including his book on developing secure programs and his paper on quantitative analysis of open source software / Free Software. You can purchase Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Don't forget...

[SCO$699FeeTroll]

...to pay your $699 licensing fee you cock-smoking teabaggers.

debug this

[Anonymous Coward]

I received the email first thing in the morning from the IT department. Our network would be undergoing a major overhaul to correct the ad hoc growth it had experienced in the last year, and starting next week Internet access would be sporadic. There would also be a new firewall and security measures, replacing the old OpenBSD system I'd managed to get installed last Spring. Happy for the heads-up, I went to work right away to make sure Linux had no place on our network. This was not the first time that I had faced this threat.

Since the Open Source Mullet had been canned, a new threat had arisen at my workplace: the Fat Perl Hacker had assumed most of the Open Source Mullet's system and network administration duties, and it was no mystery to anyone at my workplace that he had a hard-on for Linux tucked away under his enormous, cascading gut. Since he was a major suck-up and workaholic, he had a lot more credibility than the Open Source Mullet this would be a real challenge for once. Dealing with the Open Source Mullet had been cake.

One day about a year ago our network guy gets asked to draw up firewall plans for this subnet of servers we have. Our network guy was your typical GNU-slinger save that he had a cascade of flowing hair down the back of his head and not a beard hanging from his face. And yeah, you can guess what he thought those firewalls were gonna run. Fast forward two days. I'd caught wind of the plans and had charts, graphs, and comparisons written up detailing OpenBSD and Linux security. Since this GNU guy had a mullet and dressed like a slob, I got taken seriously. Not to mention my data, impenetrable by any hippy "logic." OpenBSD was the more secure, even to the beancounters and idtiot management. So thanks to me, our firewalls happily run OpenBSD and not Linux, which would have buffer-overflowed into no-man's land every other hour. The Open Source Mullet gives me a lot of dirty looks lately.

That night, I went to work on my strategy. First, I would document the changes in Linux and OpenBSD since a year ago when we last went with a security plan. Linux was still at version 2.4, while OpenBSD had raced from version 2.8 to 3.1 a major revision! This was good so far, and I included the relevant diffs for each. I wondered what the Fat Perl Hacker was up to and pushed ahead with my preparations.

Tuesday morning, I went to talk with the VP of Operations, who had final say on the network project. I wouldn't leave anything to chance. But after chatting with him for a few minutes, I learned of a major monkey-wrench I hadn't expected: instead of a Unix firewall system, he was planning on installing a dedicated firewall box running Windows XP. Thankful for my fortuitous social engineering, I went back to my desk and began making over my strategy to deal with this new threat. Not only would I have to deal with Linux, I'd have to eschew the Windows option now.

Sitting in front of my iBook after work, I realized that taking on Windows XP in the same manner I was going to deal with Linux would be foolish if not wasteful. Obviously the Windows option was not about numbers, anecdotes, or experience. It was a bean-counting decision and all of the security statistics in the world wouldn't matter. Since I hadn't the foggiest about how our accountants viewed the whole operation and didn't have time to learn, I'd have implement a rapid-fire real-life assault on the Windows box, which was sitting on the VP's desk awaiting its place on the network. It was time to put on my Black Hat, and that night I stayed up until 02:00 researching Windows XP vulnerabilities. Linux would have to wait.

With just two days before the network changeover was to take place, I marched into work Wednesday morning knowing that what I did in the next few hours would decide the fate of our network security. To my surprise, just moments after I had sat down, the Fat Perl Hacker asked me to join him for a cigarette outside away from the ears and eyes of the

Re:Kids these days....

[incubusnb]

why is this modded funny? it should be informative because its true

Please, someone debug +1, True

[Anonymous Coward]

The World's Most Dangerous Leader

Cheers,
K. Trout

Re:The Three R's of Windows Debugging

[Anonymous Coward]

How, exactly, do you stick a non-physical object into an arsehole?