Slashdot Mirror
Debugging in Plain English?
sameerdesai writes "CNN is carrying a story about Researchers from Carnegie Melon: Myers and a graduate student, Andrew Ko, have developed a debugging program that lets users ask questions about computer errors in plain English: Why didn't a program behave as expected? I guess with recent exploits and bugs that were found this will soon be a hot research topic or tool in the market." We recently did a story about revolutionary debugging techniques; the researchers' website has some papers and other information.
Untill I can have a full conversation with a computer (a la the Turing effect, not the limited versions that Alice et. al. can accomplish) I'll be happy with source code, thank you very much. It's just another layer blocking me from the code anyway (read In the Beginning... lately?).
All these talks of "revolutionary" debugging techniques bother me a little. There's only one debugging technique, and that's the debugger's skill and experience. Debuggers, traces, logs and other printf()s and LEDs flashing are just tools.
Andrew Ko's invention is just another tool. It won't do the debugging for you. Just like modern cars have diagnostic computers, but somehow it appears you still have to fork off $30/hr for the workmanship to get it fixed...
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
Article says adding Whyline to java makes it 10x more complex. Seems to me like just another example of Computer Science grad students trying to justify their existence.
When I think about some of the bugs I have found (and coded), the Whyline approach seems very far-fetched. The degree of self-awareness (introspection) required by something like this makes it seem like the program would be able to avoid the trap in the first place. It would require the "analyst" or "observer" module to observe not only a stack trace and PC trail, but also would require the module to understand what is supposed to occur.
I don't expect this early research tool to catch all of these, but I'd like to hear the researchers' response on how their system might (after years of development) answer questions about some of these bugs:
- Why did the Mars Pathfinder software deadlock (priority inversion)
- Why did the Mars Polar Lander crash (improper state management)
- Why did the Ariane 5 blow up (arithmetic overflow in a register)
- Why did the Patriot missiles miss in the 1991 Gulf War (accumulated time error)
- Why did a radiation therapy machine zap patients with the wrong doses (inconsistent state between GUI display and internal software state)
I'm sure there are some others on comp.risks and elsewhere.
Another point: this approach is still "just" a testing tool. In other words, it can only find errors on paths that have actually been taken in tests, which means the testing program must cover enough cases to generate the runtime errors in the first place. In all of the above cases, it was the testing program that permitted the bugs to be fielded.
Programmer: "Why didn't it work?"
Computer: "How should I know, I just do what I'm told."
"I use a Mac because I'm just better than you are."
This reminds me of back in approx 1985 or so, someone "invented" a human language programming environment called "The Last One" or something like that. This would supposedly make it simple to write programs without having to learn C etc. Some programmers quaked in their boots. However the real issue with programming is learning the contructs, not the language (ie. if you understand what a linked list is, then writing one in C vs Pascal is pretty simple). Anyone that thinks that programming in English is easier is seriously misunderstanding programming. The ultimate test is to look at the languages that have survived: The more "human readable" languages like COBOL have not survived, but the more cryptic ones like C have. The big "killer app" for making programming simple for the non-programmer was the spreadsheet and that's hardly a natural language.
Now debugging is pretty much the same deal. Verbose English debugging interfaces might make it simple to learn to do very basic debugging, but once you get into things a bit deeper (and get more experienced), English becomes a huge liability and you'll be wishing for more concise and expressive languages.
Engineering is the art of compromise.
In the products I'm involved with I often get stupid reports from the field of the form "framing error causes unit to reset". When I get one of these, first thing I do is get back to the user and figure out exactly what they saw and what heppened withouth them trying to figure out the symptoms. In the "framing error" problem, what was really happening was that a power glitch was being caused when the RS232 cable was attched (because of bad grounding). This caused a reset. However, the user was a "super user" who knew bad things happen to serial data when you plug/unplug cables. One of the buzzwords he knew about was a framing error. So he "half solved" the problem by saying that a framing error caused the problem.
There is a big difference between observing and fixing problems. QA is about observing, not fixing, problems. It is better to provide a good way for users to make accurate error reports (eg. backtrace/log/whatever) than try have them try explain what went wrong.
Engineering is the art of compromise.