Slashdot Mirror
How Do You Get Better Bug Reports From Users?
itwbennett writes "You can try to train them, you can try to streamline or automate the process, you can demand that all bug reports go through a middleman (i.e., a QA tester) or you can throw up your hands and accept that users will forever submit bug reports that in no way help you solve the problem. Like the stages of grief, you've probably tried or experienced all of these at some point. But have you found any approach that really works for getting useful bug reports from your users?"
If your users are, say, someone who works in a different department of the same company as you do, then I've found that the best approach is to positively encourage them to send me reports. If I can't figure it out from the report, I'll even walk over and ask them to show me exactly what they did to cause the bug. When I do that, I give them some tips on how to write a report that explains that more clearly: "I logged into the website, clicked 'Foo', entered 'baz' into this box, then clicked 'Submit'. I got an error page instead of this other page I was expecting."
If your users are mostly developers, then by all means give them easy access to stack traces, memory dumps, etc.
If your users are mass-market end users, then include with your application a system that tracks user actions and sends it (along with a crash dump) to you at the user's request if there's a problem.
If your users are calling in to your company to complain, plan on using some kind of desktop sharing software so that the user and you can see exactly what happened.
If your users are dumber than a pack of hammers, then I have no answers for you.
I am officially gone from
In your bug tracking system, provide a bug template they need to fill in.
Encountered situation:
Expected situation:
Error messages, if any:
Steps to reproduce (please provide screenshots if appropriate):
If the template is not fully filled out, simply throw it out with "insufficient information/cannot reproduce". Eventually your users will grow up.
Also, there's nothing wrong with walking up to a user's desk (if physically possible!) and ask them to show you what they did.
Your users aren't code masters and never will be.
It doesn't involve users to be code masters, it just involves them engaging their brain a bit. I frequently get bug reports along the lines of "something broke last week, came up with some error (I don't remember what), but I rebooted it and its fine for now; please fix it so it doesn't happen again". You don't have to be a "code master" to figure out that reporting a bug and not actually tell me _what_ broke, what the error was or let me log into a system that is currently exhibiting the problem so I can look myself is not going to be condusive to me fixing things.
And this stuff happens again and again with the same customers... "one of our users is having a problem accessing some websites, please can you fix it?" - ok, so I have to go back and ask "which user" and "which websites", if I'm lucky the customer will give me this information, if I'm unlucky I get "I didn't ask". A week later I'll get an almost identical problem report from the same person about a different (but extremely similar) problem, and again none of the information I ask for _every_ time is included.
Also the great one that comes up occasionally is "this has been broken for a month and you haven't fixed it yet!".. well, if you'd actually told me that there was a problem I might've known to look into it, but since this is the first I've heard of it...
Right. Because users are filing bug reports. They're calling for customer service/support. You get good bug reports by having good people answer the phone. I'll accept, "something broke," from a user. However, I find it inexcusable that a Helpdesk rep sends me a bug report that says the same thing or, worse, asks me to call the user for details. That being said, if you can't rely on Helpdesk or the user, then your software should be able to rollup everything you need to resolve the issue.
Requiring a user, after the fact, to recall an error message is futile. They simple have seen to many varied ones and their brain goes 'oh an error message' not 'oh a 504 error' or 'oh a invalid data type error'.
Believe it or not a user that doesn't remember the error message is not the worst kind of user.
I have some users that love translating text errors into numeric error themselves. Any time a page doesn't load, it's a 404. So that's what they report. "I'm trying to connect to thisdomaindoesntexist.com and I'm getting a 404."