Slashdot Mirror
How Do You Get Better Bug Reports From Users?
itwbennett writes "You can try to train them, you can try to streamline or automate the process, you can demand that all bug reports go through a middleman (i.e., a QA tester) or you can throw up your hands and accept that users will forever submit bug reports that in no way help you solve the problem. Like the stages of grief, you've probably tried or experienced all of these at some point. But have you found any approach that really works for getting useful bug reports from your users?"
When writing error messages, don't have it spit out something sensible. Have it spit out something completely crazy but memorable, which you can then grep the code for. Something along the lines of "The Cake has hit the Fan" or "The Chickens are eating Pie". This improves the odds the user will remember it and report it correctly, giving you some hope of finding where the bug is in the code.
"Actually, I enjoyed this in the same vague, horrible way I enjoyed the A-Team" P. Opus
On every page of the site that I did for this company, in the upper right corner, there is a button labelled "BUG!". Click on it and a dialog box comes up that says simply "What's wrong with this page?" and has a big blank area to fill in. The dialog box has a "Submit" button.
What is not visible is that the JavaScript code for the BUG! button is grabbing all the information it can from the browser itself - what the current URL is, all the global JavaScript variables, name of the current logged-in user, time and date, browser type, web page contents, etc. All grabbed automatically.
And all stored in a special file on the server. The only thing the user has to tell me is what she doesn't like about that page.
The BUG! button was invented - by me - as a reaction to Bugzilla, which seems to be designed to keep users from reporting problems. I can ignore pointless bug reports, but I want to hear everything.
You can reproduce the issue by getting their keystroke history. File an FOIA request with the NSA.
- Record all their actions on a self overwriting one hour long file.
- Give them a "one button" way of reporting a bug. The button saves the user and the time, then waits for five minutes and then sends you the recorded actions file.
It's simple to develop and gives you a lot of information. It might be illegal in some countries.
Requiring a user, after the fact, to recall an error message is futile. They simple have seen to many varied ones and their brain goes 'oh an error message' not 'oh a 504 error' or 'oh a invalid data type error'.
Believe it or not a user that doesn't remember the error message is not the worst kind of user.
I have some users that love translating text errors into numeric error themselves. Any time a page doesn't load, it's a 404. So that's what they report. "I'm trying to connect to thisdomaindoesntexist.com and I'm getting a 404."