Slashdot Mirror
Ask Slashdot: How To Get Non-Developers To Send Meaningful Bug Reports?
DemonGenius writes "I'm in the midst of a major rollout of one of our primary internal applications at work and we have a beta version available for all the staff to use. The problem here is most of the staff don't know how to send reports meaningful enough to get us devs started on solving their problems without constant back and forth correspondence that wastes both developer time and theirs. Some common examples are: screenshots of the YSOD that don't include the page URL, scaled screenshots that are unreadable, the complaint that wants to be a bug report but is still just a complaint, etc. From the user's perspective, they just send an email, but that email registers in our tracking system. Any thoughts on how to get the non-devs sending us descriptive and/or meaningful reports? Does anyone here have an efficient and user-friendly bug tracking system/policy/standard at their workplace? How does it work?"
Make your software send it...
You can not teach the world, so why try?
A more pressing question is how to get developers to stop ignoring bug reports.
Seriously, don't just keep blowing off the important user bugs for multiple release cycles. Once your bugs have been blown off for 6 months you stop submitting new ones.
I admittedly didn't read the full thread so far, and I know at least some of this was covered.
But have either a web site or a bug reporting facility in your app filled with a template that the user can fill out. Something like:
Brief description of problem:
Steps to reproduce the problem:
What you saw:
include crash logs, snapshots, etc
What you expected to see:
Severity: Crash/UI/Performance/etc..
Version of app/configuration info (filled in AUTOMATICALLY if possible).
and of course do NOT make them fill in every single field (if it's not reproducible, they won't have exact steps.. but you might be able to narrow down on the problem with multiple different bug reports).
Personally, I hate these templates in our bug system, and cmd-A, and type over them, since I already know what kind of info to put in.. but it is useful for regular users (or even developers, if there are special steps, like special debugging tools for your specific app/tools).
I've had success by educating a small group of business users to serve as triagers of bug reports - it's a lot easier to have them enforce your rules about what a validated bug looks like, particularly when you've armed them with a template to check against (example - without a screenshot including a URL, time of incident, browers/OS, the report gets sent right back at them) or have them validate the issue on their own before an engineer even looks at it. Eventually the business users get good enough to aggregate incidents together to locate true bugs and also can shoulder some of the prioritization burden. Remember that its in their rational best interest to care about software quality even if they can't speak about it in a sophisticated way.
We have also posted examples of good bug reports and a wall of shame for bad ones - if you can get incentives together (or punish those that resist reeducation) that helps too.
From the question:
It's an internal app so just have the app log everything that the user does in that app.
Then, when the user calls to say there is a problem, the dev team can pull the logs from that machine and recreate the exact sequence of events.
And don't worry about the logs becoming too large. If the dev cannot figure that out then there are larger problems there.
Also, have the app check the versions of the libraries and such in the OS.
Give them a "Report bug" clicky that brings up a form for them to fill out and which automatically extracts relevant information.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
You have to coach them. They don't really understand what you need.
When I get a email from someone about a bug, I go meet with them. I ask them all the questions I think may be relevant. What were they doing, how were they doing it. Were there any extra small steps or actions that jump out. Sometimes I explain why I'm asking certain questions and relate them back to previous bugs or issues.
I think what you need is someone to be the go between. Get a tester to receive those emails, recreate the issue, then file a bug report. Don't allow the end user to file bug reports directly into your system. It will mess with your tracking data. A high number of worthless bug reports closed quickly may look good in the reports but does not help anyone.
Im a gamer, not a grammer major. This post is full of spelling and grammer mistakes.
We've had success by making it very clear what information we need. We've given them the baseline information (username, account, environment, etc) that they have to send with every single bug, and we've made it clear that the steps they need to give us need to be something we can follow exactly and get the error every time.
Obviously there are exceptions, and our user base includes people who are actually technical enough to exercise judgement over what we need, but for the most part it's just training and education. They can't know what information we need, so we need to tell them. It's a hard problem, but not unsolvable.
If the software is working the way its designers and architects intended for it to function, there's no bug.
absolutely correct, unless you consider usability to be a feature.
if you do, then every time a user is confused about your application, it's a bug. it may not be easy to fix. but if you don't consider it a bug, track it, prioritize it and potentially fix it, then you don't care about usability.
> If it's a C++ app, then sure, having a built-in crash reporting mechanism shouldn't be that hard to build in
That's precisely what I do. The default exception handling routine sends an email to me with the app, version, username, machine id, error description, call stack, and any useful data that that I saw fit to include while coding. It has saved me mountains of pain over the years, and also fuels my reputation as the all-seeing eye.
Dyolf Knip
Finding a decent way to look through GB of logs to find something that happened a week ago is going to be MUCH more of a challenge
I'm running a website with 2-3k concurrent users; using web server access logs for read-only users, and a more detailed custom logging system for any interactions that any user makes, only generates ~100GB of logs per month. In the grand scheme of things, this is a tiny amount (grep from the command line is still perfectly adequate). For someone who can dedicate a day or two to writing a log browser then I'd expect the search to be even better, with graphs and automated anomaly reports and such (As someone who actually has done this, I'm not sure whether to find your comment of unrealism worrying or complimenting :P).
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
> If it's a C++ app, then sure, having a built-in crash reporting mechanism shouldn't be that hard to build in
That's precisely what I do. The default exception handling routine sends an email to me with the app, version, username, machine id, error description, call stack, and any useful data that that I saw fit to include while coding. It has saved me mountains of pain over the years, and also fuels my reputation as the all-seeing eye.
I do much the same but include credit card information, mother's maiden name and social security number.
I've had enough of software folks looking for requirements and now good "bug reports". Users/Customers do not have requirements they have PROBLEMS. It is up to the developers to create tools that solve those problems. The user doesn't even know what's possible or easy or hard to implement, so they can't tell you what the requirements are. Same for bugs. The user can't tell you details about which internal parts of your code have a bug, they tell you (at best) what they were doing when something happened. They don't know a crash from accidentally closing a dialog box. The solution - especially for internally developed software - is to go visit the users and have a conversation. Talk to them. Understand how they view your software and you can then understand their explanation of what it did wrong. This will also give you insight into how to make it better. Stop expecting detailed bug reports and start understanding users.