Slashdot Mirror
Ask Slashdot: How To Get Non-Developers To Send Meaningful Bug Reports?
DemonGenius writes "I'm in the midst of a major rollout of one of our primary internal applications at work and we have a beta version available for all the staff to use. The problem here is most of the staff don't know how to send reports meaningful enough to get us devs started on solving their problems without constant back and forth correspondence that wastes both developer time and theirs. Some common examples are: screenshots of the YSOD that don't include the page URL, scaled screenshots that are unreadable, the complaint that wants to be a bug report but is still just a complaint, etc. From the user's perspective, they just send an email, but that email registers in our tracking system. Any thoughts on how to get the non-devs sending us descriptive and/or meaningful reports? Does anyone here have an efficient and user-friendly bug tracking system/policy/standard at their workplace? How does it work?"
Make your software send it...
You can not teach the world, so why try?
Either that, or send them all to reeducation camps.
Try not to stress about it, it's hopeless.
Sent from my PDP-11
A more pressing question is how to get developers to stop ignoring bug reports.
Seriously, don't just keep blowing off the important user bugs for multiple release cycles. Once your bugs have been blown off for 6 months you stop submitting new ones.
So it is an ASP.NET app? Seriously, override application_error in the global.asax, and just send the emails with detailed error dumps and messages straight to you, in real time.
and find out what you think might be wrong, instead of what is wrong.
Engage the users. It's your problem, not theirs.
I admittedly didn't read the full thread so far, and I know at least some of this was covered.
But have either a web site or a bug reporting facility in your app filled with a template that the user can fill out. Something like:
Brief description of problem:
Steps to reproduce the problem:
What you saw:
include crash logs, snapshots, etc
What you expected to see:
Severity: Crash/UI/Performance/etc..
Version of app/configuration info (filled in AUTOMATICALLY if possible).
and of course do NOT make them fill in every single field (if it's not reproducible, they won't have exact steps.. but you might be able to narrow down on the problem with multiple different bug reports).
Personally, I hate these templates in our bug system, and cmd-A, and type over them, since I already know what kind of info to put in.. but it is useful for regular users (or even developers, if there are special steps, like special debugging tools for your specific app/tools).
I've had success by educating a small group of business users to serve as triagers of bug reports - it's a lot easier to have them enforce your rules about what a validated bug looks like, particularly when you've armed them with a template to check against (example - without a screenshot including a URL, time of incident, browers/OS, the report gets sent right back at them) or have them validate the issue on their own before an engineer even looks at it. Eventually the business users get good enough to aggregate incidents together to locate true bugs and also can shoulder some of the prioritization burden. Remember that its in their rational best interest to care about software quality even if they can't speak about it in a sophisticated way.
We have also posted examples of good bug reports and a wall of shame for bad ones - if you can get incentives together (or punish those that resist reeducation) that helps too.
From the question:
It's an internal app so just have the app log everything that the user does in that app.
Then, when the user calls to say there is a problem, the dev team can pull the logs from that machine and recreate the exact sequence of events.
And don't worry about the logs becoming too large. If the dev cannot figure that out then there are larger problems there.
Also, have the app check the versions of the libraries and such in the OS.
Or they will just stop sending bug reports. If you are an ass to your users then they will weigh how much they care about the app working vs how much of a pain it is to deal with you. If the app is relatively unimportant to them or you are a very big ass then they will stop sending bug reports (and possibly stop using your app). In which case the dev loses.
YSOD? Maybe you need to be more clear to your users. I don't know what YSOD is and I work in the industry... Make sure your users understand what a bug report is, and how it helps to give as much information as possible. Avoid using terms they won't understand, and assume they don't know what you want. Some users will try to help if you tell them what you need, but give up if they feel like they have to figure it out on their own.
Give them a "Report bug" clicky that brings up a form for them to fill out and which automatically extracts relevant information.
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
You have to coach them. They don't really understand what you need.
When I get a email from someone about a bug, I go meet with them. I ask them all the questions I think may be relevant. What were they doing, how were they doing it. Were there any extra small steps or actions that jump out. Sometimes I explain why I'm asking certain questions and relate them back to previous bugs or issues.
I think what you need is someone to be the go between. Get a tester to receive those emails, recreate the issue, then file a bug report. Don't allow the end user to file bug reports directly into your system. It will mess with your tracking data. A high number of worthless bug reports closed quickly may look good in the reports but does not help anyone.
Im a gamer, not a grammer major. This post is full of spelling and grammer mistakes.
If the software is working the way its designers and architects intended for it to function, there's no bug.
absolutely correct, unless you consider usability to be a feature.
if you do, then every time a user is confused about your application, it's a bug. it may not be easy to fix. but if you don't consider it a bug, track it, prioritize it and potentially fix it, then you don't care about usability.
> If it's a C++ app, then sure, having a built-in crash reporting mechanism shouldn't be that hard to build in
That's precisely what I do. The default exception handling routine sends an email to me with the app, version, username, machine id, error description, call stack, and any useful data that that I saw fit to include while coding. It has saved me mountains of pain over the years, and also fuels my reputation as the all-seeing eye.
Dyolf Knip
Finding a decent way to look through GB of logs to find something that happened a week ago is going to be MUCH more of a challenge
I'm running a website with 2-3k concurrent users; using web server access logs for read-only users, and a more detailed custom logging system for any interactions that any user makes, only generates ~100GB of logs per month. In the grand scheme of things, this is a tiny amount (grep from the command line is still perfectly adequate). For someone who can dedicate a day or two to writing a log browser then I'd expect the search to be even better, with graphs and automated anomaly reports and such (As someone who actually has done this, I'm not sure whether to find your comment of unrealism worrying or complimenting :P).
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
> If it's a C++ app, then sure, having a built-in crash reporting mechanism shouldn't be that hard to build in
That's precisely what I do. The default exception handling routine sends an email to me with the app, version, username, machine id, error description, call stack, and any useful data that that I saw fit to include while coding. It has saved me mountains of pain over the years, and also fuels my reputation as the all-seeing eye.
I do much the same but include credit card information, mother's maiden name and social security number.
I've had enough of software folks looking for requirements and now good "bug reports". Users/Customers do not have requirements they have PROBLEMS. It is up to the developers to create tools that solve those problems. The user doesn't even know what's possible or easy or hard to implement, so they can't tell you what the requirements are. Same for bugs. The user can't tell you details about which internal parts of your code have a bug, they tell you (at best) what they were doing when something happened. They don't know a crash from accidentally closing a dialog box. The solution - especially for internally developed software - is to go visit the users and have a conversation. Talk to them. Understand how they view your software and you can then understand their explanation of what it did wrong. This will also give you insight into how to make it better. Stop expecting detailed bug reports and start understanding users.