Slashdot Mirror
Ask Slashdot: How Does Your Team Track And Manage Bugs In Your Software?
Slashdot reader jb373 is a senior software engineer whose team's bug-tracking methodology is making it hard to track bugs.
My team uses agile software methodologies, specifically scrum with a Kanban board, and adds all bugs we find to our Kanban board. Our Kanban board is digital and similar to Trello in many regards and we have a single list for bugs... We end up with duplicates and now have a long list to try and scroll through... Has anyone run into a similar situation or do things differently that work well for their team?
The original submission ends with one idea -- "I'm thinking about pushing for a separate bug tracking system that we pull bugs from during refinement and create Kanban cards for." But is there a better way? Leave your own experiences in the comments. How does your team track and manage bugs in your software?
The original submission ends with one idea -- "I'm thinking about pushing for a separate bug tracking system that we pull bugs from during refinement and create Kanban cards for." But is there a better way? Leave your own experiences in the comments. How does your team track and manage bugs in your software?
We count the cases in production. That's how we track bugs. :-)
Bugzilla. It sucks. But that is what it is there for.
Or some such nonsense.
JIRA, plus great integration into 'fr agile' methodology.
We begin fixing any bug as soon as it is discovered.
We use Redmine with an assortment of plugins.
(Recurring Tasks, Base Deface, Base Rspec, Customise Core Fields, DMSF, Edit Custom Fields, Git Remote, Wiki Extensions, Redrisk, Scrum)
Works great for us by integrating development and maintenance. + it's free and sits on its own little linux VM.
46137
My team gets around this whole issue by simply writing bug-free code from the get-go, and moving on. Saves a lot of hassle!
But not the junior engineers who are copying and pasting code from Stack Exchange?
To track our bugs. No need for duplication of effort.
Partly though I think because if the close it and it isn't fixed,
it's their problem rather than the sw devs problem.
If they haven't either closed it or marked it as not fixed, isn't it still their problem? And how do you release with a heap of untested code?
We implemented a rule where whenever a bug is found in code, the coder has to go lick the bug zapper. It only took three hours... before we killed all the developers. It didn't fix our existing bugs but we haven't had any new bugs (or code) since! ;)
Anons need not reply. Questions end with a question mark.
excel for the long list. notepad for my personal (short term) list.
Scott
On our projects, we simply count the lines of code that have no bugs at all. It takes far less time, and we can keep track of the good lines on the whiteboard in the conference room.
A republic cannot succeed till it contains a certain body of men imbued with the principles of justice and honour.
Blame it on the other team.
Table-ized A.I.
Depending on organization your let someone confirm the bug and avoid dups, then you put it on the Kanban board. Alternatively you have on inside of the team confirming it.
But: why are you mixing Kanban with Scrum in the same team? That does not really make sense. Unless you switch from sprint to sprint to one or the other paradigm.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
You write that you use Scrum with a kanban board, and a single list of bugs. How do you manage the list?
I don't think the problem here is necessarily the tools, but might be the process. Do you ever groom the backlog/bug list? How do you prioritise your bugs? You may wish to look a bit on the team culture and think carefully about how you are dealing with older tickets.
It also sounds like you are building up a debt of bugs, which isn't necessarily the end of the world, but you may want to ensure that you triage and process the bugs in some form.
OP needs to triage the list he has, not duplicate all his bug tracking problems. One pass to get the list cleaned up now, then implement an ongoing practice of reviewing new submissions.
I'm in operations, not development, so I only do some little scripting, but I sit right by people who are apparently honest-to-goodness software developers. This year, I've at least gotten them to accept some tiny bit of structure and (in theory) the idea of testing things somewhere other than the production systems.
Most of our code was written by brainiacs who have since left. A lot of stuff is just a "black box" to almost everyone. We rarely know how or why it works. When it stops working, we rarely know why, and if we get it to work again, we rarely know how we did it. We don't know why feature X worked on Debian but not on Red Hat in the last release - nor do we know why it now works on Red Hat but not Debian.
There's talk of a total re-architecting. That might help.
If it weren't all being paid for with black money, we'd be in trouble.
By which I mean, this isn't a collaborative process, or more precisely, it's not improved by collaboration, as so many things *aren't.*
Then, use any database that makes sense. Preferably one hosted on your local server so it's not deathly slow and/or intermittent. Tack on any UI with a minimum of interface fluff. Add keywords. Using keywords, scan periodically for duplicates. This needs to be ONE person's job, preferably done for the first hour of each day. Never, ever fire that person.
It's a human language problem. AI isn't good enough. You still need a person familiar with your code and package to review it.
It's expensive. This is software. You have to pay to play.
Please do not read this sig. Thank you.
Why? Mantis Bugtracker will do all of that for you plus provides an API for external consumption/modifications.
There is literally no reason for homegrown software anymore. Just search GitHub and you'll find 3 good solutions and 10 solutions that, all integrated together, work just as good as what you have to maintain.
Last company I was at, they shat on Jenkins because it was too slow (50 avg builds using a 2 core 4gb aws VM.. yeah) so they wrote their own. Then complained that it doesn't scale, doesn't handle errors gracefully, and they haven't had anyone to maintain the code for 2 years.
IT who don't know history are doomed to repeat it
If you do not, no amount of "magic" management and tracking will help. If you keep them low, however, you do not need a lot of tracking, every bug will be unique enough to be memorable or fixed fast enough to not need tracking.
Of course that means you need to have really good architects, designers and coders. Hard to get but worth the price they will ask.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
What ad revenue? I use adblock you fat retard.
You're the exception then. Everyone else doesn't use adblock.
OSTicket http://osticket.com
I liked Trac a lot. It integrated Wiki, ticketing and Code repository information all in one and it was easier than Trac. However, it is not new or cool so developers don't think it was hip and always looking at something else.
SCRUM and Agile had a great idea - improve the process - but often the implementation of these makes it worse. Just like religions in general .
I wait until some manager comes by my desk and asks me to stop whatever I'm working on and fix whatever bug someone just complained about.
We just open tickets describing the issue and toss it over the fence to the team in India to handle.... Why would we want to waste our time fixing bugs when we can pay somebody else to do it? ;-)
The agile way is to keep it as a backlog item until the PO feels its time to fix it.
The real way is to listen to what front line support say and fix it when the noise becomes too much.
If its not a priority for anybody, then no one cares about the bug.
You get your job done. Then you look at the bug database and choose a bug to fix. The sooner you get your job done, and the more bugs you fix, the better you look to the boss.
this question really needs to be asked?
An ouija board and a pack of hunting dogs.
They're all basically the same
RT, Bugzilla, JIRA, Traq
Pick one.
let me correct that before the shitstorm of asshats start applyling logic to words. the slashdot traffic specifically does not generate any revenue directly. slashdot actually does in theory cost me money for bandwidth, but it fits in the tier plan i pay for already.
the slashdot comments however do generate material for me to discuss on my blog, and non-slashdot people who visit the site do want content to read. it takes me hours and hours for all those books and blog posts and comments, but what else more interesting am i going to do with my time given my limited resources?
also - forget what i said a few weeks ago. i did mention that my income is not from ads but people buying things from me. that was actually two random people who within a week bought a book, and i projected out that weekly income several years forward.
It creates a unique hash so that the same error is not duplicated, replicates the all the stack that was instrumented with all the parameters and class variables, and the database accessed so far ( by Entity Framework) . Check it out at ErrorUnit.com
Lifes a game play to win!
I am more of a half-hardware, half-software person, and was involved in very different projects, each with their own ways of doing it. Here are examples which did work:
- ClearQuest (ugh) with very draconian rulesets. This is for a team that has a long history of doing things, and with at least 100+ people developing something and there always are some people who won't do the right thing. Not much tight synchronization between contributors required.
- One gigantic e-mail thread where one person summarizes open issues every week, and closes them when the right code shows up on the master branch of the git repository. This involved roughly 4 people writing the code, writing test suites, writing build systems, and dealing with customer crisis. We agreed that any overhead to setup/maintain a proper bug tracking system wasn't worth the effort given the expertise of the people involved (Verilog hardware designers). Need quite a bit of tight face-to-face synchronization, but that was fine for everybody.
- Simple Bugzilla.
- JIRA with very lax policies (anybody can create, reassign, close tickets. Basically no admin/manager at all)
- Gigantic Excel table maintained by a manager/moderator.
What I learned was that
- A tool is just a tool - what actually matters is the people. Claiming a human problem to be a methodology/workflow problem isn't going to make anything work.
- Start simple - any workflow that tries to anticipate all the possible things that can go wrong will end up being overbloated, and the methodology itself will be ignored because (again) the people doesn't wholeheartedly agree with it. Start with something bare minimal and build up as you (and everybody else) understand the culture of the team.
In this case the original submitter seems to be concerned of seeing duplicates... I'd recommend one person volunteering to be the 'duplicate cleanup' agent for a while, and let that person try doing it manually for a week or two, and observe where/how dupes get created and what the cleanup job consists of. Or, just live with the duplicates and close them as they appear, if the amount of duplicates isn't that much.
A post-it note on my desk lists all the bugs that I know about, but the testing team and users have not found yet. If they will need dedicated testing I will make a Bug or Defect issue for them, otherwise I sometimes just slip a fix in along with something else that's going to be tested.
Use of labels in Jira helps keep some of the junk findable.
I have used HP Agile Manager in combination with HP QC in ALM (or whatever the fuck) in the past and don't recommend this approach. Having the defects in a separate system that was supposed to sync across, but didn't always, made things harder not easier.
If you end up with a lot of duplicates maybe you just need to make better use of the fields in your system to keep everything sorted. HP AGM was a pile of shit but we were able to use it's tagging feature and custom views to get stuff somewhat organised.
You shared your delusional karma point total on here, care to share your analytics for your website? Sales figures?
Nope. Confidential business data.
Lots of people have responded suggesting various tools or whatnot. It is quite likely that you already have a good (perhaps not the best) tool for the job, and what you actually need to do is change the processes that humans are following.
The best way to approach this is to use the "5 whys" approach, where you state the problem (or group of problems) and ask "why" 5 times to try and work out the root causes, then address them.
Eg, Problem: We have too many duplicate error reports. Why? Because testers don't know about existing bugs before raising new ones. Why? Because the testers don't search for existing issues on the backlog before raising new ones. Why? Because they haven't been trained properly.
Other possible answers for the above train may be "our backlog doesn't make searching easy" or "testers don't have a consistent way of recording issues so its difficult to find the same issue raised by different testers".
Anyway, you can do the 5 why's process yourself, but some potential root causes and responses you can take:
You have too many bugs in your software, so it's going to be hard to get on top of them no matter what processes you put in place.
* Perhaps there are developers in your team who don't have a good understanding of the system and generate bad code -> training or peer-programming might help.
* Perhaps your fundamental architecture needs to be refactored / improved / better documented to reduce the incidence of bugs.
* Perhaps your team has too much work load or is under too much pressure to get features out on time that you don't have adequate time to address your bugs, and they're building up into an unmanageable pile.
* Perhaps people on your team don't think it's their responsibility to fix bugs in their own code or they don't think it's 'fun' - someone else's job - and other people aren't proficient in that area of the code and so take longer to fix the bugs, or create new bugs in the process.
* Perhaps your team is just more interested in pushing through to 'new' things, and don't take the time to clean up bugs as they go, building new code on top of bad.
Other problems might arise from the testers.
* Perhaps you have lots of different people testing the software and raising bugs (instead of a core team of a few testers who become intimately familiar with the software), and they all have their own writing style or way of recording bugs, making it hard for any tester to know if any other tester has already raised a bug.
* Perhaps your testers just don't talk to each other - "oh yeah, I saw something like that 2 weeks ago...".
* Perhaps your testers don't search through the existing issues before raising new ones - perhaps because the tool doesn't support it well, or simply no-ones told them its a good idea to do.
* Perhaps your software doesn't have a good way of reporting bugs - if your software crashes, having consistent and common output to allow testers to identify "crash X is the same as crash Y" means they don't need to raise a second bug to report Y, instead they could just find the report for crash X (by searching for unique output in the crash) and add their new incident there.
Another thing to consider is that maybe you just need to groom the backlog from time to time. Go through and look at *all* the bugs in one sitting, and you might quickly find duplicates that you can merge/link together. You might also find that there's a bunch of bugs that you know you'll just "never" fix because they aren't important, so these could be closed or moved elsewhere to minimise the noise in the system, letting you focus just on the important stuff that matters.
Don't expect that there is going to be a 'quick fix' from adopting a new tool. It is very likely that some of the above human processes are contributing to your unmanageable backlog, and moving to a new tool won't markedly change your performance.
I've had a meandering career which went from academia to industry programmer and back to academia. My new academic field (phylogenetics) can have a fairly high programming content, but universities don't usually have the level of organization around programming that an industrial organization does.
Shortly after I started my new job, my boss/head of lab came to me. He'd received a survey asking organizations about their programming procedures (i.e. much like this question.) Having learned about CMM (Capability Maturity Model) at my previous job, I had the perfect answer for him. "Tell them that we are a CMM level one organization".
Translated: CMM level one means everybody does whatever the heck they feel like.
Quattuor res in hoc mundo sanctae sunt: libri, liberi, libertas et liberalitas.
And how do you release with a heap of untested code?
You just build the installer, and upload the package. Exactly the same with tests, or without.
If it was different, you'd have to account for all those times where you tests, they just didn't test for the bugs.
At our organisation I'm responsible for two large (sub)systems -1 DB with more than 6E12 records and one critical online and batch document processing system- and a handful of smaller ones. We don't run any software to track bugs.
We work in a way to avoid the chores of tracking bugs by dividing our software into smaller components with clearly defined interfaces. Typically, after a release where testing wasn't what it should have been, an overseeable period of instability starts and issues are dealt with until total stability sets in. Usually one person leads the development of a component and a second person is knowledgeable.
We reengineer stuff only to improve performance or quality. As interfaces are set that's not too big of a problem. (Even though making stuff simpler and better is actually very hard.)
Not saying this is applicable to any organisation. But in short I'd say to divide your problem and to fix all bugs until all issues are fixed. Have a lead developer and a backup for each component. Have an experienced developer (one that can actually code) dividing big problems up into smartly smaller ones.
I hadn't the slightest objection to his spending his time planning massacres for the bourgeoisie... (P.G. Wodehouse)
Basically when you buy a dead tree newspaper the severity by which any bug should be addressed is related to which page it appears on. If your software is mentioned on page 15 then you're good. If you hit the front page then you have a critical vulnerability and you should move to addressing the bug.
We're an agency.
I'm leaving at the end of this month. :-)
We suffer more in our imagination than in reality. - Seneca
Hey, you. The customer complained about a bug, go contact him and solve it ASAP.
What we do is first classify variations of bugs. During development testing cycles, any defect found in testing becomes a defect subtask, and is prioritized right along with any other work subtasks under our current feature.
After code is deployed into production, any reported defects are captured as a higher level defect ticket (not subtask) against the application/component, not necessarily a specific project effort. If the defect is a true production issue that must be corrected, necessary work subtasks are attached to the defect, it is corrected, tested and deployed as an independent work effort.
If the defect is either acceptable until the next release or working as designed but not intended (not really a defect), then either the defect ticket is migrated to become a new change request in the project pipeline or the defect is closed and a change request ticket is created referencing back to the defect as part of the story's requirements.
We use JIRA. All items that are believed to require a DEV ticket must get past a team of gatekeepers (Triage Team.)
Tickets are reviewed for completeness, reproduction details, any logs that may be required and sample user(s) that can be used to reproduce the issue.
The Triage team searches for duplicates and attempts to replicate and then either promotes the ticket to DEV, sends it back to the requester for more detail or closes it as a dupe and links the original.
Any support tickets that come in with the same issue are linked to the DEV and once the DEV is closed, tickets are set to Pending Rollout.
The process sucks. Support hates it because it is often a struggle to get through the obvious bugs and DEV likes it because they get less duplication.
Your setup will work fine if the definition of a story includes all the tests that will be tested as part of validating the story, and the definition of done includes all tests passing, and preferably automated. Your stories will be higher quality the first time around.
It works fine, it's easy to understand, it doesn't get in the way, and it more-or-less does what we want, so we can spend our time wrestling with the bugs themselves, rather than fighting the bug-tracking system. We've used it for about 10 years now, and we're happy with it.
I don't care if it's 90,000 hectares. That lake was not my doing.
Stick with whatever tool your team knows how to use well... switching tools too often isn't going to make up for the minor efficiency boosts for upgrading your ticketing system. That said, maybe you want to use a new project / product / major release to use as an opportunity to introduce (but not test -- you've already done testing using a throwaway project, hopefully) a new ticketing system.
Kanban has a triage process, where you spend a bit of time doing some housekeeping on your tickets to find and eliminate duplicates and mark low-impact as "won't fix".
Later on, when you notice some bugs getting lots of duplicates filed, you'll come to the conclusion that those are pretty high-visibility bugs, which might possibly be a useful metric the next time you prioritize.
All you need is a decent tool and a reasonable methodology, and of course someone to track and enforce it.
I've used everything from a home-grown system to JIRA to Quality Center to TFS. I've seen basically no enforceable rules (which sucks) to ultra-precise rigid workflows (which really sucks).
We once had a severity 1 issue come up which we quickly fixed. Turned out it wasn't a sev1 by definition (no workaround) but since we fixed it quickly we didn't care all that much. Then a senior manager saw we had had a sev1 for the release. After explaining to him that it wasn't really a sev1, he demanded we change it because it was a blemish on the release and he wouldn't have it. Per the rules in the tool, you couldn't change anything in a bug once it was closed.
I had to call in a favor to someone I had a good relationship on the IT tools team. He wouldn't reopen it so I could change the severity, but he agreed to change it for me after I had a few other people join in on the email chain and back up my story. It took a few DAYS to make that happen.
I have done a lot of work on metrics around bugs, trends over time, etc. It can be a REAL pain if over a couple of years you've gone through process and people changes. If you care at all about trying to pull metrics,define the key fields for your data as best you can, and enforce consistent use of them.
My beliefs do not require that you agree with them.