Slashdot Mirror
The Sad Graph of Software Death (tinyletter.com)
An anonymous reader writes: Programmers, raise your hand if you've been on a project where bugs keep piling up, management doesn't dedicate time to fix them, and the whole thing eventually bogs down. Gregory Brown summarizes that situation in one simple little graph from an issue tracker, and discusses why so many companies have problems with it. "This figure tells a story that is no way surprising to anyone who has worked on software projects before: demand for fixes and features is rapidly outpacing the supply of development time invested, and so the issue tracker is no longer serving as any sort of meaningful project planning tool. In all but the most well-funded, high functioning, and sustainable businesses — you can expect some degree of tension along these lines. The business side of the house may blame developers for not moving fast enough, while the developers blame the business for piling work on too quickly and not leaving time for cleanup, testing, and long-term investments. Typically, both sides have valid concerns, but they don't do an especially good job of communicating with one another." What methods have helped you deal with situations like this? What methods haven't helped?
Communication is the main task (and, IMHO, should be the sole one) of managers.
Get rid of that wave of mongols that call themselves "Managers", give the task to someone that can understand both sides, and you will see things going better.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
I like to think of technical debt in the ratio A:B ... A is always 1, perfect. B is what you achieve. So a ratio of 1:2 might not be that bad for a core enterprise system. 1:50 could be fine for a quick and dirty proof of concept.
When designing a new system, module, or major upgrade...a frank discussion is needed with the requisitioner on what A:B ratio is being aimed for. Model it out, with risks, attaching notional monetary values to the cost of poor quality.
Then...your job is done, the informed choice is theirs
The business side of the house may blame developers for not moving fast enough,
Yes, but that is the most pathetic excuse ever. No one is preventing the business from hiring additional workers, if the business lied about its workers' capabilities or is managed by idiots who substitute optimism for basic estimation skills that's their own problem.
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
...is the spoon a failure? Or are you just a fucking imbecile?
Technology is not magic. The problem here is caused almost wholly by people who do not understand, and actively refuse to understand, the technology they are working with. So they wind up being completely unreasonable in the long run and then blaming coders when things go south. It's like someone buying a cheap sedan and expecting it to go a million miles non-stop without any service or maintenance while driving it like it's a rally car. The car's just fine, the driver has unrealistic expectations.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
You're skipping the part where the project was understaffed in the first place, requirements were ill-defined and ever-changing, and the final arbitrary, unrealistic, pie-in-the-sky delivery deadline date (the ONLY immutable thing in the whole goddam project) was pulled out of someone's ass. Oh. And the devs got pulled away on occasion to fix hair-on-fire bugs in projects they worked on previously (or even better: had no experience with).
... is, if you're still receiving bug reports and feature requests, that means people are still interested in using your software.
You know a particular program is really dead when you stop receiving feedback from users about it -- that means either it is finally perfect and bug-free (ha ha, not bloody likely), or everybody has given up on it and moved on to something else.
I don't care if it's 90,000 hectares. That lake was not my doing.
Figure out what those bugs are coming from. Are they documentation, failed unit tests, new features, badly colored GUI widgets, or more hardware features?
It isn't going to help to have a recruitment blitz for more hardware engineers if your technical writers can't keep up with the documentation.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
These are my two favorite books for dealing with the problem.
Clean Coder is the latest Uncle Bob book, and IMO his best. It shows how to act in a way that managers don't make weird demands of you, and how to handle them when they do. Essentially it boils down to this: be professional.
Zero Bugs teaches how to reduce the bug count when you write it, so you don't get overwhelmed as you go along.
Generally I've found managers/customers understand that features take time, and they are happy as long as you are reasonably close in your estimates. But when your product is buggy, and you miss your delivery dates by months, that's when they start getting upset.
"First they came for the slanderers and i said nothing."
This is of course a well known problem documented in the paper Big Ball of Mud by Brian Foote:http://www.laputan.org/mud/. The basic problem is that many systems grow organically as new features are required, but as new features are added to the system it becomes more complex and tightly coupled. Another aspect I have noticed might be the 'Black Hole' syndrome. In systems that are custom written and business critical there is no clear scope of the project. With no clear scope anything new that the business needs is simply thrown into the existing system. The never ending scope creep means that the system takes more and more responsibility. And so grows a monolithic tightly coupled black hole. Because it is at the centre it tends to attract any new requirements because anything new needs to interact with it. And the more you add the harder it is and the more bugs you introduce.
There are well known solutions of course. The first would be to read the link above. However, there are two broad areas to address, and you need business buy in for both. The first is software development discipline; proper code repositories, regular check in, unit tests, code reviews. And while there should be nothing new here there is often resistance from management. You need to explain to management that poor quality means lower development velocity. Taking time to do testing and code reviews may appear to take time, but try not doing it and you find out pretty quickly the folly of ignoring quality. This is just basic coding hygiene. You can of course also apply agile principles and practises, such as time boxing of iterations and regular feedback from users.
But okay, you have an existing system you need to fix. Service Oriented Architecture is more than just a buzzword; it is the principle of separation of concerns. First thing to do is define what the system does. Does it do multiple things? You need to have a clear idea about what the system does, and to then begin to cleave them into separate systems with clearly defines scopes. Sometimes this means identifying relatively simple subsystems to separate. Break the system apart and introduce well defined contracts.
Refactoring code without getting a eagles eye view of the whole system and where to introduce the interfaces is dangerous. Often this process of architectural clarification can cause systems to experience complexity collapse as duplicate code is reduced and removed. Premature re-factoring at the class level may introduce little benefit. Better to pull off the small subsystems with a clear scope and purpose and ensure the code in these new subsystems do not include unnecessary external domain objects.
Again, without having the business behind you a project like this is doomed. You need to present a clear plan to the business and explain how it will improve quality and the ability to add new features, while reducing development costs. You need to explain that this is not a issue with the quality of software developers, but rather a systemic problem that can be corrected. And of course for this you need someone with the courage to tell this story in a compelling way.
4. With all the bugs fixed, you can now fire the programmers.
5. Rehire the managers.
6. Repeat!
That's all BS.
The problem is: If you find yourself in this situation, your manager is used to crapping on the floor. Just like dogs, poorly housetrained managers are much more of a problem than fresh managers. Worse still, many employers actively discourage hitting your manager with rolled up newspapers. Quit your job if it's the only way to find new managers. It's true that more are bad than good, but with experience you will learn to recognize the stink during interviews/office visits.
Don't be afraid to bail in the first week at a new job. You were looking for work when you found that place.
A manager who is used to 'his team' taking it in the personal life isn't going to change his ways. It's mostly working for HIM/HER.
Recognize the root of the problem and if you can do anything about it besides leave.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Like the article said, this could well be a symptom of poor or clueless project management. Duplicate issue reports and low-priority nice-to-have items may be overwhelming actual problems. On the other hand, this could also be a case of software "maturity" where it becomes difficult, if not impossible, to fix any bugs without introducing new ones.
The least bad "solution", as a technical lead or developer, is to abandon the problem tracking system. To politically sell this idea it describe it as a "critical bug tracking system" and make it a simple command-line or email interface that only the lowly code grunts and technical leads will be comfortable with.
Another "solution" is to start closing problems and declaring they cannot be fixed without a redesign of the product.
One thing to remember is that substituting product management with a bug tracking system is seriously lame. We all use software, from web browsers to compilers to editors to source control systems to databases, that has bugs. Each new release of said software will fix some bugs and introduce new ones. Most of the time, for most users, if we had a choice between fixing those bugs and making the product in question ten times as fast or use 10% the memory we'd take one of the latter over the bug fixes. What I'm trying to communicate is that an "issue tracking" system can't usually communicate what customers want or need perfectly.
I think a lot of the problems software development has, could be solved by ending the absurd game of telephone we are all playing as to what the people using the software are actually wanting.
It's easy in a bug fix swamp to lose sight to lose track of what the hell working even means, or what is important to HAVE working.
It's impossible to avoid creating massive technical debt without understanding the possible futures for what you are working on, futures that sorry to say non-technical people are awful at interpolating from the user desires and issues of the day.
In so many companies I've worked at, developers are given at most the tiniest bits of information or contact with real users or customers. If you ever want software to really improve, that must end. Heck, most companies should probably have programmers shadow a real user for at least a day at least once a month.
I think companies are loathe to do this because of the myth that software developers are anti-social - in reality I don't think it's any more true than any other group of people (except of course for sales people). And on a side note if someone is difficult for a customer to get along with, why on earth do you think they would be good for team dynamics....
"There is more worth loving than we have strength to love." - Brian Jay Stanley
On a couple of occasions, there have been attempts to completely sidetrack us onto some other highly urgent project for a product that is not what we normally work on. This ignored several relevant points.
1) We are primarily C/C++ programmers. These other products were primarily Java based.
2) We knew nothing about the internal architecture of these Java-based applications.
3) We are not co-located with the developers who work on these Java applications. Questions would have to be posed by phone/email/skype.
4) Sidetracking us like this would result in our current product withering.
Whenever the question of management arises for programmers, I always return to the same manual. This single document is answers many of the questions regarding failure of IT projects in general.
http://www.computerworld.com/a...
Thanks for these headlines - reminds me of how good I got it. My manager is a former developer. I have, maybe, two or three hours of meetings a week. The issue list is planned out every Monday - if something high priority comes in, something gets taken off the list. If anyone starts monopolizing my time he fends them off or clears the schedule.
I have an acquaintance who went to work for a huge web company (not that one, the other one.) He's a pretty experienced developer, so he grilled them on their development process during the job interview. All the right things were said. They were all lies. He quit after two weeks.
My Other Computer Is A Data General Nova III.
Looking at the graph in the article, there's on obvious inflection point that occurs on 7-Apr. Prior to that, the two lines (opened and closed items) are basically tracking each other. After that point, the opened items (red) retains the same slope; but the closed items (green) switches to a different, shallower (but thereafter basically constant) slope. And thus the two curves veer away from each other from that point.
So: What happened on 7-Apr? Did one or more developers quit, burnout, take a long vacation? Maybe they haven't been replaced yet?
After that I'd try real hard to stop new features from coming in, and start thumbing through a Brooks book to look for suggestions in an emergency like this.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
I was handed a problem where SQL couldn't handle the parameters in code from a product that wrote code in a convoluted way. It took me two weeks to handle it, but the fix I eventually pushed out made sure that the issue would never happen again.
Despite the fix working for about ten years now, I never got an apology for being punished over the fix taking more than the nitwit manager saying it would take to fix.
If you want code teams to fix stuff right, make sure that the code guy is a manager. If you want someone to blow smoke up your ass and punish people that fix things promote suckup people that have no idea on how to code, or hire h1b's.
_ _ _ Go for the eyes Boo! GO FOR THE EYES!
I spent most of my time at my previous job automating tasks for other people so that they could stop doing the repeated and error prone work.
Then I got laid off because the bean-counters at central corporate did not believe I had enough "billable hours". The corporation actively encourages you to work slowly and inefficiently because they make more money that way...
And they wonder why the offshore sector in Norway crashed a year ago with the oil price no longer inflating profits.... sigh
The graph of software death looks much worse than this. What we see in this graph is that the number of open issues in this project grows linearly with the number of issues resolved. This is normal; the number of open issues directly corresponds to the size/complexity of the software. Many issues are likely not ever going to be resolved and in practice this is fine; for most software the economically optimum quality level is simply not "issue free".
Also, it is highly likely for software to gain features that are not going to stay. The bug count for such features will grow and make your graph look very sad. However, once the component is dropped and all open bugs can be closed, you will often see that a relatively large number of bugs were in that component. Without such information, it is impossible to tell what a graph is trying to tell you.
The sad graph of software death does not just have the number of open bugs growing; that's normal. It has the created:resolved rate itself growing. This might, however, be very subtle; perhaps even this projects' bug count is growing exponentially, but with a mere 15 data points it is impossible to tell.
So if your project looks like this graph, which I'm pretty sure it will if you're dealing with somewhat mature software that is continuously being developed, be happy about. You're just fine. Your software is not about to explode anytime soon. If, however, you see the created:resolved rate itself growing then you're in trouble.
0x or or snor perron?!
Weekly meeting, meeting about meetings, meetings to assess the requirements for feature request, code reviewing others, triaging automated tests etc all happen first at my work. Add to that ~10 stat holidays and 15 vacation days: guess what doesn't happen on the weeks those happen? You guessed it coding. The emails still need to be answered, meetings get scheduled around your days off etc. So that 7 hrs out of the office comes right off the development time. Then you end up with a meeting about why features are coming slow :)
Management of at least feature bugs: the problem is prioritization. People use the bug trackers as a note pad to drop any idea for a feature they have, regardless of if a customer has actually asked for it, whether another bug for the same or a mutually exclusive feature already exists, whether the feature will actually raise demand or sell service contracts enough to justify the expense of development and ongoing support etc. Bug trackers become a pool of ideas that no one has bothered vetting.
My suggestion would be: management should be the only ones that can add bugs to the tracker and their compensation should be linked to how clean they keep it, something like: -$100 for every bug that doesn't get worked on in a year: either you aren't staffing properly, are loose with what you let in, aren't doing the necessary work to fill out the requirements enough so that devs can start working on it etc. All management failures. I'd be okay with that as a dev: if I can't explain why a technical feature or fix is necessary and what business impact it would have well enough that my manager prioritizes it then it shouldn't happen. If it not happening means the project fails well who denied the features resources ... managers. You know the guys the business has entrusted the project to?
To keep trackers clean I think everyone should keep their own separate list of ideas. The tracker should only have stuff that is currently being worked on or can reasonably be expected to make it in the next release. Have a meeting every few months and everyone can try to sell their ideas. A bunch get picked as the priorities for the next release and added to the tracker. Those that are no longer relevant don't even get seen by more than one person (or the guy with bad ideas quickly gets found out and shot). This has the added benefit of the devs participating and seeing the thinking behind the choice of features for a release rather than having it slowly revealed as the boss comes by and drops a new item that "must happen this week" on their desk.