The Sad Graph of Software Death (tinyletter.com)
An anonymous reader writes: Programmers, raise your hand if you've been on a project where bugs keep piling up, management doesn't dedicate time to fix them, and the whole thing eventually bogs down. Gregory Brown summarizes that situation in one simple little graph from an issue tracker, and discusses why so many companies have problems with it. "This figure tells a story that is no way surprising to anyone who has worked on software projects before: demand for fixes and features is rapidly outpacing the supply of development time invested, and so the issue tracker is no longer serving as any sort of meaningful project planning tool. In all but the most well-funded, high functioning, and sustainable businesses — you can expect some degree of tension along these lines. The business side of the house may blame developers for not moving fast enough, while the developers blame the business for piling work on too quickly and not leaving time for cleanup, testing, and long-term investments. Typically, both sides have valid concerns, but they don't do an especially good job of communicating with one another." What methods have helped you deal with situations like this? What methods haven't helped?
Communication is the main task (and, IMHO, should be the sole one) of managers.
Get rid of that wave of mongols that call themselves "Managers", give the task to someone that can understand both sides, and you will see things going better.
Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
One concern of mine with development (I'm a developer) is that business managers, and potentially owners, are vague, while developers need details to get the job done right. With that said, this is what I mean: Business managers tend to expect developers to be magic because they are doing something they don't want (care to) understand, and expect it done like all other workers: you're paid, so do the job I tell you, and do it now.
Developers are in a creative role. (I mean the people who craft the solutions, not the ones who copy+paste code and use frameworks to do 90% of the work.) Therefore, details, and time to get it right, are important. When you pile on features, like piling on ideas, you overwhelm the creative process.
Regarding bugs: That's either an inherited mess, a mess created from poor development/programming, or a mess created from feature creep. When vague directions come from the higher ups (because they don't want to deal with the details/process of making it happen, only the monetary rewards), we get that ever impending deadline that seems ridiculous.
I haven't met many developers who were not enthused to create something great. That's what most of us are here to do. But we need details. And we need time to get it done right... so we aren't chasing after bugs created because someone "wanted it out the door" thanks to their salesperson overpromising. Y'know, can't make the sales person look bad... even if he makes you like a piece of a shit behind your back.
Just sayin'
From experience.
Welcome to the world, where bugs are just part and parcel of life.
Just cruising through this digital world at 33 1/3 rpm...
seems to be a popular choice for management these days.
The graph in TFA shows that half of the maintenance team was removed on March 7th. Both new issues and retired issues are going linearly with time.
I like to think of technical debt in the ratio A:B ... A is always 1, perfect. B is what you achieve. So a ratio of 1:2 might not be that bad for a core enterprise system. 1:50 could be fine for a quick and dirty proof of concept.
When designing a new system, module, or major upgrade...a frank discussion is needed with the requisitioner on what A:B ratio is being aimed for. Model it out, with risks, attaching notional monetary values to the cost of poor quality.
Then...your job is done, the informed choice is theirs
Can somebody make the equivalent graph for the Firefox project? I suspect it would look very similar. We've seen Mozilla throw all sorts of new, and completed unwanted, shit into Mozilla lately. Yet there are still many bugs, some of them years old, that haven't been fixed.
But I suppose we don't really need a graph like that to know that Firefox is a dying project. Its ever-dropping share of the browser market is good evidence of that. And contrary to the excuses we get so often from Mozilla's supporters, no, Firefox is not dying because Google is advertising Chrome, nor because of mobile browsers, nor because of any other bullshit like that. Firefox is being thrown away by its users because so much unwanted shit has been forced upon these users.
The business side of the house may blame developers for not moving fast enough,
Yes, but that is the most pathetic excuse ever. No one is preventing the business from hiring additional workers, if the business lied about its workers' capabilities or is managed by idiots who substitute optimism for basic estimation skills that's their own problem.
Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
it's saturday, and that's all i've got.
...is the spoon a failure? Or are you just a fucking imbecile?
Technology is not magic. The problem here is caused almost wholly by people who do not understand, and actively refuse to understand, the technology they are working with. So they wind up being completely unreasonable in the long run and then blaming coders when things go south. It's like someone buying a cheap sedan and expecting it to go a million miles non-stop without any service or maintenance while driving it like it's a rally car. The car's just fine, the driver has unrealistic expectations.
A bullet may have your name on it but splash damage is addressed "To whom it may concern."
The Sad Graph of Software Death
Right. Here's one of my big gripes with Slashdot. Months ago, you brought in video. But you still don't have articles with images, even when they are the main point of a summary? That's pretty lame.
systemd is Roko's Basilisk.
Before adding new features, debug the old features. If you're adding features when your system isn't debugged, of course it is all going to come crashing down. You'll find yourself hitting an old bug when debugging new stuff, and it is distracting to jump all over the place coding. Focus on one part of the code at a time if you want to do it right.
... is, if you're still receiving bug reports and feature requests, that means people are still interested in using your software.
You know a particular program is really dead when you stop receiving feedback from users about it -- that means either it is finally perfect and bug-free (ha ha, not bloody likely), or everybody has given up on it and moved on to something else.
I don't care if it's 90,000 hectares. That lake was not my doing.
DUCK
This is why Confluence 3.5 died. RIP.
Saving money by using offshore developers.
Figure out what those bugs are coming from. Are they documentation, failed unit tests, new features, badly colored GUI widgets, or more hardware features?
It isn't going to help to have a recruitment blitz for more hardware engineers if your technical writers can't keep up with the documentation.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
1. Fire all managers.
2. With the money this frees up, triple the programmer headcount.
3. Profit.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
These are my two favorite books for dealing with the problem.
Clean Coder is the latest Uncle Bob book, and IMO his best. It shows how to act in a way that managers don't make weird demands of you, and how to handle them when they do. Essentially it boils down to this: be professional.
Zero Bugs teaches how to reduce the bug count when you write it, so you don't get overwhelmed as you go along.
Generally I've found managers/customers understand that features take time, and they are happy as long as you are reasonably close in your estimates. But when your product is buggy, and you miss your delivery dates by months, that's when they start getting upset.
"First they came for the slanderers and i said nothing."
This is of course a well known problem documented in the paper Big Ball of Mud by Brian Foote:http://www.laputan.org/mud/. The basic problem is that many systems grow organically as new features are required, but as new features are added to the system it becomes more complex and tightly coupled. Another aspect I have noticed might be the 'Black Hole' syndrome. In systems that are custom written and business critical there is no clear scope of the project. With no clear scope anything new that the business needs is simply thrown into the existing system. The never ending scope creep means that the system takes more and more responsibility. And so grows a monolithic tightly coupled black hole. Because it is at the centre it tends to attract any new requirements because anything new needs to interact with it. And the more you add the harder it is and the more bugs you introduce.
There are well known solutions of course. The first would be to read the link above. However, there are two broad areas to address, and you need business buy in for both. The first is software development discipline; proper code repositories, regular check in, unit tests, code reviews. And while there should be nothing new here there is often resistance from management. You need to explain to management that poor quality means lower development velocity. Taking time to do testing and code reviews may appear to take time, but try not doing it and you find out pretty quickly the folly of ignoring quality. This is just basic coding hygiene. You can of course also apply agile principles and practises, such as time boxing of iterations and regular feedback from users.
But okay, you have an existing system you need to fix. Service Oriented Architecture is more than just a buzzword; it is the principle of separation of concerns. First thing to do is define what the system does. Does it do multiple things? You need to have a clear idea about what the system does, and to then begin to cleave them into separate systems with clearly defines scopes. Sometimes this means identifying relatively simple subsystems to separate. Break the system apart and introduce well defined contracts.
Refactoring code without getting a eagles eye view of the whole system and where to introduce the interfaces is dangerous. Often this process of architectural clarification can cause systems to experience complexity collapse as duplicate code is reduced and removed. Premature re-factoring at the class level may introduce little benefit. Better to pull off the small subsystems with a clear scope and purpose and ensure the code in these new subsystems do not include unnecessary external domain objects.
Again, without having the business behind you a project like this is doomed. You need to present a clear plan to the business and explain how it will improve quality and the ability to add new features, while reducing development costs. You need to explain that this is not a issue with the quality of software developers, but rather a systemic problem that can be corrected. And of course for this you need someone with the courage to tell this story in a compelling way.
Developers how have not figured out new ways of solving old problems. They dogmatically hit the wall in the same way over and over again. It's worse when developers with lots of experience get promoted into management and stay there because they can charm they way there. If they realize that it's a different type of job and start managing people (keep egos in check, deescalate tensions, etc.) while researching new ways of solving old problems and let people develop, they build people up. If they just try to farm people and keep on top, they don't realize when poison pills get introduced into the code because they are often the ones doing it. If you see a manager who thinks that Emacs (fine or vi) is the only way to develop code, you are seeing one of those. Progress in technology means faster and more effective ways of creating new technology. Orthodoxy is people trying to suck the blood out of the young while they are still young and set them on the way to thinking that the only way to advance in career is riding on their coat tails. Development is transformation of problems into solutions. It means culling the crud work -- not getting more effective at doing the crud work.
Any guest worker system is indistinguishable from indentured servitude.
But not quite so pronounced. Plus, the CIO (company founder) regularly kills off tickets he deems irrelevant.
Though, I'm pretty certain the company I work at is a scam. Our numbers are terrible and plummetting, yet presentations from leadership are always as glowing as can be and I know the majority of employees aren't drinking the Kool-Aid anymore.
Having worked in a couple companies with this kind of problem, I think the issue is that all too often dev groups dont directly build their prioritization in terms of every single item tied to a piece of revenue. Gambles or demands to land new business frequently beats investment in retaining customers or lowering operational costs. I think at least two business realities compound this issue. One, salespeople on quotas set priorities in some dev groups, and they are inclined to overcommit and gamble. Two, "Lean startup" and "minimum viable product" language is great for early rounds of funding and tactical direction, but I have not yet worked in a company that had a clearly articulated line in the sand to evolve from MVP to "enterprise software". Sooner or later, customers want stable, fully featured software. Report generation is my personal albatross.
Scaled Agile Framework offers a solution to this problem. I've used it and it works, but it needs full support from senior management to be effective.
That's all BS.
The problem is: If you find yourself in this situation, your manager is used to crapping on the floor. Just like dogs, poorly housetrained managers are much more of a problem than fresh managers. Worse still, many employers actively discourage hitting your manager with rolled up newspapers. Quit your job if it's the only way to find new managers. It's true that more are bad than good, but with experience you will learn to recognize the stink during interviews/office visits.
Don't be afraid to bail in the first week at a new job. You were looking for work when you found that place.
A manager who is used to 'his team' taking it in the personal life isn't going to change his ways. It's mostly working for HIM/HER.
Recognize the root of the problem and if you can do anything about it besides leave.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
The author challenged the readers to make a single change that would have an immediate effect without writing a single line of code. I can only think of one... immediately stop implementing new features. That would keep the open/close ratio from getting much larger without any coding. But how to make the business case for doing so? My argument would be that the graph shows that their current systems are broken and worthless. The business can recover their investment in the current systems by taking the time to fix the open issues and removing broken functionally that's not needed.
Then ignore them and do what you need to do.
Most software is not fine art, it's a tool made by paid professionals for paid professionals. If there isn't a viable return on the investment made for maintenance and enhancement, the program will languish and die. Programs are not people a right to live and be cared for. Often the business decisions are faulty and software that is worthy of investment is ignored. These businesses are often not competitive and they die along with their creations. Sometimes a program's value only warrants a minimal maintenance investment. Admittedly, it's no fun doing a half-assed job, but sometimes you just have to hold your nose and do the minimum.
All too often business decision makers don't understand the tradeoffs and value of software maintenance. Equally, developers ignore the business aspect of their creations and cry for purity from their ivory tower for programs that do not warrant further Investment. Both camps have to learn to take the wider view.
I've learned to advise that when software is developed, a maintenance budget be crafted from the inception of the program and projected through the entire program's expected life. Having this planned in the beginning of a system's lifecycle promotes satisfaction, success, quality and long life.
Greed is the root of all evil.
Like the article said, this could well be a symptom of poor or clueless project management. Duplicate issue reports and low-priority nice-to-have items may be overwhelming actual problems. On the other hand, this could also be a case of software "maturity" where it becomes difficult, if not impossible, to fix any bugs without introducing new ones.
The least bad "solution", as a technical lead or developer, is to abandon the problem tracking system. To politically sell this idea it describe it as a "critical bug tracking system" and make it a simple command-line or email interface that only the lowly code grunts and technical leads will be comfortable with.
Another "solution" is to start closing problems and declaring they cannot be fixed without a redesign of the product.
One thing to remember is that substituting product management with a bug tracking system is seriously lame. We all use software, from web browsers to compilers to editors to source control systems to databases, that has bugs. Each new release of said software will fix some bugs and introduce new ones. Most of the time, for most users, if we had a choice between fixing those bugs and making the product in question ten times as fast or use 10% the memory we'd take one of the latter over the bug fixes. What I'm trying to communicate is that an "issue tracking" system can't usually communicate what customers want or need perfectly.
I think a lot of the problems software development has, could be solved by ending the absurd game of telephone we are all playing as to what the people using the software are actually wanting.
It's easy in a bug fix swamp to lose sight to lose track of what the hell working even means, or what is important to HAVE working.
It's impossible to avoid creating massive technical debt without understanding the possible futures for what you are working on, futures that sorry to say non-technical people are awful at interpolating from the user desires and issues of the day.
In so many companies I've worked at, developers are given at most the tiniest bits of information or contact with real users or customers. If you ever want software to really improve, that must end. Heck, most companies should probably have programmers shadow a real user for at least a day at least once a month.
I think companies are loathe to do this because of the myth that software developers are anti-social - in reality I don't think it's any more true than any other group of people (except of course for sales people). And on a side note if someone is difficult for a customer to get along with, why on earth do you think they would be good for team dynamics....
"There is more worth loving than we have strength to love." - Brian Jay Stanley
On a couple of occasions, there have been attempts to completely sidetrack us onto some other highly urgent project for a product that is not what we normally work on. This ignored several relevant points.
1) We are primarily C/C++ programmers. These other products were primarily Java based.
2) We knew nothing about the internal architecture of these Java-based applications.
3) We are not co-located with the developers who work on these Java applications. Questions would have to be posed by phone/email/skype.
4) Sidetracking us like this would result in our current product withering.
Greg,
In reality this trend is being seen in all of engineering in both hardware and software development. This is trending like this in both Commercial and Defense Contracting practices. Don't feel you're alone in this. I think we may be seeing another symptom of the brain drain. All of the competent baby boomers are retiring and peeling back the onion to show what is left of the people that just "rode the gravy train" and now are unable to step up to fill the roles left behind.
In short we've a severe lack of leadership, people willing to step into that role because there is no merit structure for it, and an entrenched environment where there is retribution for "rocking the boat". The only silver lining is to allow the Mid-Career folks (probably Gen X and Ys) to step up and make positive change and offer leadership without retribution, and a revamped merit system that will influence that for the positive.
The next 5 to 10 years is going to be a wild ride.
Whenever the question of management arises for programmers, I always return to the same manual. This single document is answers many of the questions regarding failure of IT projects in general.
http://www.computerworld.com/a...
Thanks for these headlines - reminds me of how good I got it. My manager is a former developer. I have, maybe, two or three hours of meetings a week. The issue list is planned out every Monday - if something high priority comes in, something gets taken off the list. If anyone starts monopolizing my time he fends them off or clears the schedule.
I have an acquaintance who went to work for a huge web company (not that one, the other one.) He's a pretty experienced developer, so he grilled them on their development process during the job interview. All the right things were said. They were all lies. He quit after two weeks.
My Other Computer Is A Data General Nova III.
You need a combination of tools, Experience in reporting issues, and Management that will actively seek out issues, asking the right set of questions.
If you're a lead you need to get your butt out of the meeting room and sit with your developers asking pointed questions with no retribution. IF you do that successfully, you will build confidence in your staffing that they can come to you to report issues. On the other side, you need to demand quality issue reporting with potential avenues for resolution. If run rules are clear and set correctly this combination works.
Getting in the working area with the developers once in while will garner respect among your development staff. Offer them a way to report and work issues will do some things:
1) Make them feel like a professional
2) Feel they make a contribution
3) Provide an open dialogue with engineers of different disciplines
THIS TAKES MONEY. If you allow issues to get sidelined (in my experience sometimes for years) you're retention, morale, and quality will suffer.
Grow a pair and PRIORITIZE!!!! If you can't do that then everyone will chose their own uncoordinated path.
Looking at the graph in the article, there's on obvious inflection point that occurs on 7-Apr. Prior to that, the two lines (opened and closed items) are basically tracking each other. After that point, the opened items (red) retains the same slope; but the closed items (green) switches to a different, shallower (but thereafter basically constant) slope. And thus the two curves veer away from each other from that point.
So: What happened on 7-Apr? Did one or more developers quit, burnout, take a long vacation? Maybe they haven't been replaced yet?
After that I'd try real hard to stop new features from coming in, and start thumbing through a Brooks book to look for suggestions in an emergency like this.
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
I was handed a problem where SQL couldn't handle the parameters in code from a product that wrote code in a convoluted way. It took me two weeks to handle it, but the fix I eventually pushed out made sure that the issue would never happen again.
Despite the fix working for about ten years now, I never got an apology for being punished over the fix taking more than the nitwit manager saying it would take to fix.
If you want code teams to fix stuff right, make sure that the code guy is a manager. If you want someone to blow smoke up your ass and punish people that fix things promote suckup people that have no idea on how to code, or hire h1b's.
_ _ _ Go for the eyes Boo! GO FOR THE EYES!
Communication is the main task (and, IMHO, should be the sole one) of managers.
You mean except for budgeting, staffing, scheduling, conflict resolution, planning, reporting, coaching, motivating, forecasting, negotiating, delegating, and the thousand other things a manager actually has to do in the real world?
If you think communication is the only thing a manager should have to do you are pretty clueless about what it takes to manage a group of people. Effective management is a hell of a lot more than just "communication".
That graph might be perfectly ok. All it shows is bugs outpacing fixes. But if the product has been expanded with more feature areas, each bringing more minor bugs but also a higher product surface area.
The best thing to do is probably to go down the list and close off the ones that won't be fixed. We know it's a bug but it's low impact and not worth fixing - at least be honest about it.
The graph of software death looks much worse than this. What we see in this graph is that the number of open issues in this project grows linearly with the number of issues resolved. This is normal; the number of open issues directly corresponds to the size/complexity of the software. Many issues are likely not ever going to be resolved and in practice this is fine; for most software the economically optimum quality level is simply not "issue free".
Also, it is highly likely for software to gain features that are not going to stay. The bug count for such features will grow and make your graph look very sad. However, once the component is dropped and all open bugs can be closed, you will often see that a relatively large number of bugs were in that component. Without such information, it is impossible to tell what a graph is trying to tell you.
The sad graph of software death does not just have the number of open bugs growing; that's normal. It has the created:resolved rate itself growing. This might, however, be very subtle; perhaps even this projects' bug count is growing exponentially, but with a mere 15 data points it is impossible to tell.
So if your project looks like this graph, which I'm pretty sure it will if you're dealing with somewhat mature software that is continuously being developed, be happy about. You're just fine. Your software is not about to explode anytime soon. If, however, you see the created:resolved rate itself growing then you're in trouble.
0x or or snor perron?!
If the graph just reflects all open issues, including features, then it might actually be perfectly normal, particularly if you are just throwing everything that everyone ever asks for into the bug tracker, if there has been some kind of burst of planning activity, or if the software has just been released or tested, resulting in more issues being reported. The graph is also from a very short period, and the number of issues is quite small anyway.
The real sign of crisis won't come from graphs or metrics. How do you know what the metrics are actually telling you? Managers have to communicate with their teams and stakeholders properly to assess if there is really any kind of problem.
At our company we have introduced the idea of Technical Debt. It allows developers to put a price tag on "bad code that needs cleaning up" and we can better prioritize that against new features with stakeholders. It's a good first step...
It stands to reason that if adding people to a project makes it later then taking them away makes it earlier.
That's why I is a manijer and you isn't.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Weekly meeting, meeting about meetings, meetings to assess the requirements for feature request, code reviewing others, triaging automated tests etc all happen first at my work. Add to that ~10 stat holidays and 15 vacation days: guess what doesn't happen on the weeks those happen? You guessed it coding. The emails still need to be answered, meetings get scheduled around your days off etc. So that 7 hrs out of the office comes right off the development time. Then you end up with a meeting about why features are coming slow :)
Management of at least feature bugs: the problem is prioritization. People use the bug trackers as a note pad to drop any idea for a feature they have, regardless of if a customer has actually asked for it, whether another bug for the same or a mutually exclusive feature already exists, whether the feature will actually raise demand or sell service contracts enough to justify the expense of development and ongoing support etc. Bug trackers become a pool of ideas that no one has bothered vetting.
My suggestion would be: management should be the only ones that can add bugs to the tracker and their compensation should be linked to how clean they keep it, something like: -$100 for every bug that doesn't get worked on in a year: either you aren't staffing properly, are loose with what you let in, aren't doing the necessary work to fill out the requirements enough so that devs can start working on it etc. All management failures. I'd be okay with that as a dev: if I can't explain why a technical feature or fix is necessary and what business impact it would have well enough that my manager prioritizes it then it shouldn't happen. If it not happening means the project fails well who denied the features resources ... managers. You know the guys the business has entrusted the project to?
To keep trackers clean I think everyone should keep their own separate list of ideas. The tracker should only have stuff that is currently being worked on or can reasonably be expected to make it in the next release. Have a meeting every few months and everyone can try to sell their ideas. A bunch get picked as the priorities for the next release and added to the tracker. Those that are no longer relevant don't even get seen by more than one person (or the guy with bad ideas quickly gets found out and shot). This has the added benefit of the devs participating and seeing the thinking behind the choice of features for a release rather than having it slowly revealed as the boss comes by and drops a new item that "must happen this week" on their desk.
I think even Microsoft and Apple suffer from this. Its possible Google and even the Linux distributions suffer from pushing out stuff that as we say is "half baked".
The problem I see is the marketing level controls the release dates. This happens when a company like Apple decides on a date and sticks to it, no matter if the software or OS is ready. Its clear you see it in the rapid releases of patches shortly after a major release. This really should not happen but it does because the company decides it must release not matter what to meet a deadline. My problem is, that even with software the new is not always better then the old. Since many times this is software people need to work correctly. It does seem rather obvious to ask why release something that truly is not as good in reliability as the previous version? Not to pick on Apple or Microsoft but you go to the Mac App store and look at Apple's latest software or OS (El Capitan) you will see plenty of disappointed upgraders. Also Windows 10 upgraders have expressed similar displeasure from upgrading. Not a good thing, when you push out stuff still in beta.
The problem with graphs like this (not disputing the truth of the data) is that too often they are used to blame management rather than the source of the failure, which is a slack development atmosphere.
When projects are planned, an MRD is created and then project completion estimates flow up from the developers. The developers are committing to completion in a certain time frame. Also, the developers are the ones doing the work, including creating the bugs.
In any case, the problem here flows back to several factors, all of which lie with the devleopers:
0) Lack of sufficient skill in estimation
1) Lack of proficiency to get it right the first time
2) Lack of willingness to stick to their delivery commitments
3) Lack of upward communication when they get in over their heads
4) Sense of entitlement to more resources when they get behind
As the owner of the company, I'm not going to go out and procure additional resources to catch up on these failures, and the company has zero obligation to spend that money. Developers are responsible for obtaining the necessary skills (in both estimation and execution) to do their job function, stick to their delivery commitments no matter what it takes, and tell management when they're in over their heads. The sense of entitlement is the most disturbing part, because as an employee, your job is to work to achieve the company's goals, not your own personal goals. Also, we go to work to work, not to play. When you work for yourself, you can do that, but when you work for someone else, you are being paid to do what they ask you to do.
There's a serious lack of ownership culture in today's development society. We have to change that if we're going to avoid massive project failures like the one depicted in this graph. It costs me a fortune to bring in resources that do what I need them to do, but we don't have these kinds of project failures and we have a base of incredibly happy clients.
I don't mean to discount the disease that permeates management at most companies - that all decisions are made based on the next upcoming fiscal milestone and to appease shareholders. My directors are under strict orders to keep their eyes on the quality/time prize, and thus they have absolutely no visibility to financials, and are not paid as such. My directors' bonuses are tied only to project quality and time. My earnings are entirely based on financials, obviously, and I would like to keep my company until I retire, which requires long-term thinking on my part.
but also hardware. That graph with the x-axis in months, not days, would be almost exact for F-16 development. In hardware, it's related to the "death spiral" where unit costs go up, causing unit sales to go down, causing unit costs go up, causing...
Shouldn't the red line plateau at some point? It's not like a finite piece of software can contain an infinite amount of bugs.
Also note that older software accumulates hacks for platform problems (OS, runtime library, third-party library, you name it) which accumulates into the code base. As software keeps going, it builds up fixes, workarounds, hacks, and so on which lets it survive in the real world. As platforms evolve, you can either rewrite working code to eliminate these hacks or do new stuff. Guess which wins.
Everybody's talking about communication, but that's drinking the Kool-Aid, and being a good codependent.
This is the opposite of a communications problem. What the graph shows is disinvestment - it shows a clear policy that's been extremely well communicated right down to the coders. The message is: "it's working well enough, let's stop spending so much money (programmer time) and get something else done, that's now more vital." Without more context, we can't in fact rule out the possibility that this is the best business decision - although my experience is that disinvestment is frequently very premature. Investment can't be infinite, it can't keep expanding just because bug reports continue to come in at nearly the same rates (but, on average, about ever more specific and less economically damaging issues.)
I suspect (more than suspect, I've painfully experienced) that managers are often beyond ignorant about risk, but the truth is that most of their "ignorance" is pretended. Your communication skills are fine, and so are theirs. Many managers will disinvest when code is still quite dangerous because the economic rewards to them personally are large and immediate for doing so. They will build in or lock in technical and risk debt (and legal jeopardy for that matter) that can bring their whole company down to bankruptcy in full knowledge of those risks; as long as they can be reasonably sure that they will have moved on to a still better job by that time, with some bonuses tucked away that were fattened by their cheating the long-term interest of the corporation. Which they don't own. (See the Great Subprime Robbery of 2007/8.) That's corruption, of a kind very common in a "manager's economy." Maybe more common than not. It's not poor communication, and the tech field isn't immune, it attracts these managers because it's where the money is.
The reason that VCs have learned painfully to keep founders around is that founders aren't susceptible to this particular form of corruption. Founders have their payoff, now the only thing that really matters to them is how the long-term results reflect on their image. Mercenary managers are presented with very different, more perverse incentives and respond to them just as you would expect.
I worked for a company with PHBs above me as far as the eye could see. The word came down to reduce the bug count. Incentives were tied to the bug count coming down. The good news is that bugs were closed. The bad news is that many of those bugs were not fixed. So, yay, goal achieved, I guess, but now we didn't know what was broken, in many cases.
I left without waiting to find out whether there was a perceived drop in quality, with more incentives for filing bug reports, (many of which were closed but not fixed in the first round of incentives).
Your comment contains a good insight, but it makes an entirely unjustified followup.
I believe that you've identified the cause of the problem correctly (or at least one cause), but you're way out when it comes to apportioning blame. The main failure to understand the technology lies with the coders themselves, who most definitely think it's magic --- if they didn't, they'd realize that it's the enemy and they would keep it heavily chained. Instead, they trust it implicitly, despite a million examples worldwide every day of it failing to do what they want.
Although I'm a decades-long techie and tend to view management in this area as a bad joke, in this case non-technical management gets almost a free pass out of jail --- after all, they can't be expected to understand software. In contrast, software engineers would be expected to understand the ramifications and risks of their work 100%, and they do not.
Far from 100% understanding, it's generally not even 5%. They fail to appreciate even the most elementary concepts of risk and error rates in software. They hold a deep religious devotion to the god of It Will Just Work, and the word "religious" is chosen apppropriately because the devotion is based on faith and is unquestioned. Today's coders are not engineers, but acolytes.
http://www.cs.umd.edu/class/spring2005/cmsc838p/VandV/belady.pdf
Get rid of testers (added process bloat) and that will hold developers more accountable for what they check in rather than them throwing shit over the cubicle fence and waiting on it to be thrown back.
Doesn't care about good design just sprints.
Most software houses today believe software fits a factory model when in fact it is more like craft. I think this is the typical pattern: A software product is created, usually by a small, motivated group perhaps even 1, 2 or 3 people. The product ships and the team is disbanded and the product is handed over to GIT. 6 months later, nobody remains who understands how the software works. Management wants to "improve" the original product, mostly to provide a reason for charging for an upgrade. A new team is assembled. The new team cannot or is not willing to invest the painful effort to figure out how the original software worked. They return to management and say, "this software is crap", the only thing we can do is re-write it or hack it. So either the hacks begin (most common) produced by people with limited knowledge and a time horizon of 6 months or a year, or the software is re-written with a loss of features, reliability and continuity.
There's a simple solution to a growing quantity of open bugs: just hire someone to close them. There's plenty of ways to close bugs:
(Everything I know about software development, I learned from the Mozilla project.)
Hey at least your always in demand.
I've found that a proper "QA" group is also pretty good at discerning bugs VS features.
or it will own you. Been there done that.
What I call the business of engineering - engineers need to think like business people. No you won't be allowed to disappear for 8 months and rewrite the product - need to keep the lights on... (plus - you created the current crap, why would you get it correct the second time?).
As others have said - use the principles of Agile. Have a clear definition of Done (Code, code review, test execution - story acceptance [tests pass & PO sign off]). This is basic 101 stuff. Plus a Clear process (use "Scrum" to enforce the good ideas). If you aren't doing this your company is behind the adoption curve. It was a new concept in 2001 - 20 years later it isn't new.
Listen to your customers. All of the little bug reports need to be grouped - New Features - per Feature, and Poor Quality (per Feature). And then own it. The business person should be able to work with you to help prioritize where investment makes sense. Where are customers complaining the most - and why? Looking at new Features should give the PO an idea of what the "Future" product needs to look like (customers do business differently as time goes on). Poor Quality - focus on distinct areas. You can't tackle it all in a single release.
Second - would a new architecture make sense? SOA/Micro is nice - but you can still create crap - you've done it before. Just that crap is isolated. Plus there's a whole new set of problems to deal with (lost or replay messages etc). But if you can make a Business case for a new architecture - then do it. And TCO can be measured (incoming bug rated decline or something like that - get your metrics defined). What are the current problems and what design changes will take care of them? Plus it helps set priorities for refactoring.
Third - any commonality to your quality issues? Figure that out and solve it. Mentor others, solve the problems, and finally - if you need it - define some process. Process will not improve quality by itself (actually process won't solve anything - you define process as the repeatable method to keep the problem solved).
Focus on each problem area - add *some* missing features, fix *some* bugs, and then move onto the next pile. Biggest impact first - make customer happy.
Listen to your customers !! Are you doing a good job? Yes - Keep going. No - "pivot"
It requires dedicated focus and commitment of everyone involved to decide it is a problem they want to solve. The company must sign up to be Agile as well - and you'll need to make the business case for why Agile is a good thing. Enlist sympathetic business people/managers. Make the story compelling.
Think like a business person.
Use the force and become a Change Agent!
This article describes Microsoft software, Windows, Office, etc., Get the product out the door, generate income. Let the users find the bugs on their own nickel.
There's far fewer problems if:
a) management requires buy-in from the programmers
b) management requires new scheduling and budget for enhancements and changes (other than
show stoppers)
c) management puts all enhancement requests in a "next release", not "put it in, whatever it takes".
d) management puts its money where it's mouth is, and hires a good percentage (>> 1 person) of
programmers with experience, and not just out of school in the last year.
e) management is willing to say "I'm sorry, we can't, or won't do that enhancement".
mark
The business side doesn't respect the complexity of software development. Sets all goals, hence typically unrealistic (e.g. make a real hover board).
The engineering side doesn't respect the 3 choices of engineering: quality, time, cost (choose 2)-- by means of smoking the pipe dream of "better faster cheaper" and rapid prototyping (e.g. agile). Also engineering misses the boat of prototyping != product since all the out of the college folks (old and young) are attracted to prototyping/betas.