Slashdot Mirror
Is Your Development Project a Sinking Ship?
gManZboy writes "Everyone knows that some software development projects succeed and other fail -- the question has always been 'why'? I'm sure we all have our favorite (likely anecdotal) explanations. Well, these guys decided to actually go out there and do a formal survey, and they've got some real data on why projects actually fail (as reported by development project managers -- care to guess where 'changing requirements' ranks?). They've developed a diagnostic formula people can use to gauge the likeliness that the project they're working on right now is (or isn't) going to fail."
I used to blame "constant client requirement changes" for failed projects as suggested by my project manager.
s before signing off the budget.
Later I realized, as suggested by the senior management, that a good project manager should not let that happen had he properly designed and managed the project.
Recently I started to think that maybe all failed projects are due to the delays inevitably imposed by the senior management who requires many policies/protocols/documents/approvals/discussion
These delays introduce deadline pressure to project manager, and allow too much time for client to ponder about other features, and most importantly, give breathing space for competitors to come up with similar products BEFORE we do.
Rock that crushes, Paper & Scissors that don't matter.
Changing requirements is far less bad than a frozen spec based on overanalysis by MBAs who never spoke to customers.
People who sit around for months or years trying to design the perfect system. It doesn't exist. Compromise gets projects done.
... did they fail or succeed?
It's as simple as that unfortuantely - we *never* learn from our mistakes. Over the last thirty years every system we can dream of has been built from nuclear power plant control system to stock market analysis systems.
Yet we keep playing the buzzword bingo with our new systems, e.g. "Extreme programming", we still keep promise a schedule we can't keep to, we still allow the customer to shift requirement much later in the project than should be allowed, management still don't have enough dialog with the programmers on the ground floor, the list goes on..
Wake up! We're not special.. the construction industry has been doing huge projects of equal complexity for centuries. Get past your intellectual snobbery and start working together..
Simon.
I've done a lot of thinking about this...I've come to the conclusion that too often, management tries to replace good ol' fashioned thinking with process. It doesn't work. People tend to get focused in on what they're doing to the exclusion of all else, and that means the smart people are cubbyholed and only have half the story and can't see where other parts of the project are failing, and dumb people have free reign over their little part.
If the ratio of intelligence to complexity is too low, then the project will fail no matter what process is in place or who is managing it. That's all there is to it. There's a lot of cool stuff out there to be done in development...sadly, most of the good ideas will never make it because the people working on them don't use common sense and best practices...they're just not smart enough to see what's important and what isn't.
This isn't one of those self-important diatribes from a holier-than-thou developer, either...true I'm a developer, but I admit when I'm too dumb to handle the particulars of a project; usually, that means the project is too complex for most people, but they press on anyway. Those projects always go down in flames eventually.
You have to know what the strenghs and weaknesses of your team and its members are, and exploit those to the fullest. Maybe, then, you can barely accomplish a project if the goal of that project is simple enough.
but have you considered the following argument: shut up.
We've had people who didn't know how to accurately scope business requirements, get buy in from other departments and generally "play nice" enough to keep everything running smoothly. Your P.M. needs to be able to be a hard ass, but also to be a buddy.
It boils down to excellent management skills and excellent people skills and without both you're setting a project up for disaster. A good P.M. needs to know when to tell senior management it's asking for the impossible too, and a good P.M. needs to know he has kung-fu so he can get away will telling senior management their idea won't be implemented.
It's not to say that a good designer or developer cannot be a good project manager; it's just a different job, like asking a plumber to rewire your house.
Ah yes, the Heisenberg method of project management...
Not only do client requirements change, but management is also responsible for fubaring things.
;)).
i have been part of a project (past tense) where:
- management delivered a much too low cost estimate in order to get win the bid.
- management then expected the project manager and team to meet the deadlines that were doomed in advance.
- the software design lead designed a behemoth of a framework full of performance and design issues.
- management did not understand that if you have unexplained system behavior, you cannot say when you will have solved the problem.
- hardware design was not reviewed, just like software design. this lead to huge problems just before and during acceptance.
-near the end of the project, more and more people were reassigned to a new project that has the ability to make the department manager look good to the head office. he wants to move up. In effect, succes of the former project became a more and more distant possibility until failure was assured.
and there are probably some other things that i either forgot or purposly left out (trying to repress memories maybe
You jackass! I actually clicked on that....
Anyway, since I can't read the story, I'll just blather on about conditions here where I work... the #1 cause of failed software projects is poor client management. We all know that the client doesn't know jack about making software... if they did, they wouldn't need to hire us. Yet when the client contradicts themself, the opportune moment to flash that consulting brilliance and prove why we cost so much is immediate. It would be so easy to say "what you've just asked for is impossible because it conflicts with this other thing you asked for". Yet from a marketing perspective, that's bad for business. Why? Because our company's goal is to maximize profit, not to maximize software quality. Believe it or not, we often get contract modifications to fix those very problems we could have circumvented because the developers foresaw them coming. Essentially, we get paid to build the same system twice. How's that for consulting brilliance!
I'm in QA, been doing this for 9 years. This parent's post is right on. Aggressive scheduling KILLS projects. It's what causes tension.
It ALWAYS comes down to people. This article looks to be a discussion relevant more to a commercial environment than an open source one, but I guess the same fundamental principle is true - without the right people you will not succeed. This means competent and motivated technical people, clueful and skilled management, and customers willing to be reasonable and pay for what they are getting. Take away any one of these elements, and there is no technique in the world which will result in something everybody can define as a success.
These guys break down the problems into useful categories, which will be helpful for good teams who want to know how to be more efficient. But for my money a group of serious, decidated people who honestly want to get the job done and do it well will usually get there, barring external factors beyond their control messing it up. It might take a while, cost $$, etc. but they'll make it, because they WANT to.
Many (I would even say most) successful open source projects succeed because they have one or several individuals willing to put the work in to make something happen. The tools they use or the way they work are less important than determination to get it done and do it well. Those without that wither on the vine.
In theory, commercial companies and development teams should be motivated by the $$ they are paid, but that doesn't always translate into doing the job well. There are PHBs, lazy workers, unreasonable customers, and all the other joys of life out there. There is no magical "business formula" which can transmute this combination into a good product.
Don't get me wrong, project management and efficiency techniques are a very good thing, but only when you've got the people to make good use of them.
I have lots of evidence of failed projects due to failure to plan.
It can take months or years of thought and discussion to reasonably avoid extreme catastrophies.
While it is silly to try to plan every detail and anyone who claims to do so is lying, a simple elegant, successful general approach is seldom the first one to pop into the head. It takes a lot of thought. Of course, for those incapable of such forethought, why not fail earlier rather than later.
can be used as an excuse to avoid process, which is a distinct animal from bureaucracy.
Good process is independent of the intelligence of the humans implementing the process.
Good process amplifies the effectiveness of all participants.
Good process generates tracking data that can be used in negotiations between development (reality) and management (theory).
In 90% of the subprojects of construction, a manager can walk by in a few seconds gauge: - progress - quality of work - time to completion - implications to dependent subprojects Can't do that with software.
Hey, I'm just your average shit and piss factory.
I have seen this time and time again, people trying to use Process as a mechanism to put everyone on the same level. It sure works - the people who are normally 10x as effective as others are hamstrung and unable to be effective at what they do.
This is the real tragedy of Sarbanes-Oxley. It is being used as a trump card for every process flunky that comes down the pike to implement their favorite process to the fullest. This is going to have a real and very unfortunate effect on companies that become too infected with process to move.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Aaaah, that one is subject to the 95% rule:
The first 95% of the project takes 95% of the time, and the remaining 5% takes the other 95% of the time"
(loosely quoted from some fortune)
All too often the PM is some MBA who couldn't keep his job during the .com downturn, and now makes "specs" out of a vacuum, or worse, textbooks from experts; without understanding the customers needs.
At least when the sales guy sold a feature, you know that it's important to the customer.
Most project management methods push waterfall development - with its huge reliance on time-consuming and error-prone upfront analysis & requirements gathering.
Of course, they hate requirements changes. And of course, their initial requirements are usually wrong - and fail to meet the need.
The answer isn't to stop changes - but to use methods that aren't so vulnerable to impact of change - like patterns, agile methods, passionate & highly skilled staff, etc, etc.
Our most famous crash and burn ( to date ) was an attempt to migrate a number of different applications to an Oracle Applications suite.
We expected a web based desktop client wouldn't require configuration; a jinitiator component required numerous desktop visits.
We expected streamlined operation; In fact the replacement product required more end user data entry and provided less critical information ( i.e. fewer metrics the end users have come to expect).
Management drank the marketroids cool aid. They should have asked the end users to evaluate the software before commiting to a purchase, rather than shoehorning the end users into accepting what was the cheapest.
A Human Right
So I worked on a project that spent 8 months getting through "project approval" on an 18 month scedule. Of course by the time it was approved, they still wanted it to be delivered in 18 months (from the start date) so we now had 10 months on an 18 month schedule.
Long story short - we delivered in 13 months, and were blamed for being late... Nothing like marketting and management not taking any blame for taking 8 months to approve the beginning of the project
I have mod points and I am not afraid to use them
On the flip side, if you show that you are brilliant up front and point out the problems early, they will tell everyone else what an incredible job you did, unlike those lazy, incompetent morons who they hired the last time who had to redo the whole project.
What planet are you from? If you tell the customer it can't be done for this that and the other reason, you're more likely to get canned as they move on to another consulting firm that can get the job done.
Even in this day and age, most people see software as a magical phenomenon that can do anything provided the magician is powerful enough. Telling them "no" means you're but a lowly apprentice.
I wonder if that actually has worked for anyone:-(
The risk based spiral I read about in Rapid Development many years ago has done the job for me (I rarely deliver late). Any changing requirements come in a later version and after an early version is available the customer has a lot more focus on what they actually want (because its an impossible task for them to know before development begins).
Risk based spiral also catches gotcha operational problems early on and forces me to be more formal with the release/unittests. It also keeps the customer onside since they see the project evolving. Any changes are fine and are all part of the process. Although this is only for small projects and YMMV as ever.
Of the six reasons adduced for project failure, these cannot be accurately applied post facto:
. pdf/
0 1/10/1844255&tid=156&tid=9/.
Use of an inappropriate methodology: Had they used a different methodology, they might have simply stumbled on different "gotcha's".
Lack of formal project management practices: This reason means they know a number of issues got out of control. They do not know how much more those issues would have been controlled, nor how much additional control might have slowed progress.
In addition, the "Requirements volatility" category begs a big question: requirements DO change over time; how is this category different from "Lack of formal project management practices" that would have planned for requirements changes?
It is interesting that "Project complexity" falls low on this list, because it is the most clearly proven reason why project plans fail. See this website for a fairly formal proof that project complexity cannot be estimated in advance: http://www.idiom.com/~zilla/Work/Softestim/kcsest
This proof has been discussed at slashdot here: http://developers.slashdot.org/article.pl?sid=02/
I'm working for a large Telco doing roughly 80-120 IT projects every calendar year worth about $200M. Most of them get through in one way or another, but some fail spetacularly and all of them have ridiculous overheads, delays and frustrations.
Best example of a crash-and-burn is a transaction engine designed to process a simple text file from another company. Should have been 6 months/$500K, project actually folded at 2 years / $3M and now we're going round for a second bite at the cherry (but with a new project name!!!).
Why do they fail ? Lot's of reasons.
Sometimes the user's requirements are unclear. Sometimes we're using the wrong spanner for the job. Sometimes the team loses the plot and we get a jumbo jet when we wanted a paper air plane. And we're always under pressure on time, but that's business - if we don't get there first someone else will.
What's the root cause?
Complexity. We let our systems get too complex and now a two line code change can cost >$500K because the down stream effects will hit ten other systems that generate $1M/day of revenue.
The moral - KISS. Use the simplest solution for the job. Don't let the sales guy run away with it, don't let your geek-ego run away with it, don't let the user's get over excited and your project might just come in on time on budget. As someone else said... it isn't rocket science... or shouldn't be...
Don't look back the lemmings are gaining on you
Thanks mirrordot!
Tiwana and Keil were asking MIS directors what *they* thought, not project managers or developers, leading me to believe that this is more based on client perception than someone with experience working on said projects.
That said, they ranked changing requirements last when talking about risk of failure, and actually said that inappropriate methodology was the top reason of project failure.
Now, while a lack of any sort of methodology is a disaster waiting to happen, I have a difficult time believing that a bad fit for a project creates more risk than project complexity and shifting requirements combined, as they suggest.
*sigh*
Do you really believe that a client is going to place shifting requirements as a risk? After all, they're the ones asking for the changes!
> management tries to replace good ol' fashioned thinking with
> process
That's because that's what they teach in business schools. Businesses are about repeatability and consistency. It's good if you can make a cup of coffee. It's great if you can sell a cup of coffee for more than it cost you to make it. If you can make a good cup of coffee and sell it profitably one million times, you have a healthy, sustainable business. Obviously, the third action requires a process.
That's why sometimes people make a distinction between management and leadership. Management is about incremental cost reductions and process improvement. Leadership is about changing directions and determining new courses of strategy.
The problem is managers often confuse themselves for leaders and leaders confuse themselves for managers. That's why many start-ups fail: the person who started it had great leadership skills (coming up with a new idea for a product), but poor management skills (taking that product and building a sustainable business out of it).
Management is always looking for a process. The problem is creative people can feel stifled by the process and poor performers can hide behind it. A good idea is to have a mixture of people (managers and leaders) in different roles so you have elements of strategy combined with repeatability. Obviously to do that you have to have effective teamwork and people that can get along with each other. I think the relative scarcity of all those elements (managers, leaders, teamwork) is the reason why failure is so common.
Insert simplistic political, ideological, or personal proselytization here.
Go read! Better than that, get a university degree. The more liberal the better. Honestly, it is worth it.
Yeah a good old liberal education. That's the ticket!
Then there'll be no need for stringent project requirements. Your design committe can concentrate on making sure the user interface is esthetically pleasing, non-offensive, accessible to the visually challenged, the mentally challenged and all other differently-abled beings regardless of their gender and sexual preferences, culturally neutral, politically correct in all ways, environmentally friendly and completely sanctioned by the UN and Amnesty International and Greenpeace.
First, a survey based on data "as reported by development project managers" is suspect to a high degree. Obviously their view will differ from that of the senior manager and from the actual developer. SO I will put little stock in that survey.
But the question is valid: why does it always fail?
Personally, I see a mixture of the following:
I am not ranking them in importance, will leave that to others!
Mike
---
BDOS ERR ON A:>
It may also be that the market is not completely understood and, as such, neither is the solution. If you can say you completely understand any market, you are a better man than I. Or maybe you have the luxury of ignoring your market.
Bottom line, I believe that there is a certain amount of understanding beyond which trying to understand the problem further is wasted effort. This point is reached far before the amount of uncertainty or "lack of understanding" reaches zero. It happens even sooner in rapidly shifting knowledge domains or markets. Unfortunately, that's also where the most profitable use of software lies.
That is all.
News flash: software is nothing like building construction.
Helping with organizational effectiveness is our job.
Projects work best when done by a small team with a strong vision. Even if the small team is a team-within-a-team. You can add project managers, junior coders, analysts, other support people, etc. But when the project gets moving, no matter how many people on the team it seems like it's always the same three or four guys who are doing the lifting. And they become very good at humoring the project managers.
You link in a ten year old article and this is supposed to be relevant? Software Engineering suffers because software can be made to seem to work and managers see it and say: ship it. They can't get away with this in construction or in electionics. But software is not as mature a discipline. But engineering for quality is a very important thing to do. And engineering is not something that software schools tend to teach. Why? Because anyone with a brain in their head went out and got rich. And they didn't care about doing things as engineers because they got seduced by the money and the stock options. The best software engineers, as far as I can tell, are the ones who were trained in another engineering discipline and then moved over to software. And then there was a whole generation of software accedemics who got seduced by the money and tried to patent their way into an easy retirement. They acted like experts and know it alls and stood in the way of real engineers trying to do real work. But engineering when applied to software has very strong results. It might not make people rich and so managers aren't interested. In software often when things are not engineered the software doesn't work. No big deal. In Civil, Electrical, and Mechanical engineering if things don't work correctly often times it means that people die, the building falls on them, they get electricuted. Fortunately there are what are known as real-time software engineers. And these are the people who do real software engineering. They aren't just trying to push through standards that they have patented clandestenly so that they can get royalties. The greed of the first generation of software engineers is why software engineering suffered. These people got rich, cashed in, and then got out of the business.