Slashdot Mirror
Is Your Development Project a Sinking Ship?
gManZboy writes "Everyone knows that some software development projects succeed and other fail -- the question has always been 'why'? I'm sure we all have our favorite (likely anecdotal) explanations. Well, these guys decided to actually go out there and do a formal survey, and they've got some real data on why projects actually fail (as reported by development project managers -- care to guess where 'changing requirements' ranks?). They've developed a diagnostic formula people can use to gauge the likeliness that the project they're working on right now is (or isn't) going to fail."
I used to blame "constant client requirement changes" for failed projects as suggested by my project manager.
s before signing off the budget.
Later I realized, as suggested by the senior management, that a good project manager should not let that happen had he properly designed and managed the project.
Recently I started to think that maybe all failed projects are due to the delays inevitably imposed by the senior management who requires many policies/protocols/documents/approvals/discussion
These delays introduce deadline pressure to project manager, and allow too much time for client to ponder about other features, and most importantly, give breathing space for competitors to come up with similar products BEFORE we do.
Rock that crushes, Paper & Scissors that don't matter.
So, if I know it is going to fail, do I still have to try?
DAMN YOU OCTODOG! DAMN YOU TO HELL!
Changing requirements is far less bad than a frozen spec based on overanalysis by MBAs who never spoke to customers.
Are they really getting at why projects fail, or are they just getting a good record of what people on failing projects like to bitch about?
Changing requirements blah blah blah not my fault blah blah blah...
Fair management expectations
+ Well allocated budget
- Patch fixing firedrills
- unnecessary marketing spinoffs
+ free donuts
= success
People who sit around for months or years trying to design the perfect system. It doesn't exist. Compromise gets projects done.
... did they fail or succeed?
Our project is currently suffering from aggressive scheduling. We are put on too tight of a timeframe to allow even the most minor setbacks. Changing requirements seems to be the nature of the game, and when we dont allot enough time to accomodate these changes, we get into trouble.
It's as simple as that unfortuantely - we *never* learn from our mistakes. Over the last thirty years every system we can dream of has been built from nuclear power plant control system to stock market analysis systems.
Yet we keep playing the buzzword bingo with our new systems, e.g. "Extreme programming", we still keep promise a schedule we can't keep to, we still allow the customer to shift requirement much later in the project than should be allowed, management still don't have enough dialog with the programmers on the ground floor, the list goes on..
Wake up! We're not special.. the construction industry has been doing huge projects of equal complexity for centuries. Get past your intellectual snobbery and start working together..
Simon.
I've done a lot of thinking about this...I've come to the conclusion that too often, management tries to replace good ol' fashioned thinking with process. It doesn't work. People tend to get focused in on what they're doing to the exclusion of all else, and that means the smart people are cubbyholed and only have half the story and can't see where other parts of the project are failing, and dumb people have free reign over their little part.
If the ratio of intelligence to complexity is too low, then the project will fail no matter what process is in place or who is managing it. That's all there is to it. There's a lot of cool stuff out there to be done in development...sadly, most of the good ideas will never make it because the people working on them don't use common sense and best practices...they're just not smart enough to see what's important and what isn't.
This isn't one of those self-important diatribes from a holier-than-thou developer, either...true I'm a developer, but I admit when I'm too dumb to handle the particulars of a project; usually, that means the project is too complex for most people, but they press on anyway. Those projects always go down in flames eventually.
You have to know what the strenghs and weaknesses of your team and its members are, and exploit those to the fullest. Maybe, then, you can barely accomplish a project if the goal of that project is simple enough.
but have you considered the following argument: shut up.
We've had people who didn't know how to accurately scope business requirements, get buy in from other departments and generally "play nice" enough to keep everything running smoothly. Your P.M. needs to be able to be a hard ass, but also to be a buddy.
It boils down to excellent management skills and excellent people skills and without both you're setting a project up for disaster. A good P.M. needs to know when to tell senior management it's asking for the impossible too, and a good P.M. needs to know he has kung-fu so he can get away will telling senior management their idea won't be implemented.
Is this simply the nerd version of the ages-old cosmo quiz? I fail to see how "The one-minute risk assessment" is any more comprehensive or meaningful than the "Does he think you are fat"-type quizes that make their way through women's magazines.
It's not to say that a good designer or developer cannot be a good project manager; it's just a different job, like asking a plumber to rewire your house.
Not only do client requirements change, but management is also responsible for fubaring things.
;)).
i have been part of a project (past tense) where:
- management delivered a much too low cost estimate in order to get win the bid.
- management then expected the project manager and team to meet the deadlines that were doomed in advance.
- the software design lead designed a behemoth of a framework full of performance and design issues.
- management did not understand that if you have unexplained system behavior, you cannot say when you will have solved the problem.
- hardware design was not reviewed, just like software design. this lead to huge problems just before and during acceptance.
-near the end of the project, more and more people were reassigned to a new project that has the ability to make the department manager look good to the head office. he wants to move up. In effect, succes of the former project became a more and more distant possibility until failure was assured.
and there are probably some other things that i either forgot or purposly left out (trying to repress memories maybe
Release managers can track requirement changes and their impact (effort, schedule) on the project. These changes can be reported separately from the primary schedule, so that everyone can see the impact of scope changes.
Change is not bad. Adapting to environmental changes (competition, customer education by early prototypes, vendor roadmaps) can make the difference between a one-shot failed project and a multi-generation successful product.
Big Visible Charts is a time-tested technique for non-political status reporting that helps everyone (from senior management to QA) take responsibility for the global impact of local changes. Grab a few unused monitors and create a wall-mounted status display with 1-minute project status updates, you'll be amazed at the results.
It ALWAYS comes down to people. This article looks to be a discussion relevant more to a commercial environment than an open source one, but I guess the same fundamental principle is true - without the right people you will not succeed. This means competent and motivated technical people, clueful and skilled management, and customers willing to be reasonable and pay for what they are getting. Take away any one of these elements, and there is no technique in the world which will result in something everybody can define as a success.
These guys break down the problems into useful categories, which will be helpful for good teams who want to know how to be more efficient. But for my money a group of serious, decidated people who honestly want to get the job done and do it well will usually get there, barring external factors beyond their control messing it up. It might take a while, cost $$, etc. but they'll make it, because they WANT to.
Many (I would even say most) successful open source projects succeed because they have one or several individuals willing to put the work in to make something happen. The tools they use or the way they work are less important than determination to get it done and do it well. Those without that wither on the vine.
In theory, commercial companies and development teams should be motivated by the $$ they are paid, but that doesn't always translate into doing the job well. There are PHBs, lazy workers, unreasonable customers, and all the other joys of life out there. There is no magical "business formula" which can transmute this combination into a good product.
Don't get me wrong, project management and efficiency techniques are a very good thing, but only when you've got the people to make good use of them.
... since 1994, the Standish Group has been publishing the results and reasons of IT projects. Go here for the original report.
We've gone from about 25% of projects being "successful" (on time, on budget, meeting stated needs) to about 31%. So translated, that means 2/3ds of the time you get into your car or get on an elevator, it'll work as you want.
Consistently, the top reasons for projects failing, for the past 10 years?
1. Unclear, poor requirements
2. Lack of user involvement
3. Lack of buy-in and support by upper management
I have to agree with other comments made, this isn't rocket science. We just need some time and maturity as an industry. Civil and mechanical engineering have had thousands of years to work out their kinks. The software engineering science has had to deal with technology and implementation far outpacing our understanding of the basics and principles involved.
But we're getting better.
Honestly, if the world at large knew how brittle, fragile and reliant on heroism most of the critical financial and industrial software was, there would be a huge outcry. It's one of the shameful aspects of our industry.
Neurowiz
I have lots of evidence of failed projects due to failure to plan.
It can take months or years of thought and discussion to reasonably avoid extreme catastrophies.
While it is silly to try to plan every detail and anyone who claims to do so is lying, a simple elegant, successful general approach is seldom the first one to pop into the head. It takes a lot of thought. Of course, for those incapable of such forethought, why not fail earlier rather than later.
In 90% of the subprojects of construction, a manager can walk by in a few seconds gauge: - progress - quality of work - time to completion - implications to dependent subprojects Can't do that with software.
Hey, I'm just your average shit and piss factory.
Having many years of successful software project management under my belt, I can tell you it boils down to two concepts: professional training and discipline.
There are a million and one books and surveys and they all say the same thing. First, there is a formal process for the development of anything (not just software). This starts with the formal documentation process and meetings to discover functional and non-functional issues. Second, there is a very strong sense by everyone to want to adjust it a little more. From senior managers who allow scope creap to managers who want steps to be cut to make up time to programmers constantly who rewrite the code because they think they can squeeze 5% more time out of a loop that runs for less than a second in a process.
Most people do not realize that in a successful formal process that the actual time in a software project that is used to build the software should amount to only about 30% of the project's development time. The other 70% is time spent on documentation, meetings, and testing to ensure that the 30% of time used on software delevopement is actually what the company is needing. And, it is discipline that keeps people on the project process in the face of the fear of not getting the project done right. The process has to be allowed to work, both to reach a project end point and to have unobstructed process from which to learn.
The part I get a kick out of is that just because people write software or run a company that they somehow think they just ought to know why projects work. If complex systems were just so easy, why would we need formal training? After all, anyone can build a bridge successfully without training, right? I am not being hard on people, though. I had this exact same though years ago and what I figured out is that the vast majority of the software industry is so poorly trained that it doesn't even realize that it poorly trained.
Successful software development books have been around for more than 30 years. Go read! Better than that, get a university degree. The more liberal the better. Honestly, it is worth it. Here is a good place to start: Systems Analysis and Design by Kendall and Kendall (ISBN 0-13-041571-5)
Bel, the mostly sane.. "Of course I can't see anything! I'm standing on the shoulders of idiots." -- Me
In my experience, it is usually drugs, alcohol, too much sleep, unconcerned management, or a combination thereof that causes projects to fail. Have you ever tried to project-manage after 8 double vodkas, a short nap, and a full rack of ribs?
We are one consciousness experiencing itself subjectively. Back to you with the weather, Bob!
Aaaah, that one is subject to the 95% rule:
The first 95% of the project takes 95% of the time, and the remaining 5% takes the other 95% of the time"
(loosely quoted from some fortune)
I'll suggest everybody who has not yet done so should RTF precedents for such a study...it is as ancient as it is true: Brooks "Mythical Man Month" describes the reasons projects blow up pretty well. For all the technology heaped on software development in the 30 years since the book came out, very little has changed: Software projects are complicated beasts attempted by mere humans. Steve McConnel's books will be more familiar to /. readers and his approach to project management tries to head off the "changed requirements" fiascos with a feedback and correction mechanism of frequent critical project reviews...I wonder if that actually has worked for anyone:-(
SLASHDOT: news for people who can't concentrate on work or have no life at all and got tired of yelling back at the TV.
i actually was "assigned" to supervise a death march project at my last employer. my "new" manager(6th one in one year) knew the project was going to be canned(didn't confirm the inevitable to me, even when confronted), and most of the people would be absorbed into other projects or simply layed off. why was it a doomed project? politics.
someone else in our organization (at another geographical location), happened to be better aligned with the top management group, and used this to their advantage to eliminate competing projects, or in some cases eliminate the internal competition and take the projects over as their own.
of course at the time i had no idea what i was getting into(or who my "competition" was). no matter what our team did to produce a superior product, our project was cancelled for reasons beyond our control. i ended up stressing out and nearly damaged my health and my relationship...
then i read a book call Death March: The Complete Software Developer's Guide to Surviving "Mission Impossible" Projects. i soon realized that we were set up as an ugly style project, doomed(in fact designed) to fail.
it's good to understand why projects fail, i have not yet RTFA, but i'm sure it will compliment some of the discussions/concepts in the death march book. good read.
three can keep a secret, if two are dead - benjamin franklin
One result of ad-hoc software design and implementation has been government regulation of software in the financial, security and pharmaceutical sectors.
One result of government regulation has been the emergence of requirements management tools like Borland's CaliberRM and Telelogic's DOORS.
These tools trace every functional requirement back to a business requirement. They also track the risk (schedule, safety, robustness, performance) of every functional requirement to the rest of the system.
Vague specification, like vague design is an indicator of not understanding the problem. The first step towards understanding the problem is categorization of ignorance, such as unexpected consequences already experienced by the project.
Good requirements management tools incorporate practices that have been proven to flush out vague specifications. Good traceability educates upstream participants so they can produce better specs in the future. Better specs yield better products, including better spec management tools
Here is a pretty good paper by Mary Shaw explaining why software is not yet an engineering discipline (IEEE). http://www.sce.carleton.ca/faculty/ajila/4106-5006 /Prospect%20Eng%20Soft.pdf/
"Is Your Development Project a Sinking Ship?"
Why yes, we make submarines. Hoo-hah!
It would be interesting to see such an analysis done with an open source-centric viewpoint: why open source/free software projects fail.
It would be necessary to structure the survey carefully to avoid the obvious results that don't contain useful information. For instance, Sourceforge is littered with old projects that never got past alpha or pre-alpha because no one was interested except for the project initiator (who never created enough of a start to encourage significant involvement from others), and the project initiator eventually lost interest him/herself. That may be the way in which most open source projects fail -- but that knowledge is of little use to someone running a project and looking for tips on management. There are of course books about aspects of this topic; but it would be nice if someone were to do a similar survey of open source projects that did get their legs underneath them, that did produce something that enticed involvement and an interested user community, only to eventually fail.
I'm working for a large Telco doing roughly 80-120 IT projects every calendar year worth about $200M. Most of them get through in one way or another, but some fail spetacularly and all of them have ridiculous overheads, delays and frustrations.
Best example of a crash-and-burn is a transaction engine designed to process a simple text file from another company. Should have been 6 months/$500K, project actually folded at 2 years / $3M and now we're going round for a second bite at the cherry (but with a new project name!!!).
Why do they fail ? Lot's of reasons.
Sometimes the user's requirements are unclear. Sometimes we're using the wrong spanner for the job. Sometimes the team loses the plot and we get a jumbo jet when we wanted a paper air plane. And we're always under pressure on time, but that's business - if we don't get there first someone else will.
What's the root cause?
Complexity. We let our systems get too complex and now a two line code change can cost >$500K because the down stream effects will hit ten other systems that generate $1M/day of revenue.
The moral - KISS. Use the simplest solution for the job. Don't let the sales guy run away with it, don't let your geek-ego run away with it, don't let the user's get over excited and your project might just come in on time on budget. As someone else said... it isn't rocket science... or shouldn't be...
Don't look back the lemmings are gaining on you
Thanks mirrordot!
Tiwana and Keil were asking MIS directors what *they* thought, not project managers or developers, leading me to believe that this is more based on client perception than someone with experience working on said projects.
That said, they ranked changing requirements last when talking about risk of failure, and actually said that inappropriate methodology was the top reason of project failure.
Now, while a lack of any sort of methodology is a disaster waiting to happen, I have a difficult time believing that a bad fit for a project creates more risk than project complexity and shifting requirements combined, as they suggest.
*sigh*
Do you really believe that a client is going to place shifting requirements as a risk? After all, they're the ones asking for the changes!
Chris Peters (former Microsoft VP) wrote an interesting documented called "Is Your Project Out of Control". It seems to have appeared on the net in various formats.