Slashdot Mirror
95% of IT Projects Not Delivered On Time
An anonymous reader wrote " The Globe and Mail reports that 'A new report conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups are not delivering some number of projects on time or to the full satisfaction of the business executive.' The article goes on to discuss the reasons for this pervasive (perceived?) problem. The article mentions Info-Tech's reasons: unrealistic time frames, staff shortages, and poorly defined project scope. However, the article's author lays the blame with vendors."
I'd say its actually closer to 100%.
Actually, it really depends on who they would ask in a company. Whether it be
the business executive (probably a higher estimate)
the IT middle manager (lower estimate)
the IT worker (who would think that they are on time)
or the customer (who sometimes have unrealistic expectations)
95 per cent of information technology groups "are not delivering some number of projects on time or to the full satisfaction of the business executive."
:-) I say that joking, but have seen discussions like this almost erupt into fist fights as the sales staff makes promises to customers that are either 1) blatantly false or 2) concepts under development and are nowhere near "production".
Could it be that marketing is always overselling the product? Seriously. I cannot count how many times I have heard (in the past now I am in science), "oh, yeah....well, you need to include feature X because we told customer Y we already had that feature". This is often followed up by the engineer muttering under his/her breath "Dumb jock.
So, this is another example of why pre-announcing products is a baaaaad idea. Treat your customers with honesty and announce the product when it is ready and not before. Again, this is why vaporware only serves to irritate your customers and build expectation of a product that is not always delivered.
I also believe the fundamental problem is that managers these days (in many cases) no longer come from the ranks and are not engineers. So, they do not always understand what is involved in 1) building the codebase 2) testing code base 3) proper interface design 4) end user testing 5) documentation 6) making sure it does not suck.
The last point is where most executives seem to get hung up. More often than not in most companies, executives really have no idea of what makes good code and all too often, what makes a good product. Come on now, a good portion of executives can barely use their personal computers to answer email or browse the Internet. When you have companies run by executives and managers that have come up through the ranks, you are much more likely to get quality which often is much more important than meeting an arbitrary deadline.
Visit Jonesblog and say hello.
95% of the time, the business changes their mind about the project and/or doesn't know what they want, anyway.
The Globe and Mail reports that 'A new report conducted by market research firm Info-Tech Research Group says 95 per cent of information technology groups are not delivering some number of projects on time or to the full satisfaction of the business executive.
The answer to this problem is change, and isn't change always the answer?
Consider if you will for a brief moment the vast difference between the average executive and the average programmer. Programmers are generally broad-picture thinkers who solve largely complicated problems that regular folks can't possibly wrap their heads around. The executive runs on a schedule and uses reports and correspondance to understand what is going on, because business folks have to judge their employees and projects.
These two groups are forced to work together, and we expect good results? We need someone to interpret between these two groups! The HR department can't regularly serve in the interpretive capacity, but perhaps they should.
Managers generally don't want to give the programmers the whole picture, because management often believes that they are superior in rank to programmers, placing the programmers on a need-to-know-basis, only. Huge mistake.
What programmers and managers need to do is realisticly approach their solutions together. They need to be honest with each other. They need to share each other's thoughts and feelings about the subject matter. It's not happening today.
The programmers need to come to the table and care about their customers a little more. The managers need to come to the table and care about their programmers a little more. The customers need to be more specific and realistic about how far their dollar can go. Then deadlines will be met and promises kept and successful solutions provided to customers.
I encourage a no-holds-barred approach to project management. The superior product is developed using the Agile method.
The dangers of knowledge trigger emotional distress in human beings.
A study shows that 95% of clients don't know what they want.
TFA does not say "95% of IT projects not delivered on time". It says "95 per cent of information technology groups are not delivering some number of projects on time..." Big difference.
Road Construction Projects? or Pentagon Contracts?
Us: We can do that for $x in 12 months.
Customer: But Joe Bloggs says his company can do it for $x/2 in 3 weeks!
Us: That's simply not possible.
Customer: Well, for that sort of savings we're going to give them a try.
11 months later and $x^2 later they're still waiting for Bloggs to finish but by then we're on the dole and Bloggs is laughing all the way to the bank.
TWW
"Encyclopedia" is to "Wikipedia" what "Library" is to "Some people at a bus stop"
Then they are either padding their project plans way too much, or are really not trying to do anything new.
It doesn't matter how much time I spend in the pre-planning stage, meeting with users, coming up with all the needs and feature requests. Once the project is in progress, and specially as you start demoing it, the end users will start adding requirements that had never come up before no matter how detailed you tried to be. And of course all the "wouldn't it be nice if"s.
Sometimes some of the new requirements or wants involve going back and rewriting a good chunk of code, or changing the DB design, etc, no matter how carefully you wrote your code and flexible the code may be.
First of all, the report focuses with Candian firms.
Second of all, rather than delving in to the varied array of processes and methodologies prevalent in the software development arena and reviewing advantages and disadvantages of each, the report goes in lenght talking about Vendor issues. I dont have a clue why.
Right from the Waterfall approach, or having no approach as well, we have RUP, XP and a mix of each playing itself out for the last few years. In my past projects, I have implemented each or a customized version of each and has varying levels of success. The biggest issue that I have so far seen is the lack of adequate knowledge in each methodology that someone who starts implementing any approach, either loses interest and resorts to a quick fix at which point the process starts to wither and die. More over, to some of the developers I have worked with, its not process, its documentation. CRC Cards is not a design tool, its documentation, its impediment to writing code. Much of it has to do with no academic background in best design or coding practices. They have heard of design patterns, and probably has used MVC to death, but when it comes to designing a system, its back to "lets start writing code rightaway and maybe the design will flesh out over time". The system gets built, but it suffers from Simplicity, it has very low quality.
I have seen a lot of firms talk about having a process, they love throwing RUP in the air, but nary a project which has successfully or much less adequately implemented any sort of process. They talk about Use Cases/ User Stories, but when the project gets kickstarted, they resort to SRS documents or less. And then they forget to adequately keep them up to date. Many even have Requirements management tools like Requisite Pro which hasnt seen daylight yet. Fuck the tools, atleast have a damn process. Many dont even define success at the outset of a project, no acceptance criteria either. They end up in a deathly downward spiral towards absolute failure dragging their clients with it.
I love XP, I love it for several reasons. Having a Client involved during all phases of the project, much less have a day to day interaction with the team is a phenomenal idea. I have had great success with this, but it becomes a bit of a problem, when the project is outsourced. No amount of communication (documents/mail/phone) can stand up to having a person next to you to tell you whats important and whats not. I love pair programming even though I could never fully implement it, when the client doesnt believe in it. I have tried pair design and have had great success. I have a few developers reviewing Test First Design and this has limited success as well. Limited, since the developers rebel, having low discipline and patience to write tests as they code. They are tutored and trained in the ways of "lets code first, maybe manually test it later and let the QA worry about it".
I would like to hear about other's experiences as well.
Rapid Nirvana
Could be seen as 95% of IT projects not given enough time for completion by marketing
The only things certain in war are Propaganda and Death. You can never be sure which is which though
This reminds me of my PM class I had. ..., then a project plan, then an end date is to be derived.
In a nutshell, the instructor said that the requirements, then the specs, then design,
The guy next to me who was a PM just shook his head and said, "No, the end date comes first and then we figure out how to get it done." I have had the same experience in my decade+ of experience.
The instructor said that's why most projects fail.
I suppose this was modded Funny because it is true. I have not seen a project yet that hasn't resulted from project bloat. As the project progresses new deliverables are tacked onto the end. One can try and have the project plan set in stone at the beginning, but it never works because all the new stuff is "critical to the business".
So how can any project meet it's deadline under these circumstances? (that was rhetorical. no need to actually answer)
-- Thou hast strayed far from the path of the Avatar.
They state:
"95% are not delivering some number of projects on time or to the full satisfaction of the business executive."
This could means that 99% of the projects attempted were delivered on time by all IT groups which is a more, or it could mean 99% of the projects were delivered late. By using the phrase "some number" this statistic is utterly general, and wholely useless.
Oddly enough, later in the same report they state "the majority of IT projects are in fact delivered on time" which really what counts.
The fact that IT groups do not deliver on time 100% of the time should be no surprise. The fact is that there simply aren't any professions which bat 100.
Botton line, stat is completely pointless.
Rich...
Excerpt of a typical meeting. Note that the day is Friday.
Manager: When can you have X finished.
Programmer: I need two weeks to do what the client asked.
Manager: I told them you were almost finished can you have it by Monday?
Programmer: I didn't start until yesterday I need two weeks.
Manager: Ok the client is expecting it to be ready Monday you'll have to work the weekend.
Programmer: I was planning to work over the weekends to get it finished it two weeks.
Manager: (Clearly Frustrated) I need it Monday. The client will have to punchlist what does not work.
The above exchange is very close to what was actually said. If this was the first company where I had heard that conversation I'd be bolting. Unfortunately no matter where I would run there would be a manger and programmer having the same conversation.
10: PRINT "Everything old is new again."
20: GOTO 10
- Failure to Understand Business Need
- Trying to Solve Training/Documentation Issues via Engineering
- Scope/Requirements Creep
Join me next week as I discuss the problem with dumbing down your architecture so that you can hire morons for less money to maintain it when all your best talent gets fed up with their 2% raises and quits."I have never won a debate with an ignorant person." -Ali ibn Abi Talib
And a lacko of respect for how difficult software can be to engineer. Managers want everything done tomorrow, and unlike engineering a skyscraper or a bridge, managers don't understand software--they can't see anything tangible, and they see a gui mock-up and think its 99% done.
Sorry if I sound cruel or mean to managers, but I've seen it over and over and over again. Managers are far too often people who:
a. Have never written software
b. Were bad software engineers, so they got promoted to management.
[FromTheMorning]
^^^ enuff said.
www.rexguo.com - Technologist + Designer
How many non-IT projects are done on time?
;)
Let's see... Boston's Big Dig. Nope. Designing a new aircraft carrier... nope. Red Sox winning the World Series... badly delayed
I wonder if in general it's creative projects or maybe highly complex projects that suffer lousy under-estimates for completion dates. Many software projects (i.e., MS Longhorn) are both.
On an unrelated note: I wonder how project planning estimate accuracy correlates with the experience level of the person making the estimate. Because if the IT industry tends to burn out the young people os that there are't many older IT people, that could contribute as well.
Duh! All Enterprise IT projects are like custom cars of the 20's and 30's. Each one is hand crafted by a small group of skilled craftsmen (coders, artists, project managers, etc.). No two implementations are the same and the users are just not that sophisticated. Every Enterprise Graphical User Interface I have seen is just an electronic version of "how we have always done it".
/bin, /sbin, kernel, etc. that we have been through in the last 5 years. 10 years ago most Enterprse systems did not exist, were running windows 3.1, consoles or just purple ink memo's.
Outsiders (the clients upper management) will apply time measurements to IT projects as if they are building bridges or building cars. These are unrealistic because both those industries have been around for 50+ years. The IT industry is immature and still growing. Just think how many languages you have to know just to do your job? Or how many versions of compilers, or how many changes in OS, dll, registry, configs,
Come on, until there is standadization in tasks (what the client wants to do), interfaces (how the client wants to do it) and tools (how we make it so), all IT projects will be optimistically scheduled and all projects will be under time pressure from the beginning.
I won't even start on budgeting of projects...
Ok, I'm down off the soap box.
-- Andy
And this is a problem because...?
As a developer, most of the time I couldn't give two craps in a tin can about what Darth Veep thinks. He is, after all, a rep from the Dark Side.
On the other hand, meeting the design goals, generating maintainable code, facilitating the QA process--those are important.
Writing software on a delivery schedule promulgated on the premise that "we need this revenue in Q2!" is short-sighted bordering on the moronic, and a great way to guarantee failure across the board.
One trend I've noticed with increasing frequency is for a suit to push for an "aggressive schedule" only to move on to another organization before the results of their actions can be felt.
I spent most of last year on a project like this. I personally spent almost 3200 hours on the project, and I know the rest of the staff was working 50-60 hour weeks the entire time. We managed to bring the project to a successful completion on time, but our Manager and his Director were both gone (to the same other part of the company) before the project was due to be delivered. The result of this was that we spent as much money as planning a much more reasonable schedule, but were specifically directed to take short cuts to create a barely workable solution that would create more work in the future.
In the end, a lot of projects either fail or are minimally usable as a result of poor management decisions. The irony is that the decisions are made in the name of saving money, yet the projects cost as much or more than if a more reasonable approach were taken.
Exactly the point. I don't know why this isn't totally obvious. Once somebody makes an estimate and establishes a schedule, no matter how they arrived at it, everything that doesn't measure up is deemed a failure. In almost 30 years of programming I've rarely seen an incompetent project team goof off or in other ways screw up a project. But I can't think how many times everything is going just fine, it's just not going the same as somebody 6 months ago thought it would. The staff is put under tremendous pressure to meet the pie in the sky deadline, features get cut, the customer is not satisfied and the project is deemed a failure. Those projects didn't fail, the estimate failed, and more often than not the estimate was simply designed to agree with what somebody higher up the food chain pulled out of their ass. How hard is it to see where the problem lies?
There have been many attempts to improve estimates by profiling code... a current one in vogue is called TSP - Team Software Process. These methodologies tend to be very top-heavy with record keeping, and assume that people work in ways that they really don't. For example, you don't think about only the UI for 11 minutes and then only the middle tier for the next 13 minutes. But that's the way you have to log your time. So your history data tends to be very approximate. On top of that you always have to factor in what fraction of time a developer is actually going to be doing productive work, which can be anything. Where I work we use 50%. So in the end after doing all that bookkeeping, you end up with a schedule based on making your best estimate and doubling it, which is how a lot of people who have no methodology do it anyway.
7) enter the market too late.
I have never worked in a company (only academic... sweet...), but can imagine 7) is why 1) to 6) are neglected.
True, but think what this implies: 5% of development groups never deliver a project that is late. Amazing...
Have you read my blog lately?
Get this:
- I scope a project
- I pitch it to mngmt
- Their response -always- is "be quicker"
- My response is, quicker means either: less qualiity, or more money, ypou pick.
- They say, you get neither.
- Ill say, sigh, Ill -try-
- It gets delivered on my original timescale
- They fuss about the project being "too late"
Do what -smart- project managers do: overestimate everything.
When I started out on my own I made a very early decision to charge 50% upfront on all contracts. It's very hard to pull off but overall it's been very successful. Process goes a little like this:
:)
1: Budgetary Quote.
2: Requirement Gathering (We assist)
3: Outline Specification (Huge number of meetings prior to this point).
4: 50% Non-refundable deposit.
5: Detailed system bible.
6: Changes to system bible (Chargeable).
7: Develop / Change / Rinse Repeat.
8: Finish Project (Final 50%)
9: Support
We have agreed to refund one deposit in the last two years (We screwed up their requirement gathering). We have had two clients pull out and lose their deposits. Everyone else has been happy, and through good communication hasn't had a problem when we have charged them for modifications to the system bible half way through the project.
Of course, the worst part of doing it this way is when some ass wastes weeks of your time and walks away with your outline spec (No doubt to give it to Joe Bloggs or use it in-house) having paid you nothing. It's probably worth noting that we don't always do the full specs if we don't trust the customer.
Oh, and one final bit of advice - GET TO THE TOP PERSON IN THE CHAIN! If someone is likely to override the decisions of the person you deal with in the company, start involving them. Even if it's just an email or a meeting to make sure they are happy with how things are going. Avoids some serious grief at the end of the project when you find you completely missed out on the features that the purse holder wanted!
People that believe in their opinions don't post AC.
So, 51% or more of the projects are delivered on time and on spec.
BUT if you've EVER missed a deadline or been off-spec, then you get counted as bad.
If you deliver 99 projects on time and on spec, but fail on 1 project, you're counted the same as if you failed on 100 projects.Right.... that tells me that the "answer" to this "problem" isn't technology. The "answer" is to have the IT managers take a few marketing classes and spread the bullshit to the "business executives".Again, all it takes is 1 failure to be lumped in that group. No matter how many successes you've had.
"So, you've solved world hunger, the arms race, poverty, racism, nationalism and have single handedly established a viable human colony on Mars. But we're really concerned about that jay-walking ticket from last year. Let's focus on what you can do to prevent such a failure in the future, okay?"
If by 'final ... time frame', you mean the actual delivery date, rather than the promised delivery date, then you are correct. But this is not just a software issue. Who decides what time your garbage gets picked-up? The garbage men. Who decides what time your pizza arrives? The Deliverator. Management is free to: 1) fire people who keep blowing management deadlines, 2) add resources or 3) do the work themselves to take up the slack. When it comes to programming, most management deadlines cannot be met by anybody, adding resources usually slows-down a project, and management couldn't code "Hello World" to save their lives.
As a result, they have absolutely zero recourse to the fact that software development takes time. They can start creating 'better' deadlines, or they can stick to the status quo and expect to blow their 'bad' deadlines.
Censorship is telling a man he can't have a steak just because a baby can't chew it. --Mark Twain
Why not have a headline "95% of IT employees will lose their jobs this year," based on an article showing that 95% of IT firms will terminate at least one employee this year.
While highly plausible, this 95% statistic is worthless, and the headline is not at all supported by the article. Actually knowing what percent of IT projects are late might be interesting.
Can anyone tell me how to set my sig on Slashdot?
Look....
;)
Programming is math. And math is hard. For starters, 99.9999% of all math is random and impossible to understand in the technical, math sense (proove) (extrapolation from Godel and Turing and Chaitin and Cantor...and yes, it really is 99.999....%).
When you sit out and design something to match a real-world process, you do fine. Then, you change something, you'll never know the impact of that change. You can't design for it in advance. A change of 1 character in the design could litterly require the entire code to be rewritten. You cannot prove how big your impact will be. Ever. That is why programmers get frustrated when customers change their mind "o but I just want it in this differnt order," "godamnit now i have rewrite half the loop..."
That is because good programming isn't just about being "smart", or about "planning"... most of the times, you are running against the fundamentals of understanding the algorithms in question, even with something as simple as lists or hashs or whatever.
The fact of the matter is is that programs are late because of bugs, and there are bugs because of the fundamentals limitations of math/the universe. You can't just smart them away cause you're some genius coder. All the genius coders write in their own designed language that best matches their thought processes and is easy to rewrite (i.e. they just gave up and went with LISP) with the assumption that they are going to rewrite most of the damn thing anyway at all stages of the game and the quicker they can rewrite it the better.
And that is why all these languages are getting closer and closer to LISP was 40 years ago...python, java, smalltalk, etc etc etc....automatic memory management and fast re-write cycle is the best way to write code for 100% of all projects (sans anything with 100,000+ intensive simultaneous users or an airplane code or something when you are bearing up against the engineering portion of things).
That's just my opinion. But I think it's fairly accurate. I've been programming from birth it feels like and I studied the math and I now I write for a large corporation where every schedule slips all the time and I have to deal interpreting customers and figuring out what they want and all I have say is:
There is no magic bullet and there NEVER WILL BE. Read Godel, Turing, Chaitin, etc. You'll be better for it. You're not going to fix your problems by using OSX instead of Windows or Oracle triggers instead of MySQL+PHP or Object Oriented instead of functional or procedural or XML web services instead of TCP/IP and binary. None of those things fucking matter, ok, fanbois? They're corporate games, that's it.
The best you're going to get is to find a language that fits how your brain thinks and that you can rewrite things in QUICKLY (i.e., don't even bother with the write-compile-link cycle... write it in LISP then have something covert the LISP to compiled code at some later day after you've profiled it)...
o, and a good text editor
and if management gives you shit, tell em to jump off a cliff. it's their only job to schedule, and if they aren't smart enough to schedule things with the understanding that schedules change for things that have nothing to do with how smart or good someone is as a programmer, they're worse than worthless anyway and you'd best find a new boss quick.
If there is a budget, if it will realistically cost 10 grand, say 15 and go "Look we have an extra 5k for you!"
Unfortunately in the business world, when you succeed in this fashion they expect you to work miracles next time under truly unrealistic conditions. Sucks.
No sig for you!!
Quit beating around the bush - you're saying we should lie.
What are the design constraints of a bridge? It generally solves a simply stated problem - get X lanes of traffic from point A to point B. And failure is not acceptable. Due to these constraints millions of dollars and years of planning and construction are available. This might be comparable to the Shuttle software discussed a few weeks ago, but not any projects I have worked on.
Consider the construction of a house, or an addition to an existing domicile. Price is a significant factor, and the customer has many arbitrary constraints (call them "aesthetics"). In many cases the customer isn't sure what they want until they see what they don't want, which requires rework. There is no official certification process for most construction trades - only specialties like electrical wiring. So it is difficult to know how good a crew is until you work with them. Many (if not most) construction projects like this run over budget or over schedule.
I think writing business software is more like building a house. The constraints are unique and vague. The workers vary in their abilities. And the customers are cheap bastards. Projects in this environment have very little chance of coming in under budget and on time.