Slashdot Mirror
Can Software Schedules Be Estimated?
J.P.Lewis writes "
Is programming like manufacturing, or like physics?
We sometimes hear of enormous software projects that are
canceled after running years behind schedule.
On the other hand, there are software engineering methodologies
(inspired by similar methodologies in manufacturing)
that claim (or hint at) objective estimation
of project complexity and development schedules.
With objective schedule estimates, projects should never run late.
Are these failed software projects not using proper
software engineering, or is there a deeper problem?" Read on for one man's well-argued answer, which casts doubt on most software-delivery predictions, and hits on a few of the famous latecomers.
"A recent academic paper Large Limits to Software Estimation (ACM Software Engineering Notes, 26, no.4 2001) shows how software estimation can be interpreted in algorithmic (Kolmogorov) complexity terms. An algorithmic complexity variant of mathematical (Godel) incompleteness can then easily be interpreted as showing that all claims of purely objective estimation of project complexity, development time, and programmer productivity are incorrect. Software development is like physics: there is no objective way to know how long a program will take to develop."
Lewis also provides a link to this "introduction to incompleteness (a fun subject in itself) and other background material for the paper."
There is only two type of software schedules
1) As long as it takes.
2) Take your best estimate , and double it and add 5 or something....
It prefer the as long as it takes. Other wise you end up with something like Windows Me.
Cruise TT
The majority of Slashdot readers are students without any notable software engineering experience. Sure, not everyone here fits this description, but it's certain that there will be lots of hearsay, what-my-professor-told-me responses, and misguided personal theories based on blind idealism.
Software development is not a science in the normal sense. Designing large software systems is an art. It cannot be pigeonholed. Stroustrup has a lot to say about this when he describes the 'interchangable morons' concept in the 2nd edition C++ book.
Anyway, read Death march by Ed Yourdon, and the mythical man month by fred brooks, and antipatterns, any time someone asks you for an estimate say 'two weeks' and then bullshit from there on.
That is how it works in the real world. The numbers are essentially meaningless, but the bean counters and suits have to justify their existance somehow :-)
Can you imagine asking Linus when 2.5 will be ready ?
As they say, the first 95% of a software project takes 95% of the time.
And the remaining 5% of the project takes another 95% of the time.
--
Mod up a post Rob doesn't like and you'll never mod again
The reason that software development timetable "estimation" (guess is a better word) is so often wrong is that quite often you are not given enough information about the projecto accuratly pin down what your milestones are much less your final delivery date.
To accuratly plan a software release you must have the project, and all it's complexities and nuances down COLD. otherwise you are not giving an estimation, you are giving a guess based upon incomplete knowledge.
The question becomes, do or, can you, know the complete details of the project? In this, software development is NOT like manufacturing, but more like home construction.
Think about it.
Bugs Bunny was right.
Very large and complex projects do get completed, sometimes even on-time/on-budget. Examples include skyscrapers, nuclear submarines, aircraft carriers, power plants (whether conventional or nuclear), oil refineries, B-747/A-320, etc. And all of these systems nowadays have a software component as well.
So the easy response is that bad management in general, and bad project management in particular, is responsible for software project failures. While this is no doubt true, the next question has to be, why do software projects have such bad project management?
I don't have a good answer, but one thing that occurs to me is the lack of a fixed endpoint. When an oil refinery ships its first load of POL, it is complete. When an aircraft carrier launches its first plane, it is complete. But the amorphous and mallable nature of software means that it is hard to define an exact endpoint, and very hard to avoid changing the definition of the endpoint as the project proceeds. So things keep "creeping" along until disaster occurs.
sPh
Yes, you can guess how long it could take, and no, there is no formula for it. I can have a great day and produce 10x as much as usual. I can be lucky and my first guess of how to do it is correct, or I can take the wrong one and loose a week. You might predict for some kind of generic person, but I know for experiences that there are very large differences between how fast different people produce code/documents etc.
So it's all a loss? Nope, but you have to remember that it's not an exact science. It involves replanning, knowing your work force, letting the work force plan on their own, more replanning, experince, guesses, and whatever it takes. Honesty is also high up on the list, and not trying to do huge amounts of work in one go. Heck, there is so much about this subject that it would ages to describe them. My suggestion is, go out in reality, work, and learn.
Straightforward implementation, no matter how complex, can be scheduled accurately. Developing new technology cannot.
-=Best Viewed Using [INLINE]=-
This article is also at K5. In fact, it's the same article. If you want to get comments from two different places, then please post the article at one place, and post a link to it at other places.
Best Slashdot Co
We can get it done by next week! We can do this because we have just #defined a day as having 2000 hours.
When making estimates, people tend to sweep under the carpet, or simplify the things they don't know, but can be quite accurate estimate the things they've built before. That's why really large project fail so badly, because every single person involved im the project has many more unknowns than known things to deal with.
So, never say "How hard can that be?" before having coded up a small working prototype.
-- Another senseless waste of fine bytes.
Lewis also provides a link to this "introduction to incompleteness" (a fun subject in itself)
I started writing a paper about this topic once, but I never finished it.
-WetDog
My company develops turn-key systems. Sometimes we also develop custom solutions for our customers. Our customer base has increased steadily after the dotcom crash, when we switched from products to services. One of the reasons our customers like us is that we don't bill projects by the hour. We will the project on a fixed price, not to exceed, basis.
The programmers who work with us on a contract basis don't bill us by the hour either. After we have the design and we distribute tasks and prior to submitting the final estimate, we ask contractors to place a fixed bid.
We've done six major projects like this since March, and in all cases we finished within budget and on-schedule, and the systems are currently in production. They are all mission-critical systems running in either robotics environments or high-availability networks.
Our economic motivation is then to do things well and quickly in order to increase our profits. That also enables us to move on to the next project faster than slaving over some customer in order to bill the maximum hours.
As far as development techniques go, we adopted XP earlier on and it's working for us.
Cheers!
Ehttp://eugeneciurana.com | http://ciurana.eu
As a software developer, I would have to say that a majority of the development that I have been involved in or been aware of is of the manufacturing variety. Most business sofware is a DIDO job. Data in, Data Out. Make some fancy forms and reports and you have turned a database into a 'billing' system or what have you. There aren't really any new algorithms needed. Of course, there are a ton of them in use in the database server, the network protocols, etc. But you aren't developing those, just using them.
The reason that estimates are always wrong are *1* unclear requirements, *2* changing requirements, *3* complicated user interfaces, *4* weak focus on testing.
I find *1* to be the biggest difficulty. The prinicipals of a software project like to say things like "Automate timeclock operations" but as a developer, you need *A LOT* of information to do that. When you ask questions like "I understand that you do not want to allow any changes to a pay period after the checks have been cut, but then what are we going to do when travelling workers report their hours late?" Management thinks you are being a pain in the ass, but if you don't get it right, your project will fail.
I agree with taking a realistic estimate and doubling the both the developement and the testing estimates.
there are 2 kinds of people. those who divide people into 2 kinds, and those who don't.
There are four parameters to a software project:
- Quality
- Quantity
- Deadline
- Costs
In a competitive environment with humans involved, up to three can be specified. Not four. Good examples are:
- Many guidelines for managing software projects tell you to reduce quantity when you get near deadline.
- Some customers have a specified budget but really don't know how much software they can get for that money. They prefer to have costs fixed than to have quantity or deadline fixed.
- Sometimes deadline is so important, that costs may 10-double in order to reach that deadline, and quality and quantity may get reduced a lot in order to finish the project.
It is extremely important to realize the meaning of all four parameters before you can talk about estimating project schedules.
Lars.
Like any public forum Slashdot has a wide range of readers, a large number of whom actually work in the software engineering field [myslef included].
Anyway my personal theory based on blind idealism is that it is extremely difficult to get an estimate for completion right; short term goals are fairly easy to predict, because you have most of the information you require to make those predictions, but longer term estimates are much more of a wild guess. I personally thing its a consequent of chaos theory - a butterfly flutters its wings in Brazil and your software project instantly takes another two years! More seriously small errors in estimating components of a large project can induce large errors in estimating the time and resources needed to complete the whole project.
Linux is right with its "release when ready" motto. Since it is impossible to tell when it will be ready over such a wide range of groups and interests, you have to pick your release moments when they happen, not try and force them to happen.
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
There are tons of tools and techniques to developing software. Best practices abound in fact. Two things present in every form of good software development are analysis/design and project management. If you do the work in analysis and design you will be capable of building a good estimate.
That's only half the battle. Once a project is underway, keeping scope in check is critical so you need good project management. If you build a great estimate through analysis and design and then throw it out the window when you start writing code, you'll never have a good estimate.
Where do major providers like Microsoft and even Mozilla go wrong? Simple, they either jump in and start coding before they've completely settled on what they're building or they change their mind in development about what they're building. Either way, it screws up delivery dates.
Of course we torture people, we need the information --Gen. Pinochet
The biggest problem I've seen is requirements creep. Most often, you don't have a firm set of requirements to start with. Management and programmers both have a tendancy to view requirements documents and other formal software engineering practices as superflourous. The problem is that without a firm set of fixed requirements, you are always trying to hit a moving target.
Another problem is attitude, mostly on the part of management, but programmers are guilty too. One faulty attitude is that we are conditioned to expect immediate results. There's also a prevaling attitude that there is never enough time to do it right, but there's always enough time to do it over. This leads to undocumented, unmaintainable masses of code that either gets thrown away after a while.
Even worse, you wind up with garbage code that SHOULD be thrown away and re-written from scratch, but winds up getting patched and modified for years. I can't tell you how many times I've had a manager say "there isn't time to rewrite it, just patch it". That would be OK if you are only going to patch it once -- but you wind up patching the same program a half dozen times, and it winds up taking twice as long to do all the as it would have if you had just rewritten it from scratch.
Why is it that the proponents of "one nation under God" are so eager to get rid of "liberty and justice for all"?
You can develop properly, but you have to design modules with all specified functionality in mind -- no last second adding in "oh yeah, don't forget the login system" or "we're gonna want a WOW display attached to the processing so add in all these hack hooks at the last second into the core engine."
If you need that stuff, design it in from the start. Too many programmers worry about general design to make future expansion easier, while leaving out consideration for real, hard requirements that won't be implemented until later in the project.
And to avoid the problem with really bad bugs that are responsible for the (double it and add 5) estimation, take a little extra time to write exhaustive testing (as far as possible) of each module, indeed each function, to make sure it doesn't do something wrong when given values out of "happy path" input range.
I am for the complete Trantorization of Earth.
where'd you get that idea?
/. readers are programmers and IT people who come here to kill time at work.
i've always thought most
-c
I have discovered a truly remarkable proof which this margin is too small to contain.
True but most experienced S/W engineers or Project managers know that most projects slip because of changes to/deviations from the original project spec.
Fixed specs are much easier to engineer than those that continually change. You wouldn't easily engineer a bridge if the river banks kept moving.
I think experienced project managers know how to control the spec rather than the project. (I could be wrong - It's just what I've seen).
"Things that you own end up owning you" - Tyler Durden (via Diogenes of Sinope).
SW development is still more of an art than a science. That said, I've seen several fairly common causes for late software:
1) Lack of up front planning - too many projects fail to do proper initial planning - specifically defining the problem to be solved, producing detailed product requirements, and a detailed project plan (and then sticking to it).
2) Late (or incomplete) requirements - if you went to an architect half way through home construction and wanted to change the design of a house; you wouldn't be surprised if it fell behind schedule and went over cost.
3) Poor risk management - failure to track dependencies, too many high risk dependencies ("we'll build it on the next OS release, with the new compiler, and that SW package that our start-up partner will finish next month"), failure to make and execute contingency plans.
4) Failure to heed Brook's Law ("Adding software developers to a late project - makes it later.")
5) Failure to have read Deming ("You cannot test quality into a product").
6) General design failures - not assuring that product is scalable, reliable, testable, etc.
7) Failure to place a senior developer on the team that knows about the previous issues.
[Insert pithy quote here]
The best defense I've heard is that "Yes, everyone's estimate will be way off, but they are independent estimates of different pieces of code and when aggregated the standard deviation drops to a reasonable value". IOW, the estimate I pulled out of my butt will be way optimistic, but your estimate will be pessimistic, and it will all cancel out.
There are a few problems with this, rather nice and neat statistical trick:
1) As Michael Milken found out, the observations are not independent -- there are two many interactions between the components being estimated. In Milken's case, he argued that a diversified portfolio or junk bonds would have high yield, low risk charactersitics. Unfortunately, the performance of shaky companies in a market downturn is rather strongly corelated.
2) You need something objective to estimate. In our case, we measured the number of easy, medium, and hard member functions of classes that had to be implemented. See the problem? You need to cast your interfaces in stone, external, and internal ones, right at the start. On simple projects this is easy, but not on hard ones, as much as we all agree it is desirable. There is something called learning from one's mistakes and it will happen with anything novel.
3) This presumes that the design is sound. To ensure this we reviewed and analyzed and studied, and "damnit, you indented 3 spaces instead of 4...", well you get the idea. The closest scrutiny will find the obvious bugs, but not the really tricky ones.
4) This technique does not encourage the one thing that saves you in the face of change -- adaptive and modular design. You make things modular so change affects as little as possible, and you make things adaptive so change is as painless as possible. IOW, you plan for making changes bacause of mistakes. Naturally, this violates (1) above, so it is not permitted. The mantra is "Design it right the first time!" We know that we can get 95% or 99% or maybe even 99.5% of it right, but never 100%.
In the end, sure, we "finished" on time, but, er what was finished didn't work very well, and had to be rescued by the few who knew what was going on. To be fair, the design efforts and documentation helped provide a somewhat modular system, but the really important parts weren't documented -- we had reams of paper describing the "trees", but not nearly enough describing the "forest" as it were.
So, I'm skeptical.
I've heard that these techniques encourage "discipline" and help mediocre programmers contribute acceptable code. Well, where I work now, we have a policy of not hiring "mediocre" programmers. I can dump a suspicious log on someone and be assured that they WILL fix the problem -- I don't have to argue that there IS a problem ("but, the process, the process says this WON'T happen... your log must be a lie...")
You could've hired me.
...the real reason estimating doesn't work is that there's no way to predict how much time programmers will spend reading Slashdot...
Home of
I remember being in my software engineering class in college the day the professor was lecturing on "CoCoMo" (think it stood for "Cost Completion Model").
He very carefully laid out the algorithm - I don't have my textbook handy, but it involved elementary mathematical operations on estimated man hours, estimated lines of code, estimated overhead, etc., then at the end -- and I am not making this up -- they multiply the result by a "magic number".
Where did you get the magic number, oh sage of the ivory tower? Well, we just made it up -- it seems to work.
It hit me then that the whole discipline of estimating cost completion is all bullshit. You might as well be estimating with a crystal ball or divining the future with chicken bones. Since I've been working, the best advice I've gotten so far has been "take how long you think it'll take and double it".
If it ain't broke, it doesn't have enough features yet.
Where I work, we just take any estimate and multiply it by 3 and that seems to be a lot closer to the end result.
Of course that doesn't stop the managers from asking every day, starting on the first day, whether or not the 3 month project you're working on is complete.
You're close, but I think you're maybe even too optimistic.
I learned this about 25 years ago, while at a startup that was trying to build a computer out of the then-hot 6502-class microprocessor. The company tanked, never fully delivering. The smarter folks there (alas, there were not enough common-sense smarts where needed, just comp-sci-smarts always looking for another feature) knew the real Rule of Three:
Take the amount of time you think it should take. Triple it. Then increment the unit of time.
So three days is nine weeks, two months is six quarters.
Double plus one is, well, just too optimistic. Of course there are a lot of people who understand a "rule of three" that forgets to increment the unit, so the rest is just quibbling.
But hey, Microsoft did finally deliver something labeled Cairo (X P)! Lessee, that was due in what, 1995?
And Linux, while ten years old, still manages its desktop (rendering, fonts, etc.) somewhat worse than the Win95 GDI did. Nobody's immune.
When you construct a house or a power plant, you are in a business with subcontractors, that can take some of the risks. It is generally accepted to set a fixed price, because the procedures that are involved, are mostly known.
In software, however, most projects do not rely on known procedures. It is fairly easy to estimate the costs of creating 1000 different window layouts, which is a known procedure, but it is a very difficult task to estimate the costs of implementing the layouts.
If software would use as much energy on estimating each new task as construction projects did, developing software would be extremely expensive. Just imagine that you had to do a while-loop according to an ISO standard, and another while loop according to another ISO standard, because the two while loops were in different functions that were categorized differently by a third ISO standard. Instead we hire a bunch of programmers and make them program themselves. Sometimes we do it a little more complicated, like Open-Source, Xtreme Programming etc., but it's still a bunch of programmers hacking around.
The trick is to manage it anyway - and that's why managing software projects will always be risc management and not very predictable.
Lars.
This is the key insight, and the one that the "software engineering is a science, you hackers should go away" crowd ignores. There is no one answer to how well software development can be planned / estimated, it depends *enormously* on the kind of project.
I once has a person stand up and tell me that his organization could estimate sizable projects with great accuracy. After some questions, it turned at that the projects were essentially the same thing again and again. Duh.
On the other hand, I usually do a pretty good job of estimating costs and schedules, even when there are some significant unknowns. So perhaps the situation is not as had as some people are making it sound, either.
I've being working on large mostly web based applications for the past five years. There are always problems with timelines for software design and development, but I must say that most of the problems I've seing are not actually related to computer science. Whenever the task of time allocation is done by a marketing person, the times are wrong. When I get into the room with a couple of other developers, look at the problem that is set for us from the very beginning, from basically the reasons why this application is developped and who the users are to the point where we have to understand the datamodel, the time assignment exercise becomes much more clear. For a person with experience, it should be possible to estimate time for a task if this task resembles some of the person's experiences in the past. Of-course everything new has new risks associated with it, that is why you must always allow some slack for every specific task within the entire project. It also helps if you know who your developers are and if you have seeing these people at work before.
You can't handle the truth.
Works very well.
You have a shippable system every 2-4 week cycle.
Each new cycle nets more features.
WRT software schedules - seldom has so much crap been written by so many for the benefit of so few.
The problem is that the term "software schedule" is too wide a field to say anything meaningful about it. If you want to estimate how long it will take to put together a customized ecommerce web site, and the organisation has already built 5 of them, there is no problem. If you want to solve some problem that hasn't been solved before, it could take a week or a hundred years. Recognising the difference between these two cases is less simple than one might expect. And, if there's genuinely no novelty in the problem one should not be writing software at all. Someone should just write an application to solve that general class of problems.
People get unstuck when they break the problem down into small chunks and then guess a number on each chunk. Often the initial decomposition misses crucial interactions and needs to be refactored later on. This is a bit like answering the problem about how long is a piece of string, by saying - well the string eginning, a middle and an end, I estimate each piece is 5 inches long, so the string is probably about 15 inches long. Unless the breakdown has brought genuine insight into the unknown aspects of the project, the estimate it provides is worse than useless. However, since one can then stick things out in MS project, print out pretty GANT charts, etc, this estimate is given more credence than a number generated by just reading the spec and making an educated guess.
Part of the problem is that it's described as software engineering. Then we get all sorts of morons saying: civil engineers can tell us how long it will take to build a bridge, the problem must be that software engineers are unprofessional or that the subject is in it's infancy - things will improve. No, they won't, for the same reason that mathematicians couldn't tell you how long it would take to solve Fermat's last theorem.
http://rareformnewmedia.com/
I agree entirely. My software engineering class involved an overview of various estimated tools and it was all BS. The systems used lines of code as a metric. What a joke. I remember the magic number, which is just a way of them reverse engineering on historical data. The bean counters are desperate for a way to understand programmers without knowing a line of code.
-- Solaris Central - http://w
I'd have to agree with this. There are two major problems, the first being that the users don't really know what they want and the second being that almost always, the problems being solved are new problems, and therefore it's difficult to know what solution will best solve the problem.
Scenerio 1:
Q: It has to do *this*, how long?
A: X days (Not very accurate)
Scenerio 2:
Q: Find out what it has to do, spend TIME specifiying it, then tell me how long.
A: X days (Can be very accurate)
The Problem I (10+ yrs pro developer) keep running into, is that you figure out what it is to do, specify it very well, and then as you start developing it and delievering pieces for review, that specification is changed and you are plopped solidy back into Scnerio 1. Worse is when you think you're done, and begin QA and get SLAPPED back into Scenerio 1... or even Scnenerio -1 where you are trying to hack your guess at how it works into how it really should work.
M@
Krispy Cream is people
I'm not saying the majority of Slashdot readers are professional developers, but don't judge the readership on the first-posters.
That aside, my experience in software development (only 3 years) ball parking (1-3 days, 1 week-3 weeks, 1 month-3months) is usually possible, but tends to become wildly inaccurate beyond a few months. Regardless of what methond we use to determine timelines, some things always seem to slip, while others take a fraction of the expected time.
Well organized projects have unpredictable schedules. Badly managed projects have even more unpredictable schedules.
--john
The time taken for ONE itteration of the software lifecycle will be equal to the "critical path". (A "critical path" is defined as the path that absolutely cannot be reduced in time, because of dependencies on other work.)
Because the bulk of the work has been done, in the prior itteration(s), what you will get is a curve that is tending to a limit. That limit is the theoretical amount of time it would take to produce the optimal program for that specification.
Once you have extrapolated the function, then you can produce an estimate of how long (and how many development cycles) it would take to produce a software component of a given standard.
Note that CPA (Critical Path Analysis) also defines the optimal project team size and structure. No matter how many people you have on a team, you can NEVER produce a program in a shorter time than it critical path.
(ObTrivia: CPA is the computer equivalent of asking the following: If it takes 1 man 60 seconds to dig a post-hole, how long would it take 60 men?)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Ask a sharp programmer to estimate the time to develop a software solution and he might shrug and look irritated. Ask him if 2 weeks will be enough time, and there is an 80% chance he will say "of course" no matter what the task!
Gung-ho programmers are optimists. Couple optimism with the ennumerable factors involved in programming a non trivial application and you will get what we have today.
By the way. I am a programmer and I have little to no confidence in my time-estimation abilities, or anyone elses. It has taken me 14 years to come to grips with that.
--- -- - -
Give me LIBERTY, or give me a check.
- The requirements were fully understood and detailed ahead of time
- The biggest technical risks were understood ahead of time
The problem is that you can't fully understand the requirements and the technical risks ahead of time, since they can only be fully known by building the software. In the time that it takes to fully understand the requirements and the technical risk, you could have released version 1 of the software, which would be incomplete, and would miss the mark but would be:- A better attempt at understanding requirements and technical risk than armchair analysis
- An actual deliverable that, if you are lucky, might just be valuable to the customer
How many times have you spent 10% of your time on the hard stuff, and 90% of your time on the easy or silly details? Why is the program 90% done for 1/2 the project duration? Why do the requirements and the specs always change? Why does the customer always hate the first version of the program?That's why the open source rule of "release early, release often" is the proper way of doing things in commercial software as well. The emphasis on planning and control makes the difficulty of software development worse that it could be, because it focuses effort on nearly meaningless exercises (business and technical analysis) that don't uncover real risks nearly so efficiently as building working software.
See the agile software development movement for more.
People like to compare the software development process to manufacturing. But people also ignore the fact that before manufacturing there is design, which culminates in the first version of the object. Manufacturing produces versions 2 and beyond.
The process of developing software is more like the process of producing the ultimately detailed design. For software, manufacturing is a mechanical process -- duplicating the initial working version.
Now, with this view, ask how often the design for a product is completed on schedule, especially for a large complex product like an airplane (or the Intel Itanium processor :-)). I don't believe (I have no firm data) that the experience is a lot better than the experience for large software projects.
Chris
I would agree that experience improves ones skills at estimation. I wouldn't necessarily assume that this is a benefit or byproduct of CMM or any other methodology. It's simply a matter of having more data on which to base your assumptions. I've also discovered that this experience doesn't necessarily translate well from one type of project to another or across problem domains. To get a decent estimate you need smart people who are experience at doing your type of project in your problem domain. No way to get around it.
--john
Assembling software from reusable pieces requires three things that most software companies don't typically have:
1. Discipline. Your average programmer will have read about various programming methodologies, but skipped past the parts which would make their code an easy-to-reuse template in lieu of fast development time. As with any gamble, you should know at exactly what point you want to quit, have an A-line for version 1.0's feature set, all that jazz.
2. A big code base. Because of step 1, or maybe just a lack of previous projects, one's code base is typically limited to what you can find in a computer science textbook. Having a good database of classes and patterns that have turned out to be useful, and having easy access to this database for the information you need is the difference between a library and a code base.
3. Incremental development. Throwing together a large software project, all at once, and then testing the whole thing is very tempting, and happens more often than most people like to admit. What should be happening is a series of incremental integrations into the final product, with unit tests of each part. Otherwise your large project can become a giant, complex nightmare. Making complex software shouldn't be made quite so complicated.
While making a "software assembly line" takes slightly more work and trouble than your average car assembly line, it has incredible cost savings in the long run.
"Look at me, I invented the stove!" -- Ben Franklin
Our marketing department makes them for me! They sell something to the customer on a certain date, then I'm told what I'm programming and when it's gonna be done. You gotta love the job market today...
The issue is not physics versus manufacturing, it is scope and cost containment like is done in manufacturing. As a person who has lead multi-million dollar projects, I have grown used to the cliché that goes something like this:
If we built homes like software we would all be living in the street, penniless...
The major issues I have seen revolve around a lack of scope and cost control. In many cases it is because there is little penalty for being late or over budget. In cases where penalties exist it is often beneficial to then over estimate the effort or cost required. Then once the money is approved, using it is becomes easy.
Going back to the analogy consider the following:
Scope
If you were building a house, each piece has a specified cost, known in advance to a very large degree. In addition, altering the scope itself often incurs a penalty, because the work is not done by the owner. You plan a three bedroom, 1.5 bath home. Midway through planning you decide to make it a two bath home instead. The architect will charge the "re-scoping" fee and the builder will add the material fee. Now do the same after construction has begun. The architect gets their fee, the builder adds the material and resource costs, plus a "revision" fee for changing your mind after construction begins.
During a software project, it is common for individuals to approach the developers and ask to expand the scope. This would be analogous to approaching one of the work crew and asking them to just add the extra half a bath. The difference is the work crew would get fired, and the developer gets bonus points for adding the feature, either directly or indirectly.
If the developer chooses not to do it, or pushes them to the project manager, the client may label them uncooperative or difficult to work with. The project manager not wanting to be labeled either may coerce, cajole, or beg the developer to accomplish it, without a scope revision. Failure to do so by the developer results in real financial impact at some point, and offers little incentive to hold the line.
Cost
I call this the "Porsche syndrome".
I go into the Porsche dealership and see a new 911 Carrera Coupe. Smiling the dealer offers to sell it at a deep discount, with options and accessories $84,000 (U.S.). Whewwww baby!!! I cannot afford that. "Look," I tell him, "my wife will never approve that, you need to get it down to $28,500 tops." Would any of us expect to have the price cut down? By half or more?
Okay, how about "Look, what will it take to get it under $30,000? Seriously now, what do I have to give up" As the dealer is escorting me to the door he explains the only way I will get this car under $30k is with a mask and a gun or from a scrap metal dealer.
Yet, daily we go to developers and tell them to do the same. We ask for an estimate and then go back with "This is too much, it needs to be smaller or it won't get approved!" --Insert blank stare here--- The idea that if something cannot be cost justified it should not be done, is often lost in the "request" itself.
To nearly guarantee a project is on budget and time requires things many companies are unwilling to provide. Strict scope control procedures, with oversight by the person responsible for the money. That means each change, regardless of how trivial must be approved by someone above the project management team with business justification. It also means that requests for scope change cannot be made to developers directly, by anyone.
I was very happy with the people who built my home. When speaking to many of my friends and coworkers who built their homes, they describe it as a process akin to having their flesh removed. Everything required such effort and detail that many would not do it again.
Most of them were looking for the relationship to be like one at the office. We all want to get along and help each other out. This is not a commercial arrangement, and when we put the commercial context around it, we see it many offices lack structure.
Internal organizations can be setup like commercial ones, but it is usually unwelcome as the perception is everyone should be working for the greater good of the company and this has the appearance of bureaucracy. Even if inaccurate, everyone "wanting to get along" prevents it from being implemented.
You are absolutely right for most inexperienced developers. It was certainly the case when I was 24 and first started fixed price contracting. The reality, is that with a small amount of positive feedback most developers can start to get this right - typically within 25% within 3 months, and within 10% in a year. In my case I under bid the first project by a factor of five, and spent 3 months working at about $0.50/hr, the second project was within 50%, and the third nearly dead on. Working and getting paid by the job is experience that I think nearly every programmer needs BEFORE being allowed to work T&M or salaried.
There are secondary effects of working by the job - you very quickly learn to do only what you are getting paid for - and don't spend a lot of time on personal research projects or unnecessarily rewriting other peoples code that is working just fine but doesn't conform to your personal style. KISS is absolutely a necessary personal style - anything else and you are doom to continuous cycles of project overruns and long talks with management about why your project is another month or two away from completion.
[Anyone know how to build a Director Xtra C/C++ .dll to]
:-) Once I understood, I could estimate it pretty well.
NO, and more importantly I would not bother to estimate the time to build it until I understood how.
If you need to double your estimate and add a random number, then I hope that you're a very junior coder, or that you're unemployed.
Some companies actually do business this way. It scares the hell out of you if you are the client, but it is even scarier if you are working for the company in question.
After the October date was missed there was a meeting of all the Project Managers, Program Managers, Subject Matter Experts, and other people involved in the project. They worked round the clock for nearly 3 days to come up with a revised project plan and estimate of how long it would take to finish the system, test it, and bring it online. The number they came up with moved the date into late January. Executive management didn't like this and decided that the new date would be mid-November. Their plan for squeezing out these extra two months? Just make everyone work harder. Needless to say we've got a group that is completely burnt out and getting less done in more time. Nifty.
As long as "suits" continue to make schedules based on business needs (read "corporate politics") and not based on the complexity of the problem this is going to continue to happen.
--john
There is nothing wrong in principle with measuring what has happened in the past, and using that to predict what will happen in the future, before you discover why it works like that.
For instance, if you measure that throughout the year, the average time between sunrises is 24 hours. You can use that number even though the only explanation for it that you might have is "it seems to work"
Of course, when you apply this to software develpment time estimation, it falls down for a number of reasons. It's not constant across technologies. It's not constant across types of project. It doesn't take into account the variation in technological risks (ie if you have done something like this before, you will spend less time finding ways to do stuff). It doesn't scale linearly with the size of the project. It varies across individuals. etc. etc.
My Karma: ran over your Dogma
StrawberryFrog
[Just imagine that you had to do a while-loop according to an ISO standard]
You could make software development more predictable, but it would probably come at the cost of *enormously* increased total resources spend. Would you rather have:
(A) A project estimate at $1,000,000, which might blow the budget and csot $2,000,000.
(B) The same project estimate at $10,000,000 and come in on budget?
With software, the first part of that expression tends towards zero since most things we know how to do we can reuse code, whereas with building it remains a large accurate estimate.
I thought that's where GoF patterns could help. When I've been asked to explain design patterns to PHBs, the analogy I've always used is structural engineering - eg. for a bridge, we could have: box girder, suspension, cantilever etc. Design patterns are just like that.
Of course, in the real world, this is only a partial solution. Over 90% of software project failures are down to requirements. If we could get that right, then software development could, indeed be a "proper" engineering discipline. The only place it is, though, is where people are prepared to pay what it takes to get it right - flight control systems etc. IIRC, one of the few people to have achieved the SEI CMM level 5 are the lot who develop the space shuttle software. At the last count, their code was costing them over $1m a line. How many people would put up with what that would do for the cost of their text editor?
This sig made only from recycled ASCII
Beer does innovate at a much faster rate. Like at a bar, those lines just get better looking as the night progresses and you have to take them all home. It is debugging the staggering amount of code hurled the morning after the hangover that is the headache.
How many engineers do you know who understand terms such as
algorithmic (Kolmogorov) complexity terms. An algorithmic complexity variant of mathematical (Godel) incompleteness
Not many. They exist, but they are too few for the numbers of software projects out there. There is also the problem of clueless PHBs who refuse to hire competent engineers with actual degrees who studied "mathematical (Godel) incompleteness" in university and can make use of it for accurate planning.
I've a good friend from university days who stuck it out for about 7 years of study, both computer science and mathematical modeling. He now works for a large company heading large software projects, and his job is to make sure the abstract stuff is covered. Abstract means the kilo-lines-of-code are calculated within reason, the defect tracking statistics reflect reality, and that the end goal is well defined so accurate planning estimates are worth something. He hasn't had a project go over schedule in the last 5 years, but he has had to dump some projects after management tried to fuck them over.
His only horror stories come from clueless VPs who suddenly want to add the latest buzzword to the project. Now every contract for a project contains a whole section dedicated to any changes after the initial spec is finalised. If the client wants to change something, even something very tiny, the whole project starts over with a large payout for the cancelled version of the project. With contracts like that, software projects always stay on time.
the AC
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
Yes but. The important components of a skyscraper are steel beams. Put them up correctly, after calculating loads and stresses, and it doesn't matter what the twenty tons of stuff you have sitting on the 27th floor is. It doesn't matter if the beams come from different foundaries, either, because the specs are clear enough (dimensions, strength, where the bolt holes are).
Now try putting together a typically complex business software solution, meshing a bunch of different, reasonably good, existing programs and components with some custom code and configuration. Even where there are reasonably good standards spec'd in some areas of the project, if you're not solving new problems it shouldn't be a software engineering project at all - it should just be system administration using the available solutions. That it's real software engineering means you're running into unpredictable surprises where the components at hand don't fit without a great deal of extra labor.
A parallel can be found in work on the portions of the New York City infrastructure that are under the streets: We still have wooden water mains in some places from the mid-1800s, mixed with gas, electric, steam pipes, sewer, subways, gas lines ... most of which was not documented to current standards on either installation or subsequent changes, despite most of it being reasonably well done by the standards of its time (pretty amazing, those wooden water mains still working, right?).
So what happens when we finally go in to improve one of the services - say, lay new water mains? Other stuff is found that's in the way where you didn't expect it, or that need's fixing on examination when you didn't expect it. Meanwhile you've got the street ripped up but you have to cap it again quickly or traffic is too snarled for too long. So a single block's 4-week project can stretch out for over a year - dig up the street, fix one problem, discover more, recap while designing and provisioning the next stage, repeat - because it's all stuff that needs to be done once you get into it, that can't be properly assessed until you get into it.
Well, software in the real world isn't as old as New York, but if anything it's more complex, and the layers of crufty stuff that have to be accommodated in current projects are as considerable, and often as poorly documented by current standards (which will always advance so as to obsolete whatever we do now). Building a skyscraper, by contrast, is just a sysadmin job. Put the beams and bolts in the normal places, and it stands.
"with their freedom lost all virtue lose" - Milton
The Extreme Programming methodology has a way of dealing with this: basically, you only make predictions a few weeks in advance. Correct, given reasonably competent management, it should be possible to estimate what you can do in the next 3 weeks pretty accurately. The problem is, usually the customer wants to know the cost of a 6-12 month project up front, before they decide whether or not to even start it. Be honest about this ("You don't know exactly what you want yet, and even if you did no one can tell you what it will cost with any accuracy") and most customers will go to someone willing to _lie_ to them...
I've just finished working on an IBM RAC (Rapid Application Center) project. It was filled with elements similar to extreme programming, it used function counts, it seriously defined scope and development cycles. I've developed ideas about the method both in it's theory and it's method.
In Theory:
- All your resources are available to you when you need them for the length of time you need them.
- The client is with you all the time so that they are available to comment on the direction development is going.
- An enormous amount of time is spent in analysis to make sure the project goes in the right direction.
- Every task is estimated and ranked and put into a timed, development iteration schedule. If time runs short for a specific iteration then lower ranked features are "descoped."
The idea is that you have a fixed budget and a fixed end date and that based on these the one degree of freedom is the scope of the project. Therefore if anything changes it is the number of features.
In Practice the theory is adhered to closely but other factors enter into the project like:
- Scope Creep. This involves features that were ranked lower in the requirements and were descoped but become necessary for the end product to be useful or features that weren't caught by the requirements process but are necessary for the end product to be useful.
- Requirements Interpretation. They were nailed down, or so we thought.
- Budget. If the estimate comes in for 4 developers and a lead for 3 months but the budget only allows for 2 developers and a lead then there's an issue.
- Resources. If the client can't or won't provide the resources you need to extract the inputs you need from other systems then your schedule will be thrown for a loop.
- Client Participation. Asking 100% of you client's time in the project is an enormous request. And not always do-able.
How could it have been improved?
- The client could have provided the resources we needed. We were extracting information from some host databases and had a hard time figuring out what fields, rows and tables we needed.
- Our BA's could have done a more thorough job on the requirements. There were things that were missed or weren't defined accurately enough. We developed integer benchmark times when two decimal places were required.
- Our client could have sat with us to make sure what we were doing was what he wanted (which was what was originally agreed to). Nothing quite like having the client say that a particular feature was not quite what he wanted.
- Us developers? Well, there are always things that could have been done quicker in hindsight. I did some java-scripting that - in retrospect - could have been a hell-of-a-lot more efficient. I aim to correct that when I get a momemt.
- The function estimates were off and that caused some late nights and freaking out. It really is an art form.
Overall, the model was nice but our lack of adherence to it caused us unnecesary grief. While the client got a product he could use the process would have been more satisfactory and less painful if we hadn't strayed.
The lesson is that theory is all fine and dandy but it doesn't work if you don't follow it.
IMHO, as per
J:)
Oh well, no point in steering now.
It's also worth noting that there are many large projects that DON'T get completed on time/on budget. A prime example of this is the 10+ year 15+ BILLION DOLLAR "Big Dig" here in Boston. I won't go into details, you can get more info on the fiasco on the web.
Painless Software Schedules is a great one and you will get sucked in just following the links from this one essay.
-- Are you an EFF member yet?
OK, what I take it here is that you are talking about a method by which software project times can be predicted accurately. Suppose we had such a method. Since it is a method which takes inputs and produces outputs, it can be described as an algorithm. Since it is an algorithm, it be be represnted as a software program which predicts completion times. So far so good.
,world' assignment..."
Next get together a team of programmers. Set them to work on a program which determines proves {insert your favorite unsolved mathematical conjecture here}. It turns out you actually don't need the team at all, just run your software project estimator and if it comes out with a finite amount of time to complete the program, you know that the the conjecture is true.
In other words your software estimator can be used to solve the halting problem.
OK, this is a joke, but it points something about the question. I once had a CS professor who required that we right requirements statements for all of our assignments. She forbade us to include halting times, because "you can't predict whether a program will halt or not." To which I wanted to reply, "About that 'hello
The lesson is that there are some cases to which a rule like this applies and others to which it does not. There are some projects that can be estimated with simple tools, some that can estimated with complex tools, and some that are not practical to estimate at all. Even fairly seat of the pants kinds of estimates work pretty well on relatively simple problems, providing you break things down a bit and do an honest estimate the costs on individual deliverables and the individual functions you know you'll need to make them work. About the only methods that never work are pulling a number out of the air based on how much the project scares you, or using wishful thinking (whether the source is your boss or you). Nobody can give good estimates when you spring the question on them with no time to prepare. My boss's most (and my least favorite) questions start with "how hard would it be.." and my most favorite (and his least favorite) answers start with "It depends..."
Nonetheless, my experience with past projects of the kind that I do means I can do a pretty good job with relatively unscientific tools, provided the problem is like one I've solved before. However if you are writing software for space flight or some other kind of highly complex mission, I could estimate until I was blue in the face and it wouldn't be worth a damn. You want to hire somebody with experience in such projects and who has methods of estimation well calibrated from similar past projects.
I think the particularly difficult cases are ones inolving software maintenance -- extending software to perform things that weren't originally factored into the design, or adapting the software to run when the systems it depends upon change in some unpredictable way. These are cases where surprises can throw the best laid estimates well off.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
* A tester or test suite exhibiting the bug
* Someone recognizing that it is a bug
* Enough data being gathered to define the bug ("It hangs sometimes" or "I don't think the results are always correct" doesn't cut it).
* Enough eyeball hours to find the bug (this in itself makes the process equivalent to solving a crime. Do we ask the cops to schedule crime solving?)
* About two minutes (average) to devise and implement a fix
This has to be done for N bugs, where N is unknown. People who think you can estimate software development schedules with any accuracy are either dreaming or assuming that they just have to estimate how long it will take to get it coded, not how long it will take to get it working correctly.
-- MarkusQ
But not a "hack" job. I've got a mere 7 years experience in the software world stemming from a traditional engineering background and I've seen projects that were "on time" and projects that failed miserably. The problem is EXACTLY the same as any other engineering problem if you choose to look at it that way.
...etc. Each task is then estimated along with the dependencies on other tasks and how an overtime task affects other tasks (Pert and Gant charts are dreamy for this). In the end you come up with damn accurate estimate of how long it's going to take along with heuristics that describe what external can make a difference and how big that difference will be.
OK, I'm going to dive into the classic analogy to traditional engineering: the bridge project. Nobody ever answers the question "How long will it take to build a bridge?" right off the bat. Every aspect of the project is scrutinized and estimated separately. In other words, to build a bridge we need to do A., B., C.,
Now, back to the software arena. There is a big difference between a software developer and a software engineer. Software developers "hack" or piece together code that works, but there's been no real analysis done to support it (my definition - feel free to argue). Software developers are comparable to general construction contractors. For example, a contractor may build a deck without much analysis (i.e. how will it behave in an earthquake; what is it's failure temperature, etc.), but a major structure (like a bridge) requires an in depth analysis.
A software engineer, on the other hand, follows a much more rigorous analysis and design technique that can be used to estimate the overall time a project can take. To do this, one doesn't estimate how long it's going to take to build the entire project. Rather, one should divide the task into sub-tasks and continue to do that until one ends up with tasks that are estimatible with a defined region of uncertainty.
To do this, a certain amount of design needs to occur. Admittedly, the estimate for the design can sometimes be a shot in the dark. But, a good design can give not only a good estimate of the time required to complete a project, heuristics about the end product can be determined from the design. IMHO, the coding becomes an afterthought, a footnote to a good design.
OK, I'm done ranting. Start the flames.
The problem with estimating development time lies mostly in the management's concept of software development. I was hired to work on a project that was estimated by management to last two months. My estimate was four months and the actual time it took to complete was over a year. Why could I not meet the project deadline?
The customer claimed it was because I could not seem to fully complete a component of the project. What they really meant was I could not fully complete a component of the project before they would request a change to that component that in some cases required a complete rewrite of the component. They didn't think it was a big deal to add a button here or there in the application after all it was only a button. Never mind the fact that each of those buttons required stored procedures to be written and existing stored procedures to be altered. They would get upset that I could not make their requested changes in a day when they wanted to completely alter the way the interface to the application worked.
The bottom line is most people who don't know anything about software development don't think it is a big deal to add a feature here and there at the end of the development cycle. I try to equate software development to carpentry. Sure I can add another door in the center of those cabinets, but don't expect it not to affect the other doors and their space within.
10: PRINT "Everything old is new again."
20: GOTO 10
Roger Penrose, in Shadows of the Mind, puts forth a presentation of a proof via Godel's incompleteness and Turing's halting problem that shows that human understanding and insight cannot be reduced to algorithmic form.
The Large Limits paper uses pretty much the same proof, but doesn't add Penrose's assertion that human thought can't be computable, and therefore algorithmic limitations don't apply.
software engineering isn't like any other type of engineering process. Not entirely true. A lot of software shops pretty much write the same program over and over again, and these shops can do accurate estimates if the management is competent. This is like a civil engineering firm that specializes in steel truss bridges -- give them the length of the spans, maximum weight, and roadbed width, and they can crank out the design in a quite predictable time and cost to build.
However, much software development is a wild plunge into the unknown, like building a bridge out of new materials using a whole new concept for holding up the bridge. And civil engineers just don't do that; the first steel bridges were copies of wooden bridges, and grossly overbuilt just in case, then once they got comfortable with steel they gradually moved to more efficient uses of it. Nor do automotive engineers design a whole new car from scratch, without referring 90% of the design back to things that worked OK on the last model... Previous designs are re-used to reduce the risk and uncertainty.
Software producers are caught in a bind here. If you are re-using a previous design, why should the customer pay a lot for copy-and-paste? It's just bad management if you know enough about the job from experience to estimate it accurately and still have to do much coding... And if a large part of the job is new and genuinely does require new code, then it's a high-risk project, and likely to cost more than the customer is willing to pay.
I am in the process of completing a research report on this very issue. the background is the engineering development project modelling software SimVision, which we have undertaken to modify for use with software development projects.
the answer is yes, but it depends on a lot of things, because programmers are not like other kinds of engineers and software engineering is not like other kinds of engineering, to wit:
it seems that managers improve their estimating skills by experience, so using experiences managers is a good tip.
there's a lot more to it than this of course. unfortunaltely our report is confidential just now.
-- Rolf Lindgren, cand.psychol
That's exactly the sort of attitude that has caused the sort of spectactular failures of software projects to be accepted as the norm. Software Engineering is *not* "hacking" or "coding" or "programming", it's *engineering*, like building a bridge or a skyscraper. Yes, those projects go over time and budget too sometimes, but they are the exception rather than the rule.
I agree with you up to a point. I am an engineer. I have worked in Process Engineering, at AMEC, and now work in Design engineering. I have not done much coding, but I think that software development probably relates most closely to design. As I said, I now work in design. In design you can estimate a schedule, but that schedule is dependant on our everything going perfectly the first time, which we all know doesn't happen. This does also not include problems with parts we have to design around, which we then have to wait on, or a change in requirements of our part. (Sound familiar yet?)
This is all in the conceptual, design phase. This doesn't include the acutal production of a physical part. That all happens later, after our 3D model has been packaged correctly. Once the physical part has been made, then there are the joys of testing and testing and testing...
What I'm trying to get at, is that I've experienced several forms of Engineering (Yes there are many), and I think that Software development relates most closely to Design. In design, there is no reasonable way to schedule out how long things will take. We just make an estimate based on what's happened in the past, and change things as we go along.
"...At the end of the day"..."when everyone goes home, you're stuck with yourself." RIP Layne Staley
Whenever I had to make an estimate, I'd figure out the best value (we had COCOMO estimation software), and then I'd pad it...
Not because I thought the estimate was off, but because Marketing would *ALWAYS* shave the estimates, and that's what would be in the contract. Well, that plus Hofstaedter's Law.
In addition to man-hours, I'd often try to tack on N calendar-months for learning curve if we were doing something new (new platform, problem domain, etc...).
Wasn't always successful, but it worked pretty well... usually we worked out to the original man-hour estimate plus the learning-curve time. Of course, Marketing always low-balled it, and the learning-curve time always went away.
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
Manufacturing is more akin to installation of software, than creations of it. Creation of software is more like the engineering of the thing to be manufactured (design) and it's manufacturing process (implemenation).
When creating a new piece of software, there is a very large amount of discovery. It is impossible to estimate how long discovery will take, it would be akin to setting deadline for the cure for cancer. It is even impossible to know between two people which will get it done faster.
In addition, it is almost certain the case that somewhere in the design or implementation phases, a conflict in requirements is found or requirements are found to be incomplete. Because of this, the requirements must change. A change in requirements becomes a change in estimation.
Then there are people details, such as burnout, roll over, yank factor, procrastination, and dicipline. These things happen with different people at different times. The result is more uncertainty.
Personally, I like XP's method of handling estimation: the user stories (requirements from the customer) are broken into tasks by the developers working on the story. The tasks are estimated in the best possible scenario by the developers working on the task. No task should be longer than 3 days (if it is break it up). Then you use a previous measurement of how quickly you finished tasks to limit how many ideal days there are in a time frame.
The cental thoughts to this method is that A) only the people doing the work can estimate for themselves; B) the accuracy of estimation is inversely proportional to the length estimated. If a developer says something will take 6 weeks, they likely don't actually know how long it will take, but are picking a seemingly large number to give them wiggle (I call it "Scotty padding", refering to how Scotty would overestimate the time to repair the Enterprise so that he looked like a miracle worker); and C) using precious measurements adds a reality factor to how off your estimates are.
But there are problems with this method. The main one is that tasks can be unrealized until implementation. No matter how well you plan, something will likely get left out. Another problem is that is doesn't work over a long period of time. This method is used for planning an iteration (about 1-2 months). Longer than that the estimations will go wonky because the estimations for an individual task will vary greatly as more work is done. At the start of a project, it is easy to add a feature, but later on it may be easier, because you have supporting classes in place, or harder, because you need to update a large part of the system.
So I'd say it more like the weather. You can estimate well up to a certain point, but after that, you're just throwing darts.
-no broken link
Manager:How long will it take you two change the position of that button in the Frontend?
Me:152 years, worst case scenario.
Nobody ever lives long enough to prove me wrong!!!
Ultimately the project will be released too soon because of the lack of communication and the bugs will cause another rapid release, and the cycle continues.
Don't forget that leaving lots of bugs in rev 1.0 gives marketing something to brag about in 1.1 ("50% of it works like it ought to...").
For reference, I've been a professional software developer for 14 years.
The biggest mistake most people make in scheduling software is forgetting time for QA. They add up the pieces, thinking "OK, this will take this long, this will take..." and completely ignore that it generally takes as long to properly QA a project as it did to write it. A decent QA should be at least an in-house alpha test and multiple rounds of beta test. That's likely why the "take your best estimate and double it" approach arose.
Schedule slippage comes mostly from (1) not understanding the task when you make the schedule (2) not getting the resources you were told when you made the schedule (been there...) (3) people changing the spec along the way. For the last, stick to your guns. Once the schedule has been made and someone wants to change the spec, tell them up front that a change of that type will cost X amount of time in the schedule. They can have the change, but they have to be willing to pay the cost.
One of the greatest criteria for a good programmer, whether it is the quality of the code, or the ability to estimate a schedule, stems from humility. Part of the problem with people when estimating a schedule is that they thing they are Superman. They think that they are so good that the complex task that is in front of them is trivial. These people tend to have very buggy code as well (normally from insuffient testing). All programmers suffer from this to some extent. I've also noticed that these people tend to never use libraries, since they can write one better, but then use up all their scheduled time rewriting libraries and never actually working on the project.
Personally for me, I tend to do the best hourly breakdown I can and then double it before submission. This is normally not too far wrong (say one week on a 3 month project). The double factor allows for inaccuracies, meetings (which really do take time !), and spec changes. I may add more "fudge factor" depending on my feelings for how well the spec is sorted out and the quality of management (i.e. weak management will allow spec changes every week, good management will filter well).
ANdy
When I estimate, and the resources are there, I usually hit, if not dead-on, then very close. Basically I look how complex the system (in this case, embedded systems) is going to be, and can fairly accurately estimate how long it will take me to complete the program. The 20% sometimes is because things go easier (e.g. I find an OS solution so I don't have to write something) or worse (e.g. the hardware has problems so I can't test). But I can usually see the complexity - number of inputs, outputs, equations (reduced to atomic operations), and how they interact, and know my own "velocity" (See the Extreme Programming series for a larger discussion of something that does work).
But that doesn't help. The first problem is if I say something will be done by January 15th, they will still want it (without any help, tools, extra paid OT, etc.) on December 15. The technically correct estimate is not politically (or in marketing terms) correct.
A second problem is when you are at the bottom of the feeding chain, so if some of your test hardware goes bad, you can't get it fixed quickly, or if they disassemble your test setup every few weeks to ship engineering modules (which aren't replaced) to customers, so you start with the assumption of a reasonable development and test environment, and retrograde to LEDs on soldered leads to check things.
Sometimes this effect is in a different order - I depend on a computer or test hardware being engineered in parallel by another group, so the first test milestone in january can't be done until may when the hardware actually appears. Oh, and the extra time for an emulation system so we could develop without actual hardware was shot down because it was guaranteed to be there in january. I think one project didn't have functional hardware until two weeks before the first ship date.
Those are purely technical, but then there are political considerations. E.g. I'm using the Unix type work environment that exists everywhere free (Linux, Win32 with CygWin, etc.) and GCC but they have been using ideosyncratic windows tools - something not quite completely unlike make as a builder, some other C compiler (it had much better C++ support but C v.s. C++ embedded is another rwar). Some code (non-)documentation and editing tool that isn't integrated (they promise they might do something in a few years to integrate things). So I have to change from a porsche to a top-heavy underpowered motorhome and still try to keep up speed.
Then some higher up doesn't like version control tools. Not even something as simple as CVS. So we can't reconstruct anything other than release images making simple changes or backouts (or integrations) much more difficult.
Why is it impossible to estimate how long it takes to empty a 50 gallon trough with a 1 gallon bucket assuming you can do one bucketfull every 10 seconds? Well, they want it emptied in 3 minutes regardless of your calculation. No, you can't use the spigot so when the trough gets empty you won't be able to fill the bucket. Oh, and the bucket had a hole in it and we replaced it with a sieve. And didn't we tell you before the estimate that you can't empty close to the trough, you need to walk 100 feet up stairs and pour carefully through a 1 inch hole - we haven't budgeted for a funnel either. Oh and...
Estimates are wrong more because the assumptions are wrong (or those doing the calculation are wrong). Or what needs to be submitted needs to be wrong to be accepted - lowest bidder then add cost after it is half done v.s. accurate original bid.
And if the environment is such that you can't control things, something like extreme programming is the way to go since it is flexible enough to accommodate constant changes to function, priority, and staffing. Though it won't work when the problems are political.
XP calls for short release cycles of a few months at most. Do you just bid on the current short release cycle or on the whole several month (or year) project?
XP calls for implementing the highest priority features first, so features that slip past the release will be of lower priority. Do you get paid for a release even if lower priority features slip?
XP recognizes four variables in software development: cost, time, quality, and scope. Of these, one is usually going to have to give. XP recommends fixing cost, time, and quality and allowing the scope to change. It recognizes that requirements are never clear at first, and customers can never tell you exactly what they want. As development progresses, you adjust the scope to match the conditions as you find them. So, following XP, are you saying that you charge a fixed price but change the scope throughout the life of the project? I can see how that can work, but I don't think that's what people understood your post to mean, and it's not what most people consider 'fixed bid'.
We use and like XP as well, though we charge by the hour. I am intrigued to hear more about how you use XP with fixed bids. It seems like it might be a fixed bid for "whatever we can get done in 3x8 man months," though.
(my comments about what XP says come almost directly from Extreme Programming Explained, by Kent Beck).
including proof of concept coding (which we got management approval to throw away).
This is critical. When you are doing something completely new, and you need PoC, then the odds are your first cut (even with semi-formal or formal design methods) will be bad/buggy/suboptimal. Being able to trash it, without fear of repercussion is very important.
I've seen many cases where we were told "this is demo, don't worry", and then the demo became the shipping product.
I've had the luxury of discarding built code exactly once. This actually cleaned up every single bug because I was able to work from a (re)design that came from what I learned in PoC phase.
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
I forgot to mention the problem of noncooperation from other departments or your own management.
/. right now. I have just one project, and nobody wants to commit to anything.
Many times you will submit documents for approval or ask for input on a subject and they (managers/other departments) will grind your project to halt. I've been in the position where I had 5 projects (a couple major efforts, and the rest smaller projects) yet I wasn't able to actually code anything for about a week and a half because nobody would approve anything.
Come to think of it, thats why I have so much time to troll around
there are 2 kinds of people. those who divide people into 2 kinds, and those who don't.
The problem which NineNine referred to I believe is how to reconcile a large scale project using XP methodology with the MBA types who want one big estimate for when the whole damn project will be complete and how many dollars it will cost just from the original specifications (e.g. the job order the sales people already signed on to). If your management (and marketing, etc) are not in line with extreme programming methods, it's not going to work there.
now we need to go OSS in diesel cars
Where did you get the $4000? That's *SINGLE-SEAT*.
A full-bore installation of Rose and associated toolsets for Ada can easily run to $125,000 + 10%annually for maintenance!
Fascism starts when the efficiency of the government becomes more important than the rights of the people.
It can be done, teams do it all the time. It just takes skill, dedication and attention to not-very-fun process.
I got this from a friend, and it works perfectly.
......
:)
take how long you think it'll take, add one, and go up the next denomination of time.
example: 3 days will take 4 weeks, 4 weeks will take 5 months,
it's scary how accurate this is
--buddy
...richie - It is a good day to code.
XP is actually notable for its lack of "mumbo-jumbo CS-degree bullshit" -- it was developed and refined by development teams in the field (of industry and cosumer-related IT, not building space shuttles or cruise missiles, like so many other methodologies).
XP is one of a series of methodologies called "lightweight," or, more recently, "agile." A good starting point for reading about it would be the book Extreme Programming Explained by Kent Beck. (Sorry, no ISBN, look it up somewhere.) You will find it much smaller, simpler, and more informative than most books on the subject. You can also check out their website)
There are actually a lot of other methodologies besides XP that are taking this approach -- you can find a lot of links and a statement of shared values at the Agile Alliance website.
phil
At one point, NASA could estimate within 5 or 10% of EVERY development project they had running. Of course, they are CMM level 5 - which basically means they have their shit together. Most of everyone else, however, does not. In fact, I would say that the vast majority of projects out there could be considered to be in a state of chaos and I dont see that changing until two things happen: a) the "business" people think through what they REALLY want instead of just throwing a bunch of unformed ideas at the wall and hoping they stick. It constantly amazes me how little thought is given to systems by the very people who have to depend on them. (ie: solid requirements) and b) the developers must start acting like professional developers and not "hackers". I realize that there is a grey area between art and science but too many programmers I know take too many risks and don't think through their analysis. Often times, projects fail because something is not thoroughly analyzed or is not throughly thought out. Don't get me wrong, programmers don't need to be experts in risk management, but some acknowledgement of risk MUST be made by developers nowadays. You can't just go into your corner and code away.
But only AFTER it has been designed. I've been a developer for over 20 years and far too often what is done is that development estimates are demanded before the project has even been designed.
In traditional development projects, typically people KNOW what they need to do before it is undertaken. The contractor starts with a blueprint. It is actally possible to count the number of 2x6's that a house will need. One can make an estimate on the time required to nail one 2x6 to another and then multiply by the number in the house in order to estimate how long it will take.
I've had ignornant management ask on far too many occasions how long it will take to develope such and such a project. Best answer is how long is a string?
Management that has no feel for the problem is the problem. How long does it take to write a book?
Well - I suppose it depends on the book. Just because you can not estimate how long it will take does not mean that books will not be written or that they are not valuable.
I can write a book in a day... It will just be a simple book and quite short... but then did anyone define how many pages a book must contain in order to quailify as a book?
I can write a programming project in a day also. But it won't contain over 1/2 million lines of code. For a complex project... well, when we start to see light at the end of the tunnel, then we'll be able to make an estimate how long the tunnel was.
That is the best answer I can give.
... now if i could just convince my manager that the average time between sunrises is 24 hours she might stop allocating me for 28 hours days (12 hours/day working time)
Software development is a science in the normal sense.
Have you ever tried to do scheduling for lab or theoretical scientists? No?
It takes variable amounts of time. You just don't know when the breakthrough will come. You can make estimates, of course, especially when dealing with relatively routine kinds of things like drug testing, where there's a huge history to base your time estimates on.
But, c'mon... hard science does not lend itself to tight scheduling. Probably even less than programming does.
The poster who pointed out that when you're coding for an unfamiliar environment, estimating is rough, is exactly correct. That's why (for example) I can make pretty good estimates for how long it's gonna take me to code something on an OS like VMS that I know extremely well, while my guess for something like Windoze programming is going to probably be way off.
But that also applies to groundbreakers. When you're coding something really new (which still happens, yes), it's hard to guess. Reason? You're looking for the breakthrough, just like the chemist.
my old sig used to be funny, but then slashcode ate it and now it's not funny anymore
PM: How long to do this work ? :) I only want a rough guess.
ME: How about a spec ?
PM: You're kidding
ME: Roughly 6 weeks.
PM: Nah, too long we'll never get that past the customers, lets call it 4 weeks.
ME: Not again remember what happened last time, you chopped my estimate ?
PM: Don't worry I won't hold you too it, this time!
PM: That work finnished ?
ME: NO, two more weeks.
PM: You said 4 weeks, look here it is in the plan.
ME: I said 6, You said 4 weeks, and that you wouldn't hold me to it.
PM: The only thing I can fault you on is your estimates, they aren't very good.
ME: You £$%&* git !!!
And practically every project manager does the same thing.
Why engineer failure into the plan ?
In a well analyzed and properly planned project, you still can't tell when the compiler (or interpreter or virtual machine or web server or database server...) will suddenly manifest an unknown bug and you'll have to go scrambling for an alternate way of doing things.
From many years of coding and project estimating, I can say pretty accurately how long a project will take (assuming it's largely known methods and tools) before starting, but I've also learned to add a pretty large fudge factor. The pre-fudging estimate is usually pretty accurate assuming nothing goes wrong, but something always goes wrong, and all I can do is hope that the fudge factor is big enough to cover all the times things go wrong on the project.
If you can plan your projects to the point where "the actual coding stage is little more than data entry", you're obviously never innovating or doing anything really creative.
There also seems to be a professionalism problem in software development - programmers often deviate from the project spec to add things that they want to add, just because its fun for them, with no regard to the impact on the deadline or whether or not the feature is required and/or even useful for the project. Project deadlines for bridges would also often slip if some of the engineers kept deciding halfway through that it "would be cool" if the bridge pillars "looked like giant penguins" or something. "Real" engineers have the professionalism to realise that they need to stick to the spec. With software its not quite so clear that you absolutely have to, so (unprofessional) software developers spend too much time near the beginning of the project adding fun, cool, useless things instead of concentrating on what needs to be done. Then for the last two weeks before the deadline SOMEBODY ELSE (usually me) usually ends up picking up the slack and working 16-hour shifts to get the program ready for delivery.
I keep having fights with one of the developers here, who is a good programmer, but he has *no* concept of deadlines, time, or priorities. Even the *management* have started multiplying his development time estimates by a factor of three (its usually the other way round!). He's always like "I'd like to add this", or "it would be really cool if we had this feature", or "but we're going to need this eventually anyway" (for future future projects that don't exist yet). And its always "it'll take less than a day", or "it'll only take a day or two". And it ALWAYS takes several times longer than "a day or two". And these things add up, he just doesn't see it, a few days here and there soon add up to a month or two. I can't get it into his head that even if it "only takes a day", as he insists, that thats one day that we don't have to spare, we're already running late as it is. Its simply not possible to add features without pushing your deadline further back, and he just doesn't get that. Its unprofessional, and its frustrating.
My biggest problem as project manager just seems to be getting people to work on what they're supposed to be doing. It doesn't help either that my manager keeps finding other things for the programmers to do. Some of the developers are professional, and will just focus on doing their jobs without requiring nanny assistance, but some of them you seem to need to check up on several times a day to make sure they're not doing the things they *want* to be doing. I shouldn't have to do that.
Many developers are very cavalier in their estimates. They will say it's a piece of cake and that they can do it in 2 weeks. Then, after 2 weeks, the back peddling starts. There's alot of cocky developers who under-estimate projects. Management comes to expect this. If you say you don't know how long or tell them 3 months, they give it to the guy who says 2 weeks. So after a couple rounds of that, you say 2 weeks also.
Wansu, th' chinese sailor
I've seen it time and time again:
- Some guy is the best coder in the whole company yet still he will accept totally unrealistic deadlines and work late hours to try and finish it on time. Worse, if he does suceed, he's reward will be even more unrealistic deadlines for the next project.
- Very competent people keep working on the same job for years on end, earning pennies, while the guy next door is total crap and makes twice as much
- Some people are constantly interrupted by costumers or collegues with the sort of stupid questions that they could've easily figured out themselfs, if they weren't so lazy. (I call this one the Good Guy Sindrome)
What's wrong in all these cases?Lack of negociating skills. For example:
The real problem with software schedules is that most managers won't believe the estimates that software engineers give them in the first place. When you've been around for a while, you have a pretty good handle on how to estimate things. If you come up with an honest answer, 10-to-1 the manager doesn't want to hear it, and wants something earlier than that. I usually revert to the "When do you need something", get the info, and then tell them what features we can do within that timeframe. If they want more, it'll take longer. If they want it faster, they get less features.
At the least, this should include documenting what the change is, why the change is needed, who the change is for, what the impact on the final cost will be, what the impact on the schedule will be, and approvals for the change from the project management on both sides.
The group I used to work for used to do fixed-cost bids without this, and it worked fine until we had a combination of a customer who didn't know what they wanted and a project manager who didn't keep control on the customer requests. We kept the customer and actually had a good relationship and multiple projects with them for several years (until they were swallowed whole), but that particular project was a mess.
fencepost
just a little off
A couple of posters asked this question above: How do we reconcile XP short develop/test cycles with a fixed project plan + bid?
The answer is simple: During the planning and estimate parts we focus on defining the problem domain and a set of solutions for it. We don't focus on too many implementation details.
XP techniques are applied to solving each specific problem found in the requirements. For example, the problem may be something like "how do we decode this math-intensive file the fastest?". There usually are two or more answers to such a problem. First we define an interface, then we try two parallel, different solutions and try both. The one that meets that criteria best wins, and we move on to the next problem.
The thirst for features suffered by some people is often the result of poor design choices in the beginning of the project. If additional features are required, and the analysis was done correctly, you'll find that these new features simply extend solutions you were already working on (or solved). Thus, XP comes to the rescue again by letting you add the new feature without throwing the schedule out the window. Think about it: If a new feature forces someone to re-write a whole system then something must've been overlooked during the requirements analysis phase.
The most important part of this process is not to start coding and testing until the business requirements are clearly defined. We've been guilty in the past of coding before understanding the problem completely; we try to avoid that trap now. That is probably the single most relevant cause of software project delays.
Cheers!
Ehttp://eugeneciurana.com | http://ciurana.eu
The problem I have usually observed with scope and cost estimations is that they're usually done and signed off on before a programmer is consulted, and in cases where this isn't true, the programmer is usually some sort of generic programmer/manager type person who isn't expert at any of the specialties that will be required to complete the project.
Once the programmers get their hands on the project, they discover that they're being asked to deliver the moon on a silver platter, carved into nine pieces and wrapped in a red velvet ribbon, to be delivered next wednesday.
I can't remember how many projects I've been handed which I immediately looked at and said "this will go past deadline and over budget: who estimated this thing, anyway?"
I even remember one where the schedule had all programming scheduled concurrently with the design and planning, so once we had a spec for what we were going to do we were supposed to have it done already. Design and planning changed constantly, so I didn't get to start programming until I was supposed to be finished.
In the end, what I'm saying is that problems in delivery (past deadline or over budget) are usually because appropriate programming team members weren't consulted for an estimate in the first place, not because they estimated badly.
Yes, some of these large-scale projects went over schedule and over budget (the worst I can think of those that were completed is Clinton Nuclear Power Plant: estimate of 6 years and $800 million for a 2-unit, 1600 MW plant; actual was 12 years and $6 billion for a _one_ unit, 800 MW plant. Which was just recently sold for $20 million!).
But - most of these projects did get finished sooner or later, at costs no more than 4x their original estimate, and did fulfill their intended function. This cannot be said for many large software projects - or medium-sized ones, for that matter.
sPh
I agree with your major point that a majority are students or recent graduates. Certainly there are a significant number of people with significant experience. Unfortunately there is no way to separate the two and a poll would be clouded. It'd be good to see one though.
So long and thanks for all the fish . . . !!!
"They didn't want it right, they wanted it Wednesday."
"If you're not failing every now and again, it's a sign you're not doing anything very innovative." -- Woody Allen
Underestimation as a Marketing Tactic
AKA "Vaporware". Even if marketing knew when a product would be shippable, a particularly cinical marketing department may claim it to be earlier, thus freezing competitor's development.
Lack of Feedback (Moving Targets)
Software engineers are particularly bad at estimating because they have never done what they estimated. They are given a large project, give a large estimate, start working on it, and the project changes in the middle in a major way. This is a moving target; the estimate no longer applies. Major law of software development: You cannot change the spec or the development team on the project without impacting the real ship date. If you don't re-assess the estimated ship date, you are simply fooling yourself. Thus, they don't have any clue whether they hit the estimate or not. One way to defend against this is to break the project down into bite-sized pieces and estimate them; a small piece gives you a chance to do precisely what you estimated. Once you have that, you can have somebody track your estimates, and come back saying something like "On average, you go one third over your estimates. Add a third to your estimates from now on, and we'll be accurate".
Management Estimates
Often, engineers don't do the estimate. The management or marketing people tell you what must be done, and how long you have. Sometimes this is done explicitly; other times, management may have a number in mind and shame a software team into agreeing with it by laughing off any number that doesn't match theirs. Business people often negotiate the ship date with the geeks, like any negotiate with any other vendor. To a suit, vendor negotiations are how you determine the "margin", or how much the vendor is making (like when you buy a car, you and the dealer come to a number that determines the dealer's margin). This doesn't work in in-house software develoment because geeks hold back precious little "slack" or "margin" (they don't get paid profits, they get paid salaries); in a decent shop, geeks program at flank speed all the time and always give the project 100%.
See Ed Yourdon's Death March or any of Ward Cunningham's Extreme Programming books for more details, and ways to avoid the above traps. Yourdon suggests that the head geek has to take a hard stand in scheduling to prevent business interests from setting both the project spec and the ship date. He especially tells you never to negotiate schedule, and to help the suits understand why you never do. Whatever number you estimate doesn't affect the actual ship date, so playing with that number is simply fooling yourself.
Extreme Programming actually has a "planning game" (sort of a ritual dance) which places business interests and geeks on the same side of the table. Two big rules are "The geeks may not reject any part of the spec" and "The suits may not reject any part of the estimate". Once the suits set the spec, both teams break it down into pieces-parts, line them up in order of what gets done first and the geeks give their estimates. From there, the suits can choose the ship date (and can instantly see how much product will be ready by then), or can choose a certain amount of project completion (and can instantly see the ship date). The fun part about this method is that the suits can change their minds at any time by changing, adding, or removing pieces-parts, and can instantly see how that affects the ship date. The other fun part is that breaking up the project into pieces-parts allows developers to do a (small) project they estimated. This allows people to track estimated versus real time, and to give developers feedback that lets them make better estimates. Such a team will start off with bad estimates like everybody else, but they will be able to improve rapidly.
--The basis of all love is respect
"Suffering" from it right now, AAMOF...
1. Programmer comes up with new system in spare time while learning a language. New system, if polished, would actually make a nice application to sell to current clients. Programmer is excited, and shows "product" to highers-ups.
2. Higher-ups are excited, can see it may take a bit more work, and look into what it would take to get it to market. They tell sales and marketing to go see the programmer to have him demo it to them.
3. Programmer is excited, shows it to sales and marketing. Sales and marketing love it.
4. Months pass. Unbeknownst to the programmer, sales and marketing have sold it to a client, as part of the contract, to be a finished package by the end of the year - OR ELSE.
5. More months pass - higher ups finally tell programmer, and others, that this new system is wanted - and oh, BTW, it is wanted in Java - not in the VB it was shown it.
6. Three months are left to complete the project. Original programmer knows little Java. Other Java coders know little Swing. Architecture of app is changed from a simple app to a three-tier client-server system. Only two other coders have sufficient Java experience to code on it. The lead of the project knows no Java, and only takes notes at meetings.
7. Twenty-one days until deadline (ie, it has to be in QA in 21 days) - everyone sweating bullets knowing it can't be done. Oh, and BTW, at every meeting it seems like a new section not planned for is realized...
It was an ad-hoc system, and it is progressing as an ad-hoc system - a system that should have NEVER been shown to marketing and sales. I am not the programmer who originated it, but suffice to say it is a system that will be nice for our clients once it is completed. Fortunately, it sounds like things will be able to be smoothed over if we miss the deadline...
So remember, all you budding coders out there - if you create something in your "learning" time - don't show it to anyone BUT other coders. If marketing and sales come around, have them sign an NDA promising not to sell it or something - you don't want to release a product to market before it is done - quit "selling" vaporware!!!
Reason is the Path to God - Anon
Now, this paper makes a hell of a lot more sense to anyone who's read Hofstadler's Godel, Escher, Bach, but I suspect that many, even most, Slashdotters have read this one.
What makes the paper irrelevant is that we don't use formal systems to estimate software. We use our own head. We use hunches. We use intuition. These things are informal systems, capable of forms of reasoning that no formal system can achieve. That's what Godel proved.
The paper is saying that you can't take a spec, give it to an estimator program, and have the program write the estimate. You can give the spec to humans who write estimates for parts of it, feed that into an estimator program (like a spreadsheet), and you can get an estimate, but you simply cannot remove the human from the loop.
--The basis of all love is respect
The article presents an interesting arguement for why a completely new software project must have an arbitrarily large upper bound for time/quality estimates and can have no lower bound.
But herein lies the rub -- exactly how many software systems are "completely new?"
Damn few!!
The average software project in an average industry will be primarily a repackaging of previously solved problems.The majority of integration tasks will be sufficiently similar to previous integration tasks as to be known.
You will be left with a small number of "sub problems" which are unique and new. But now we have a situation where the caveats of the article are very important. Specifically, if we have decomposed the programming tasks to a sufficient degree, it should be the case that the estimation is tractable.
Also, it should be noted, that the author assumes that a good estimate is one obtained through formal methods that is objectively defensible. However, in project maangement, a good estimate is defined as one that is believable and acceptable to all stakeholders in the process. The method for obtaining the estimate is not important.
Moreover, good project management will include some significant up-front analysis. One common (at least common to companies with good PM'ing track records) is to run "monte-carlo" simulations of project work with large variances in schedule-v-actual work. With a run of a few thousand simulations, those processes that are most important to the time and budget performance of the project.
These "key" work packages are often non-obvious without this type of simulation work. However, with a good work breakdown structure and a good simulator, it is possible to generate a reasonably accurate picture of project performance based on what is not known.
This means that in the "real world" of business, the article's claim is irrelevant!!
We don't NEED objectively defined and defensible estimates. Instead we need estimates that the project stakeholders (which includes the people doing the work) can agree to.
We don't NEED our estimates to be generated by formal methodologies. Subjective estimates backed up by years of experience are just as good, and often better, from a planning perspective.
This whole article strikes me as another programmer trying to show how dumb the business people are. Hey folks, good business people KNOW that estimating is hard and that it isn't objective. But just because something isn't objective doesn't mean it can't be done well. It is possible to build models that compensate for unknowns if you can do enough decompossing of the problem to limit the unknowns to a well defined, small manageable few.
So, in the view of this PM, this is all just academic and has no bearing on the real world.
I believe there is a way to estimate project time, but of course it is a learning process, experience matters and refinement of estimation techniques is the process.
In reading a book called 'Your Money or Your Life' - don't remember the authors (try Amazon) they talked about tracking fincances, and figuring out where the money was going had to be done before being able to plan for savings or doing what you wanted in life rather than letting bills, banks and credit cards run your life. You have to track your money before you can figure out how much it costs to do certain things, and before you can reallocate it to get the best bang for buck.
In reading a book called 'Introduction to the Personal Software Process' but Watts Humphrey he talks about first tracking accurately pretty much every minute in every day, each week and figuring out where the time is going before you can reallocate to important tasks. The major outcome of this is of course putting your time where it matters, and being able to figure out how long certain tasks and projects take based on history.
The lesson is to: watch what you do accurately, categorize and analyze, set your priorities and goals to reflect what you want to accomplish (project, product, whatever), plan and repeat until you know fairly accurately how long specific tasks in each project take, and how long certain projects take.
This way you can work on decreasing production times, work on important tasks first rather than leaving them to 'whenever' and determine where time is being wasted.
I think it is totally possible to estimate project times this way. It can be done if you are willing to put in the due diligence. If not, hey take a guess and multiply it by two -- then make up excuses until its done (way less stressful I'm sure).
m
My impression is that while art projects are sometimes late, they're no worse than any other mature industry.
So the argument that "we're late because we're artists" doesn't seem to hold any substance.
As silly as this paper is, most responses to it are off-topic. What he is trying to show is that there is a good case for saying there is no general, algorithmic way to estimate how long it will take to do a given software project. What he isn't saying is that you can not make reasonable estimates on a given project.
Another reason that developers tend to underestimate development time is that they tend to have very healthy egos when it comes to technological issues. Again, when facing the complexity of modern code and systems, this is probably a healthy defense mechanism.
But when you couple all of this with a management that wants to believe deflated time estimates, it's no wonder that most project end up taking more time than initially thought.
That is all.
2-3 times is a pretty high multiplier, but there is a lot of merit to your comment. Here's a related hyptothetical question. You have a 5-month project. How do you spend months 1 and 2?
A) Detailed requirements and design work. Endless meetings. Lots of documents. Little or no code.
B) identify key requirements (highest value to customer) and implement them; provide working version 1 of the system, with modest documentation and simple code.
I've done some of each in various roles I have worked in. Nowadays I do a fair amount of work for my own clients, who seem to prefer option B.
The value of option B is especially obvious in an outsourcing situation. It's much easier to snow a client with an almighty thud of documentation than with working code. Therefore, with option B you know much sooner if your outsourcer is incompetant, so you can fire them if needed.
One of my former coworkers had a relatively accurate method of predicting schedules using a Magic 8 Ball. For example:
"Will this project be done in 8 weeks?"
shake "Outlook not good"
"OK, how about 12 weeks?"
shake "Maybe"
"Hmmm. Let's make it 15 to be sure..."
shake "Yes"
"Yep, this project will take 15 weeks."
And it really annoys managers when they discover that you used a Magic 8 Ball, and then it confounds them when it is right...
"You tell me the month, I'll tell you the year"
sulli
RTFJ.
Yep, there's a deeper problem, and it's very simple. Suppose your manager asks you for an estimate, and you say "six months" because that's how long you think it will take. Your manager works out that the project will not succeed if it takes six months, and asks you if you can do it in four. If you say "Yes", you have just become a statistic.
Saying yes does not mean that you can do it if you couldn't before, it just means that you have lied to management, prevented them from doing their job properly. If your project would take six months, but it will not make money if it takes six months, then you simply should not start that project. Failing to realise that simple fact is the major cause of late/failed projects, IME.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Paul Robinson <Postmaster@paul.washington.dc.us>
The lessons of history teach us - if they teach us anything - that nobody learns the lessons that history teaches us.
Software engineering is like any other kind of engineering. You *can* create a realistic schedule that you can follow. I have worked on a large number of software projects. Some hit their dates, others did not. I have identified certain preconditions that have to be met if you want to hit your date. (Not that these are profound, pretty much everyone would agree this is common sense stuff -- it's just that often times conditions aren't met which causes late projects.) First, the customer (whoever if calling out the requirements) can't be changing the requirments insanely. This one should be obvious, but I've experienced a large number of situations where management changes the basic premise of what they want regularly and are surprised when this impacts the schedule. Any external dependencies have to be met in the timeline called out in the schedule. I worked on a project where we had to deliver a server that talks to our customer's other servers using a proprietary protocol. The customer asks, "Can we have it by x date?" Our response, "Yes, if you can give us the documentation to your protocol and access to a testbed by x-y date." They delivered their end of the bargain (extremely) late causing us to be late. ("But you said you could hit the date!") Go figure! The third precondition is that the program manager should not be an idiot. This person needs to have the following characteristics. They need to be very technical. People who are former developers usually do okay. As a rule, people whose total background is as a marketing assistant or a receptionist(!) usually do not make good program managers. (The receptionist I had as a PM didn't do too bad because she understood that she didn't know anything about it and let me - the lead dev - call the shots.) This person should have been around the block a few times and should agressively track down any risky issue or "gotchas" in the process as soon as it is uncovered. This person should be tenatious in doing this. If you have those three preconditions met, then typically you can hit your date.
Avoid Missing Ball for High Score
With a little over 20 years experience of managing very large software projects for Fortune 500 companies I can identify the root cause for the spectacular successes and the colossal failures: Scope Creep.
If the business requirements have been properly defined and management discipline exercised to keep within the original scope, every estimate I've developed -- using a variety of methods over the year -- has been successful. But those instances where the specs continually change, the business requirements are "discovered" along the way and/or new requirements are added to the mix are all failures. This has been true whether I've led teams doing something "no one's done before" or the "same old thing" again.
Kudos to everyone here that has posted information on the REAL solutions in the form risk management, scope containment, good old fashioned discipline, and the like.
+-+-+-+-+-+-+ "I don't know what's wrong with you, but I'm quite sure it's hard to pronounce."
There are two major problems, the first being that the users don't really know what they want
Could this be why projects keep failing? "We" all claim to recognize that "they" don't know what they want or need. But we go ahead and estimate it and develop it anyay. If you haven't convinced yourself that "they" know what they want, or even better helped them to figure out what they really want, then you're not ready to start development.
To beat the construction analogy horse another inch beyond its life: if you're a contractor and someone asks you, "How much to build a house," you wouldn't give them an estimate. You'd talk to them until they had narrowed down what they actually wanted.
Nope, no sig
Let's assume that it is possible to have a fixed deadline, then in order to meet it you will need to get all the specifications for the program, such that they will not be changed while you're developing it. Moreover you need to have a boss that doesn't add functionality to an already started project. All these things are completely impossible, that's why our initial assumption was
incorrect. A very flexible deadline? Maybe...
To take a reductio ad absurdum:
You are given the task of duplicating the functionality of Windows NT. Furthermore, you are given the source code for Windows NT in a .tgz file and the associated development environment within which that source code can be tested. The question now degenerates into "How long does it take me to copy the tgz file?" That can be accurately predicted by measuring how long it takes to copy files on that environment in general, and the estimated schedule can be predicted to absurdly high degrees of accuracy with enough benchmarks of the system's file copying performance.
Here's another reduced complexity angle:
Translate a program written in Visual Basic and convert it to C++ (readably).
You actually can sit down and convert a sampling of the program and get a measure of how long it will take you to do the whole thing -- the more you sample, the more accurate the measure right up to the point where you have converted the whole thing.
Here's another example with a bit less reduction in complexity:
You are given a working program but no source code, and some expert users of that program. Here we are getting into what might be thought of as "function point analysis" but really, it is much easier and more accurate than that since the program exists and works as it is "supposed" to work, you can bang away on it, and the expert users can bang away on your version of it to ensure it meets their needs -- perhaps discovering that some of the features in the old program were not really used thereby simplifying the task.
Each step has been away from the "absurd" position of simply copying a program which was, in a sense, a "spec" for itself.
At the other extreme, we get to the problem of "write a program that will make me as rich as Bill Gates". Note that this specification is not very specific.... it is very far from being source code for a program you can simply copy, isn't it? Guess what that says about the accuracy of the schedule?
So a lot of this hubub about estimating software schedules is really hubub about the nature of the program specificiation process.
Seastead this.
Frank Lloyd Wright's buildings were often new ideas in theory and construction, much like the "unknown" part of estimating a software or web project today. His materials were often strange (or at least had traditional material joining with more exotic material) and the structures were oddly shaped.
e r$ 31
This is why Frank Lloyd Wright's buildings were often way behind schedule and way over-budget. He was a great architect and a wonderful designer, and I'm sure most of the engineers and builders were talented as well... but when you are dealing with brand new ideas, there is a certain amount of trial and error neccessary. Unfortunately he also didn't build that trial and error into his estimates.
Also unfortunate is that many of his buildings have leaky roofs.
The way that guy Joel does project management is the way I've been doing it for quite awhile, but he does say it so nicely:
http://www.joelonsoftware.com/stories/storyRead
If you are going to compare building a bridge or a house with building software, choose the right bridge or house to compare with. Most software projects are not a cookie-cutter suburban home that everyone knows exactly what it's gonna be like and how to make it. Most of the time it's more like a Frank Lloyd Wright or IM Pei house.... We know the physics and tools of building a house. But we usually want to make them more useful, more livable, and more beautiful. That last part takes more time.
-trout
I don't think i'd take ANY advice from carleton sheets....
s /1 649/1431.html
http://www.johntreed.com/Sheets.html
http://www.johntreed.com/Reedgururating.html
http://www.mazu.com/carleton_sheets.html
http://www.papersourceonline.com/discus/message
Physics allows projectable timelines? Think again. I'm currently employed on a fairly major project (http://www-numi.fnal.gov:8875) that, when I joined, had a completion date of 2003. Now it's 2005 and counting.
Software and physics have certain similarities (not least of which being that physics requires software development). The essential point is that you don't know how long it will take to do something that you haven't done yet. If you HAVE done it, then you don't need to do it again; all software design (or experimental physics experimentation) is essentially a research endevour, although the research results aren't neccessarly of interest in themselves.
However since they're met at the last second, often the code that is written suffers. From there often the QA department will find something wrong with the poorly written code, send it back to the development department who then has to spend some more time to fix the new errors that the sloppy code created. So although the "project manager's" deadline was met, the end client often is delayed by the additional things that were discovered.
It looks like several people (well, more than several) posted responses without reading beyond the lead-in. If you're one of them, yes, the argument here is in the general ballpark of "software estimation is hard or impossible", but it actually says something more specific than that.
The article does NOT say the following:
The article DOES say
From this, it does NOT conclude either of the points 1,2 above. Instead, it concludes:
Now some of the response posts, paraphrased:
No, it does not say this.
It also does not say this.
No, the article distinguishes subjective and objective estimates, and specifically discusses the case of an objective estimate with bounds in detail.
Ok, but slightly off topic: the article is specifically talking about those who claim objective estimates.
And where did you get an objective estimate of the complexity of a new project? Read the article...
Yes, you are. Your boss is monitoring you, get back to work.
Certainly. The 'manufacturing' camp of software estimators (Humphrey quote in the supplementary material) say or hint that software construction can be made into a repeatable, fairly boring process where projects are always on time and programmers are like factory workers. This may or may not be true (I don't think it is), but regardless: to make this view seem more science than philosophy some of these people have fallen into the trap of cloaking their estimating process with formal notation and claiming or hinting objectivity. This part is wrong.
On the contrary, [conclusions to the article and the supplementary material]: