Slashdot Mirror
MIT Studies Software Development Processes
IsoQuantic writes "A new MIT study (pdf) looked at SW development processes around the world. One striking difference that the researchers found for U.S. developers is the relatively small use of specifications before development begins. I can already hear my EP-zealot colleague chuckling in the cube next to me. (sigh)"
Our managers don't GIVE us development specs, or keep changing the specs every 5 minutes so that a formal document is worthless.
Or, in my personal experience, we stick to a formal document for 3/4 of the product then get hit with feature creep for the last 1/4 which makes the product late, buggy, over budget, etc, etc, etc
Sure, I've tried instituting "processes" and management's alwasy keen on the idea. But when push comes to shove, >poof.
The only time management ever stuck with a process was the medical company that, by law, required governmental oversight that demanded process. And you don't want to know how much we skirted process anyway. (Most of the times we built the product first, then wrote the "planning" documentation second.)
In a small team - say 3-8 people, you can get a hell of a lot more done without formal specs. and it is usually exactly what the customer wants.
Of course, this never works in real life because managers want documents to mark the progress against their gannt charts so they always interfere with the "make sure you spec it properly", and the manager on the other team will say "dont do a damn thing until you see a signed off spec"
Shit - its no wonder commercial software costs so much.
You can't expect to wield supreme executive power, just because some watery tart threw a sword at you
Apart from the classic Software Engineering advantages of a proper design document, it can also save you the problems which can arise if the customer and the supplier have different ideas about the eventual product.
I've seen it happen way too often; the expectations of customers can be very unrealistic simply because they have no knowledge about software engineering.
Having a complete design document with two signatures can prevent 'just add this one little feature'-type problems.
When I work by myself or with one or two developers
who I know well and get along with(and talk a lot with) We can get by with practicly no specifications even with a coding which lasts several months.
You can split up the work by writing header files first and that usually does the trick.Obviously this
cuts down on development time.
However I had on several ocasions needed to join in on a project which has been going on for sevral years, and I found it much much easier to start working quickly on the projects with more specs.
I am currently on a project run by a man who is quite anal about specs and standards and documentation, and organized testing. The result is that I spend more time dealing with standards then programing but It greatly increases the quality of the code, it makes the throwing out a week of work for incompatebilty impossible, And perhaps most importantly it makes getting aquainted with a diffrent part of the project very easy.
As for EP, I have seen it work well and I have seen it fail miserably. I have not yet gathered enogh expirience with EP to identify what makes it work.
Me.
P.s I am not a US resident, nor did I study in the US.
While I'm not disagreeing that U.S. Developers may not spend enough time in design before starting to build, I don't think that the blame always lies with the developers. My personal experience has been that, for every missing design doc or spec, there is someone in sales who shaved a week off of the estimate the developers gave them for building the solution. If MIT is studying the Software Development Processes, maybe Harvard should do a study on the Software Development Sales process, and on why US companies have developed the mentality that it is ok to give developers less time than they need to complete development, and even better to base the timeline of a project on developers working 65 hour weeks when they know damn well that they are salaried employees and thus get no overtime.
What about the twinkie? - Dr. Peter Venkman, PHD
But if I look at those that aren't suffering much right now, those doing well, I see that most of the successful U.S. deveopers and shops also specify things out. But, because we don't live and die by the regulation (those being cradle-to-grave parts of much of the world), we can also be more flexible more quickly. And we can prototype from the seat of our pants quickly.
A down side of "flexibility" though is that we often get called "not team players" if we don't instantly cow to the calls from sales and marketing every time a new feature idea pops up. One of my co-workers calls this "chasing butterflies", the lack of focus that results in never finishing anything. That's the downside that leads to bugs and slow development... and failed companies. But many more successful companies, including most of the ones I've been at, have actual product life cycles. There really are two tiers or classes of development companies that way.
The question is, why are there so many undisciplined shops? I think the answer is the easy money of the past. Suddenly we (developers) weren't being managed by engineers and developers, but rather by CFOs, shareholders and sales people. As the crunch continues, I suspect corporate Darwinism will continue and we'll scale back to more methodical practices again.
With Aegis the baseline code "always" works; it has to pass all the build tests to become baseline.
You can't add a new feature without first defining a test for it to pass, you can't fix a bug without defining a test that the old baseline failed and the new baseline passes.
So marketing can walk up and say "release now" or "add these features" and you can do either. But you can't "release these features now" because the system won't let you.
When marketing say "release now" they can only have the bits that work. And when they say "add these features" they can only get those features when they work.
Sam
blog.sam.liddicott.com
1. Get an idea and start working on it immediately.
2. Stop work on it to fix a bug, release the bug fix immediately.
3. Work on the idea again, but slightly changed.
4. When about 80% done, switch to new idea or "gotta have" feature.
5. Code like hell on original idea because it was released in its incomplete form and has now killed puppies.
6. Rinse, repeat.
All the while there's very little documentation, most of it being whatever I do.
Great, isn't it? That's how it was. I've taken control and actually have implemented a release schedule and proper bugfix releasing. I've also just gotten a QA guy, things are looking up. The processes before I got here made me wake up at night. Lo and behold, with the new processes, the phones don't ring as much.
Ahh, the joys of a very small company.
I think its more like:
Requirements are what the customer wants.
Specifications are what he's actually going to get
Design is how the analyst thinks it should be done.
Insightful?
The statement is utter crap. A fair comparison would be between an open source program that has something ugly bolted on and a closed source program that doesn't have that feature at all because it's not released and you have no option to get it.
Most open source software does what it's creators want it to do. Should they take the time to design an extensible system for everyone in the world to use for any possible use? Probably not, they just wanted it to do something. The fact that they make it available is a benefit and needs to be compared to the in house proprietary tools that aren't made available and are probably *at least* as ugly.
t
Maybe I'm misunderstanding your intent here but I'd say the release-early-release-often is a huge strength of open source development.
By showing all the warts, all the problems and solutions as they are worked on, those of us who are chasing the latest nightly CVS release get to help out by bug triaging, raising problem reports and commenting on the very latest change pretty much as it happens. This stops stupid decisions, breakages and irritations early before they are ingrained.
However, for the average user, that is too much. But just because development is pushing on at a rapid rate doesn't mean that the user has to play catch up. This is where the packagers of applications play their part - Mandrake/Debian/RedHat/etc - by providing a tested stable base. If you want the ultimate in stability, you would probably look at Debian Sarge or RHEL. You can pick a more intermediate solution which is closer to the latest releases with Mandrake numbered releases, closer still to the edge with Mandrake community /Fedore core 1 or almost at the bleeding edge with Cooker or Redhat Fedora 2 test 3.
So really, stability of releases is available now. The only scary thing here is that you have a Choice.
Cheers,
Toby Haynes
Anything I post is strictly my own thoughts and doesn't necessarily have anything to do with the opinions of IBM.
There was a little back and forth discussion, but basically the requirements were hammered out in a few days. The development team of about 5 coders took these requirements and over the next 6-8 months we used a "release early, release often" strategy. We called it "Build a little - Test a little", basically we had a GUI designer cranking out do-nothing screens, while the rest filled in the guts one function at a time.
We ended up essentially almost on time and almost on budget. The project was WILDLY successful and still in use 7 years later.
We have been funded now to produce Version 3 of this product. Most of the team has moved on, including the chief architect who drove the implementation philosophy and he's been replaced by someone who has embraced specifications like a baby marmoset clings to it's mother.
Since December, we have done NOTHING but attend meetings and write excruciatingly detailed requirements documents, and crank out UML diagrams. Not a line of code has been written (besides a bit of prototyping).
I'm doing it because I have to. I see a lot of extra labor cost (I estimate at least $250k so far) and I don't think this process has helped us. I think we could have created an entire prototype since December and thrown it all away and been ahead of the game (assuming we'd learned valuable info from our mistakes)
Requirements that are too rigid are bad. But so too are requirements that are too loose.
I like requirements that are rigid on the edge, that is where they meet the world.
But then give me free rein to change ideas for the internal structure.
Of course, it always helps if you know your customers well enought to anticipate the inevitable change request that will come down the pike. Then, you'll see if your internals can be readjusted elegantly quickly to accomodate the new exposed services they want.
It's important to listen to customers, not just to hear what they're saying explicitly, but to get an idea of where they're going and what they might want in the future based on what they believe is important and valuable. Hint: customers/users may not just tell you explicitly what they think is important and valuable.
"Provided by the management for your protection."
Maybe someone is actually using Evolutionary Programming (not)?
-- Nothing unusual happened today
The best trick I've ever seen for setting project goals is to sit developers down with business users or the end users of the product and have them observe work in action. This understanding can go a long way when a developer is back in the 'cube' for long hours of coding. Domain knowledge is critical to a developer. Otherwise you end up with some trully epic program that bears little to no pracitcal value.
There's an even better trick: put your development team in a shared workspace. Then make a domain expert the product manager and put them in the same room, so that if developers have questions they just have to look up and raise their voice a bit.
It's fantastic. The closest we get to specs is whiteboard sketches and the occasional UI mockup. And that's all we need. The developers love it, because there's no guesswork involved. And the product manager loves it because he gets exactly what he wants without having to write phone-book-sized spec documents.
On the topic of knowing the domain, I also strongly recommend the book Domain-Driven Design. It's a brilliant book on how to base your OO model on the thing that is least likely to change: the business domain.
My customers don't know what they want. They don't really understand how their own business works and they feel lost because as their business is growing they lose track of the knowledge that was very easy to keep up with when their company only had 20 people.
It depends on your software and your client base, I guess. When I program for hire it tends to be CRM and HR packages for small businesses that have outgrown Quicken, so I guess it's understandable that I see a lot of confused people who honestly just don't understand their own company anymore.
But at least in my case, my customers really don't know what they want because they haven't yet figured out what they know and what they need to know about their own business; to some extent I can help them with that but generally I just point them to some management consulting firms.
All's true that is mistrusted
As a System Test Engineer, I especially liked this part of the summary
One thing most people don't realize is that 99% of the time, only a small percentage of a program actually gets executed. Much of a program is error checking or handling of unusual events and doesn't get executed if the customer uses the program the way the programmer intended. The only reason that most commercial software is usable at all is because of this. So of course, testing and integration has a negative impact on development time.
Most software companies know this and develop software to be "good enough" since being anywhere near bug-free before shipping the first version requires too much time and effort. Software companies are "gambling" that the bugs that are left aren't bad enough to sour their customers on the product. IMHO, American companies are generally much bigger risk takers than foreign companies. This leads to either a spectacular success or a catastrophic failure
I would like to see the U.S. government start to punish software companies that take large risks with investors capital. I believe a lot of companies die because of poor implementation and not necessarily because of a poor idea. I can't think of anything easier to screw up then software development and it has been considered just an ordinary risk of running a business. I worked for a software company that was run by a man that, from what I saw, didn't give a hoot whether the software worked or not. As long as the IPO produced a lot of money he was happy.
There are many ways to reduce the risk of producing a worthless software program, including certifying programmers, code reuse, and more testing. I know most people don't want to invite the government to enact more regulations but the software industry doesn't seem to be regulating itself. There is a huge crisis brewing on the horizon and nothing seems to be getting better. If the millions lost on government computer systems, like the IRS modernization, isn't enough, we have the potential for a Microsoft virus to wipeout millions of users data.
"Meaningless!, Meaningless!" says the Teacher. "Utterly meaningless!"
When specfications are turned into an overly formal process, the spec process becomes a time-sucking industry in itself. I have seen cases where some "Spec Police" count how many slots are filled in on the forms, while most of the time it is stuff that is obvious and redundant for the module. Plus, a one-size-fits-all spec form does not work well. Different kinds of projects need different approaches. For example, a report-centric project does not need a "database change results" section because reports usually don't change the database contents. Ideally it is a give-and-take process, not Slot Policing that works best.
Table-ized A.I.
In my experience, one of the most significant and under-recognized challenges in development is the selection of the right tools, operating environment and languages in which to develop/deploy an application.
.ASP, and Perl that really should have been using something different based on the demands of the process.
These days, your typical American developer has a narrow stable of technology that he uses. He often doesn't stop to examine whether or not the application being designed is best suited for the environment in which he plans to build it.
I'm probably going to get flamed for this, but I believe we now have "vanity languages" and platforms that are driven more by marketing than fitness for a particular purpose. In the last several years I've seen lots of programs written in stuff like Java, Cold Fusion,
I would suspect a significant share of development disasters are due to the people involved choosing the wrong tools and then making things ten times harder for themselves later on.
If you are wasting months on an OLAP cube that is your problem.
I implemented OLAP cube with an excell frontend embbedded in a webpage such that all users in a multistate company could view the OLAP live from the website - and it took LESS time than any crystal report - and it was precisly what the company had been hinting at for years - only they didn't understand what it was they wanted.
Yes - I'm arrogant if it means anything to you.
AIK
In my software engineering class, my teacher vehemently states that Requirements are the Enemy of Design.
Good grief. Has your teacher ever designed anything in his (or her) life? I guess it's true what they say about "those who can't do".
You need to have an idea of what you are doing for the project,
In other words, you need to have an idea of what is required.
you honestly cannot know how much space it will take, how fast it will be, etc.
Those aren't requirements (except, perhaps, for an embedded or real-time system), those are specifications. Requirements != specifications, although there may (depending on the app) be a requirement that it meet certain specifications.
Design is impossible without requirements. Oh, you might throw some code together (there's a great cartoon I've had for years, and have used in a few presentations, reviews, etc. Room full of programmers at their computers, one guy is leaving the room and turns back to say "You guys start coding, I'll go up and see what they need."), but in the absence of an idea of what the customer wants (ie, his requirement), you might as well just be writing yet another "Hello, world!" program. It'll be about as useful.
Now, some of the extreme/agile/this-week's-buzzword programming folks might like to think they're doing something different. The only difference between that and the older formal "waterfall" method is that instead of assembling the requirements -- stories -- into one document before design and coding starts, they do it incrementally: get a requirement, write some code, test it out; get another requirement, write or modify the code, test it out. Lather, rinse, repeat.
Now, that latter method certainly has its advantages when "a customer may realize that they want it differently as the process is going along", because it does allow the design to be dynamic. But the design always tracks the customer's (possibly changing) requirements. (Or should, anyway.)
Sometimes, though, the customer knows what he wants and it isn't going to change -- a guidance program for a VroomStar I rocket is going to have well nailed-down requirements, and specifications, since the guidance computer, power available, characteristics of the rocket, etc, are going to be well established ahead of time (with long lead times) and very difficult to change. Sure, change will still happen, but it's not like the customer is going to launch a few VroomStars with a half-complete version of the guidance package and then write up the next stories for the development team.
-- Alastair
Based on my experiences of the past 20 years working in Unix OS, Networking and Graphics development organizations. Good practices and team culture can overcome weaknesses in any given development process model.
IMO, no single model works well for all projects. The development model that best fits the project requirements and the team culture will usually produce the best results.
What I've found is that when Key Practices are performed on a daily basis, whatever model is in place can be sucessfully managed.
Key Practices:
Change Control: this includes; source, binary, requirements, decisions, action items, specifications, hardware, firmware, processes, practices, etc...
Communication: the ability to notify and acknowledge change
Assessment: the ability to review change and make corrective adjustments in the schedule and requirements
Planning: the best product bosses work their plans every nite, and publish every morning
On Demand Crank Turn: the ability to turn the crank at least once a day. This includes: build, package, assemble, test, publish, assess, plan, prioritize
GigantanKramePithicus
In my experience, RUP seems far better. The customer rarely knows exactly what they want. At best they'll have an idea of what they want it to do and can visualize how they think it might work. So you start with a loose set of requirements, do some quick high-level design, and code it. Then show the customer what you've got. They can identify what is different from what they want and see some of the issues they may not have thought about. This defines some lower level requirements and designs. Repeat this process on a regular basis and you will get good, well-written, and efficiently created software.
I've had nightmares with the Waterfall process. I've had to spend 6 months writing requirements and design docs, formally releasing them, then have 2 weeks left to write the code so the software wasn't nearly as good as it should have (or could have) been. In the end, the requirements and design had some inconsistencies that could not be foreseen until you try to code them.
On another project (that I was not involved in) all of the money went into the requirements and design process and the end software was crap, wasn't user friendly, and didn't get used. But it did meet the requirements.
The problem with writing requirements completely before coding is that you have to basically do the design and coding in your head as you write the requirements, otherwise you can end up with things that are inconsistent. Written language doesn't have to be self-consistent, software does.
On the otherhand, writing software without following any guidance from the customer is definitely worse.
As an employee of a US division in a German corporation, I think I can shed some light on why US developers skimp on specifications:
We're too busy coding.
The truth is that commercial development schedules are unrealistic. If it would take two years to do a project correctly, only eighteen months will be allocated. This leaves the developer in a quandary over which part of the process to skimp on. US developers choose to skimp on specifications, while German developers choose to skimp on implementation.
That's my experience anyway. I've seen specs from Germany that are so padded I think the author must have stock in a paper mill. And I've seen the incomplete software that arose from it. In one instance a product was shipped that was completely unusable, whose only sales the first year were to the sales department as demos, but which won a corporate award for adherence to the process.
It's simply a different way of working. To the US developer, if you can't do it right, at least make it work. To the German developer, if you can't do it right, at least go through each step of the waterfall model thoroughly and methodically until your time is up.
Just about every corporation views the process as more important than the product. But their individualistic and rebellious nature means that US developers will work on the product anyway. But German developers will just do what they're paid to do.
Don't blame me, I didn't vote for either of them!
Well put. I noticed in practice both you and the client experience a development process. You develop software, while the customer simultaneously develops an understanding of their own business or problem. Looking at it this way, it is clear immediately why a waterfall development approach won't work.
But still at the development shop where I work, the gospel is to create a detailed requirements document (including quote) beforehand. The customer must sign off, and development starts. Every project I have seen performed so far has ended in sour feelings on both sides, due to the customer wanting to change requirements midstream. And everytime I'll hear other developers complain about customers who do not know what they want. It baffles me that no one (especially management) tries to find a development form that can handle changing requirements in a smooth way.
Extreme Programming promises to take care of this problem, but I'm not quite sure how you'd make an initial quote. The feeling at work is that customers will only accept a quote which promises to deliver a certain set of specifications for a certain price. Apparently, customers themselves do not realise beforehand that their requirements will change over time. Therefor, they will prefer a quote which promises them to deliver what they think they want, rather than a quote which promises them a best effort in a rather open-ended time frame.
Anyone have any experience with quoting XP projects?