Slashdot Mirror
MIT Studies Software Development Processes
IsoQuantic writes "A new MIT study (pdf) looked at SW development processes around the world. One striking difference that the researchers found for U.S. developers is the relatively small use of specifications before development begins. I can already hear my EP-zealot colleague chuckling in the cube next to me. (sigh)"
Specs!? Specs!? I don't need no stinking specificiations!
I used to work in a very loose development shop. The only specifications that were ever written down were protocol specs - and even those were often "documented" in the form of a header file.
I found some parts of this method usefull in that the specs were often written as as pseudo-code comments, and the actual code would be filled in later.
However, eventually the development pool grew, and we got a few folks who couldn't follow this method, and we lost several weeks of work. After that standardized specificiation paperwork was produced for every project from that point on.
Perhaps it's a lesson everyone will eventually learn. Perhaps I'm being an idiot.
Kinetic stupidity has a new brand leader: Allen Zadr.
Outsourcing these jobs should fix all these problems.
I Am My Own Worst Enemy
In my software engineering class, my teacher vehemently states that Requirements are the Enemy of Design. You need to have an idea of what you are doing for the project, but you honestly cannot know how much space it will take, how fast it will be, etc. Its sheer folly. And who isnt to say that a customer may realize that they want it differently as the process is going along. Design is dynamic, always growing and changing. And the Open Source Community best represents this, because a project never ends, but continues to develop in a myriad of directions.
je suis parce que j'aime
when working in R&D, the specs is always a step behind the cutting edge...
Our managers don't GIVE us development specs, or keep changing the specs every 5 minutes so that a formal document is worthless.
Or, in my personal experience, we stick to a formal document for 3/4 of the product then get hit with feature creep for the last 1/4 which makes the product late, buggy, over budget, etc, etc, etc
Sure, I've tried instituting "processes" and management's alwasy keen on the idea. But when push comes to shove, >poof.
The only time management ever stuck with a process was the medical company that, by law, required governmental oversight that demanded process. And you don't want to know how much we skirted process anyway. (Most of the times we built the product first, then wrote the "planning" documentation second.)
I have learned the same lesson. I hate bureaucracy as much as the next guy, but I really like having a good specification. Most of the programming I do is related to various forms of messaging, and having a detailed spec containing
/var/log/myapp.log' :-)
a) The purpose of the integration (business concepts)
b) The protocol
c) Examples. Lots of examples
makes the whole process a lot easier.
Especially the business concepts are important. They allow me to foresee where changes and extensions may occur, and I then put more work into those parts.
If you're fortunate enough to have a good project leader, use him to communicate with the other parts involved and make him also document all the changes in the protocol. That will save you a lot of time on the phone and quite a few 'tail -f
In a small team - say 3-8 people, you can get a hell of a lot more done without formal specs. and it is usually exactly what the customer wants.
Of course, this never works in real life because managers want documents to mark the progress against their gannt charts so they always interfere with the "make sure you spec it properly", and the manager on the other team will say "dont do a damn thing until you see a signed off spec"
Shit - its no wonder commercial software costs so much.
You can't expect to wield supreme executive power, just because some watery tart threw a sword at you
And of all the praise they lavish on Japan and Indian the conclusion brings it back to reality with:
Politicus
Shouldn't that be colleagues? :)
I prefer a void in conversation to a vacuous one.
Quite frankly this study is worthless. As a business owner, here is what I really want to know. Who is best at producing a product that meets my customer's needs the quickest and cheapest, has great uptime and the fewest bugs. I could give a rats ass how you did it, as long as it meets the above and it is documented.
-Subotai
"The only way to catch tiger cubs is to go into the tiger's den."
Hah! I'm halfway through my current project, as indicated by the constantly shrinking schedule, and I haven't even had to have ANY specifications or requirements to get to this point!
However, they HAVE managed to change the name of the project on me at least three times, and our last two-hour meeting was consumed by a lively debate on what to call a particular form, so it's not like these critical planning issues are being neglected.
Being creative isn't in the requirements.
When you use specification - you are making the "customer" the deisgn engineer.
I highly doubt the "customer" is going to deisgn great software.
for example.
A thousand customers could say I need a tool that lets me input the various factors of my budget and look at what the sums will be.
But how many could say - I need a program with a grid of flexible cells which can hold a value or a formula?
AIK
well now that I've read the conclusion, the author seems to agree that Bender is from India. :)
page 20 conclusion
"It is important to remember, as well, that no Indian or Japanese company has yet to make any real global mark in widely-recognized software innovation, which has long been the province of U.S. and a few European software firms."
Having worked in America, UK and India, I can certainly relate to the findings of the report. Here are some of my personal experiences:
:-)
:-)
- In America, they are in a tearing hurry to produce a prototype/model/proof of concept, which of course forms the basis of initial release ("it works, doesn't it? so why re-write it?")
- Gathering requirements is a pain in the wrong end - I've seen all sorts of instances - CFO disagreeing with CTO and re-writing everything, PO Manager introducing something new that CFO promptly over-rules. At the end of the allocated gatherings period you have a hash of what each stakeholder at the company wants. Needless to say, there is a lot of fun in ensuing months - Most American customers do not like to spend time on discussing/analysing progress or answering questions during the development (yes, I agree they shouldn't crop but, but most times they do).
- As a natural progression, the aspirations change over time. By the time it's written and demo-ed they want it to do 10 different things and pipe-up about how they had "already" mentioned they wanted this cool new feature. The signed copy of requirements specifications sure helps
- In Britain though, they pore over every single email the PM/Team Lead sends. Tremendous emphasis at every single aspect ("We are bloody payin you for this, aren't we?")
In India though, life is Sh*t. The documentation team, the process team and the internal audit teams sort of join hands to drown you in sh*tload of paperwork. It's good to have processes, but IMHO, they just get carried away and want copious detailed documents about everything and anything...Unless they develop tools to automate and regulate the processes, it is gruelling and something no-one looks forward to...err, except the audit team
http://efil.blogspot.com/
There's a difference between a spec and a design..
SPec documents are the "what" the customer wants.
Design documents are the "how"...
Slashdot is like Playboy: I read it for the articles
Kinetic stupidity has a new brand leader: Allen Zadr.
Apart from the classic Software Engineering advantages of a proper design document, it can also save you the problems which can arise if the customer and the supplier have different ideas about the eventual product.
I've seen it happen way too often; the expectations of customers can be very unrealistic simply because they have no knowledge about software engineering.
Having a complete design document with two signatures can prevent 'just add this one little feature'-type problems.
When I work by myself or with one or two developers
who I know well and get along with(and talk a lot with) We can get by with practicly no specifications even with a coding which lasts several months.
You can split up the work by writing header files first and that usually does the trick.Obviously this
cuts down on development time.
However I had on several ocasions needed to join in on a project which has been going on for sevral years, and I found it much much easier to start working quickly on the projects with more specs.
I am currently on a project run by a man who is quite anal about specs and standards and documentation, and organized testing. The result is that I spend more time dealing with standards then programing but It greatly increases the quality of the code, it makes the throwing out a week of work for incompatebilty impossible, And perhaps most importantly it makes getting aquainted with a diffrent part of the project very easy.
As for EP, I have seen it work well and I have seen it fail miserably. I have not yet gathered enogh expirience with EP to identify what makes it work.
Me.
P.s I am not a US resident, nor did I study in the US.
While I'm not disagreeing that U.S. Developers may not spend enough time in design before starting to build, I don't think that the blame always lies with the developers. My personal experience has been that, for every missing design doc or spec, there is someone in sales who shaved a week off of the estimate the developers gave them for building the solution. If MIT is studying the Software Development Processes, maybe Harvard should do a study on the Software Development Sales process, and on why US companies have developed the mentality that it is ok to give developers less time than they need to complete development, and even better to base the timeline of a project on developers working 65 hour weeks when they know damn well that they are salaried employees and thus get no overtime.
What about the twinkie? - Dr. Peter Venkman, PHD
But if I look at those that aren't suffering much right now, those doing well, I see that most of the successful U.S. deveopers and shops also specify things out. But, because we don't live and die by the regulation (those being cradle-to-grave parts of much of the world), we can also be more flexible more quickly. And we can prototype from the seat of our pants quickly.
A down side of "flexibility" though is that we often get called "not team players" if we don't instantly cow to the calls from sales and marketing every time a new feature idea pops up. One of my co-workers calls this "chasing butterflies", the lack of focus that results in never finishing anything. That's the downside that leads to bugs and slow development... and failed companies. But many more successful companies, including most of the ones I've been at, have actual product life cycles. There really are two tiers or classes of development companies that way.
The question is, why are there so many undisciplined shops? I think the answer is the easy money of the past. Suddenly we (developers) weren't being managed by engineers and developers, but rather by CFOs, shareholders and sales people. As the crunch continues, I suspect corporate Darwinism will continue and we'll scale back to more methodical practices again.
So, crack open a cold one and check out my double nested for-loop ollie nose-grind, dude!!
And who says we're zealots?! ;-)
RRR
Stuff that matters: circuitbreakers, vacuum-cleaners coffee makers, calculators generators, matching salt+pepper shakers
Maybe that's the problem. Too many times we worry about creating the "grid of flexible cells" and forget that the real user just wants to "input the various factors". There's a good usability lesson to be learned here...
Still - it is only what the customer THINKS he wants on day one with far less experience in computer techniques.
Customer says he want a report - but in reality he want to understand his business.
An OLAP server can help him understand his business, a crystal report can start him down a long road of report modifications which in the end will lead him to 1. an unmanagable pile of confusing reports or, b. an OLAP cube.
The design process is to understand the need of the customer - not have the customer specify how or what the solution will be.
If you provide the solution the customer specifies - you will be run out of business when he reads the next business journal anout how to do omething better.
You had beter know what he is going to read next if you intend on being a solution provider for very long.
AIK
1. Get an idea and start working on it immediately.
2. Stop work on it to fix a bug, release the bug fix immediately.
3. Work on the idea again, but slightly changed.
4. When about 80% done, switch to new idea or "gotta have" feature.
5. Code like hell on original idea because it was released in its incomplete form and has now killed puppies.
6. Rinse, repeat.
All the while there's very little documentation, most of it being whatever I do.
Great, isn't it? That's how it was. I've taken control and actually have implemented a release schedule and proper bugfix releasing. I've also just gotten a QA guy, things are looking up. The processes before I got here made me wake up at night. Lo and behold, with the new processes, the phones don't ring as much.
Ahh, the joys of a very small company.
Anyone else find it odd that the first page of this supposed 'new' study was marked June 2003? And the second page said it's version 3.1.
Which goes to show -- even if we had specifications for these things, we're just going to gloss over the details, and do whatever the hell we want, anyway. People don't read what's sitting in front of them, unless it's on some blog, it seems. If it were really important, they'd have made a TV show out of it.
[and those of you with moderator points get to vote if you think sarcasm is funny -- you can select 'troll' to vote no, 'funny' to vote yes. 'overrated' if you'd like to abstain, and 'insightful' if you read the first line, and are just trying to burn your moderator points]
Build it, and they will come^Hplain.
In my software engineering class, my teacher vehemently states that Requirements are the Enemy of Design.
Unfortunately, a lot of college professors are out of touch with reality. Software development is such a diverse area that you really can't generalize.
For example, I worked for a long time in embedded firmware and digital signal processing, both on mega- and micro-projects. You need to design to requirements for these. There can be creativity in how you impelment a design, but the bottom line is the spec. If you don't design to the spec, the satellite falls out of the sky.
Currently, I am working in multimedia, and we don't really use specs. We have high-level goals, but even these are fuzzy. Here, requirements are more of a hindrance, but we still have to draw the line in the sand for some things.
(S(SKK)(SKK))(S(SKK)(SKK))
I think its more like:
Requirements are what the customer wants.
Specifications are what he's actually going to get
Design is how the analyst thinks it should be done.
Bullet proof specs. are very hard to write.
:-)
I spent many years as a low level civil servant; so none of what I am telling about is my fault
To make a pile of money on a government contract, do the following:
1 - Bid on the contract EXACTLY as the tender is written (no matter how stupid the tender is). This is what the government will stick you with. If somebody else bids on the tender the way it should be and comes in cheaper that's ok. The government will stick them with the stupid specs in the tender and they will lose money! They go out of business and you now have one less competitor. **Include a generous hourly rate for 'extras'. Make sure that 'extras' are cost plus.
2 - Build the project EXACTLY as tendered and bid.
3 - (This is the important step) Discover that the project will not work as designed and tendered.
4 - Fix the project at public expense. This is the part where you apply the 'extras'.
5 - Profit! Note the lack of question marks at any stage of this process.
Specs are fine but they are no replacement for wisdom. If you need to cya then use specs. Otherwise, take them for what they're worth.
What the customer wants and what he needs are different things too.. as illustrated here (no idea where it came originally from, if you have the proper credits, please post them)
There was a little back and forth discussion, but basically the requirements were hammered out in a few days. The development team of about 5 coders took these requirements and over the next 6-8 months we used a "release early, release often" strategy. We called it "Build a little - Test a little", basically we had a GUI designer cranking out do-nothing screens, while the rest filled in the guts one function at a time.
We ended up essentially almost on time and almost on budget. The project was WILDLY successful and still in use 7 years later.
We have been funded now to produce Version 3 of this product. Most of the team has moved on, including the chief architect who drove the implementation philosophy and he's been replaced by someone who has embraced specifications like a baby marmoset clings to it's mother.
Since December, we have done NOTHING but attend meetings and write excruciatingly detailed requirements documents, and crank out UML diagrams. Not a line of code has been written (besides a bit of prototyping).
I'm doing it because I have to. I see a lot of extra labor cost (I estimate at least $250k so far) and I don't think this process has helped us. I think we could have created an entire prototype since December and thrown it all away and been ahead of the game (assuming we'd learned valuable info from our mistakes)
7. Add 20% (I'm almost there...)
8. Add 20% (Just another two weeks...)
9. Add 20% (Darn these last minute bugs...)
10. Add 20% (Testing takes time, you know...)
11. Add 20% (They want "web based" now...)
12....
Vote in November. You won't regret it.
Made in USA much earlier :-)
As a System Test Engineer, I especially liked this part of the summary
One thing most people don't realize is that 99% of the time, only a small percentage of a program actually gets executed. Much of a program is error checking or handling of unusual events and doesn't get executed if the customer uses the program the way the programmer intended. The only reason that most commercial software is usable at all is because of this. So of course, testing and integration has a negative impact on development time.
Most software companies know this and develop software to be "good enough" since being anywhere near bug-free before shipping the first version requires too much time and effort. Software companies are "gambling" that the bugs that are left aren't bad enough to sour their customers on the product. IMHO, American companies are generally much bigger risk takers than foreign companies. This leads to either a spectacular success or a catastrophic failure
I would like to see the U.S. government start to punish software companies that take large risks with investors capital. I believe a lot of companies die because of poor implementation and not necessarily because of a poor idea. I can't think of anything easier to screw up then software development and it has been considered just an ordinary risk of running a business. I worked for a software company that was run by a man that, from what I saw, didn't give a hoot whether the software worked or not. As long as the IPO produced a lot of money he was happy.
There are many ways to reduce the risk of producing a worthless software program, including certifying programmers, code reuse, and more testing. I know most people don't want to invite the government to enact more regulations but the software industry doesn't seem to be regulating itself. There is a huge crisis brewing on the horizon and nothing seems to be getting better. If the millions lost on government computer systems, like the IRS modernization, isn't enough, we have the potential for a Microsoft virus to wipeout millions of users data.
"Meaningless!, Meaningless!" says the Teacher. "Utterly meaningless!"
You completely missed the point. No one is forcing anything on to the customer. But quite simply, if you have EVER worked in the software industry designing custom systems for customers, you will know that the customer generally doesn't know what they want exactly. They have a vague idea and assume that you have the same idea they do in your head.
The requirements process is where you get your specifications from the customer about what he or she wants. Design is a completely seperate stage. The requirements are something you both agree on, but its not just something the customer sends to you. You give them feedback about the requirements. Perhaps they are contradictory, perhaps there are better ways to do things. This part is crucial in that the more time you put into this, the more information you will fish out of the customer, and you'll be more confident that you and they will know what the software is supposed to exactly do.
The golden rule in software engineering is the requirements are going to change. You just have to accept it. Why do requirements change? Well, usually its because the customer finally realizes that what they got and what they actually wanted were two different things. And thus you make the necessary changes until the next time they come back looking for you...
How many times have you downloaded a program think it has everything you want, until you use it and then realize theres something more it needs?
Think about something like Mozilla. It's be a sufficient browser for a while now. But once people got it, started using it, they thought to themselves "Now I need tabs!". And thus the evolution of software...
This series of comments by AIK is profound. Please get off your apple-pie-and-motherhood methodology soapboxes and pay attention.
Customers do NOT know what they want. Anyone who thinks that they do, hasn't spent enough time architecting software.
What AIK is saying is that you have to dig deeper than the customer requirements. You have to understand the space. You have to look at competitive products. You have to anticipate unstated needs. You should ask, "When I'm all done, and everything is working perfectly, what changes will the customer want IMMEDIATELY?"
I can't stand people who listen for five minutes and start to write "use cases" right away. That works for some dumbass web site, maybe, but certainly not for any involved product design.
Architects need to plot an intercept strategy several YEARS in the future. That's how you build successful software. When the customer puts his Phase N requirements out for bid, and all your competitors run for the hills, your design has anticipated his needs. You've built metatables instead of tables. You've used OLAP where you could have sleazed by with an RDBMS. And so on.
Great thread.
In my experience, RUP seems far better. The customer rarely knows exactly what they want. At best they'll have an idea of what they want it to do and can visualize how they think it might work. So you start with a loose set of requirements, do some quick high-level design, and code it. Then show the customer what you've got. They can identify what is different from what they want and see some of the issues they may not have thought about. This defines some lower level requirements and designs. Repeat this process on a regular basis and you will get good, well-written, and efficiently created software.
I've had nightmares with the Waterfall process. I've had to spend 6 months writing requirements and design docs, formally releasing them, then have 2 weeks left to write the code so the software wasn't nearly as good as it should have (or could have) been. In the end, the requirements and design had some inconsistencies that could not be foreseen until you try to code them.
On another project (that I was not involved in) all of the money went into the requirements and design process and the end software was crap, wasn't user friendly, and didn't get used. But it did meet the requirements.
The problem with writing requirements completely before coding is that you have to basically do the design and coding in your head as you write the requirements, otherwise you can end up with things that are inconsistent. Written language doesn't have to be self-consistent, software does.
On the otherhand, writing software without following any guidance from the customer is definitely worse.
As an employee of a US division in a German corporation, I think I can shed some light on why US developers skimp on specifications:
We're too busy coding.
The truth is that commercial development schedules are unrealistic. If it would take two years to do a project correctly, only eighteen months will be allocated. This leaves the developer in a quandary over which part of the process to skimp on. US developers choose to skimp on specifications, while German developers choose to skimp on implementation.
That's my experience anyway. I've seen specs from Germany that are so padded I think the author must have stock in a paper mill. And I've seen the incomplete software that arose from it. In one instance a product was shipped that was completely unusable, whose only sales the first year were to the sales department as demos, but which won a corporate award for adherence to the process.
It's simply a different way of working. To the US developer, if you can't do it right, at least make it work. To the German developer, if you can't do it right, at least go through each step of the waterfall model thoroughly and methodically until your time is up.
Just about every corporation views the process as more important than the product. But their individualistic and rebellious nature means that US developers will work on the product anyway. But German developers will just do what they're paid to do.
Don't blame me, I didn't vote for either of them!