Slashdot Mirror
What Makes Software Development So Hard?
lizzyben writes to mention that CIO Insight is running a short piece that takes a look at why the rocky culture of software development continues to exist despite all of the missed deadlines, blown budgets, and broken promises. From the article: "I was not really looking or thinking about big software projects. I was just coming out of my experiences at Salon, where we built a content management system in 2000, which was painful. I was one of the people in charge of it, and when the dust cleared, I thought, I don't really know that much about software development. Other people must have figured it out better than I have; I must go and learn. So I started reading, and talking to people, and realized it's a big subject and an unsolved problem. And the bigger the project, the harder the problem."
In related news, humans still can't seem to bridges with any reliable schedule or budget. Despite the fact that bridges have been built probably since the dawn of man, and we've been building suspension bridges for at least 500 years.
my blog
First the wag: The idea that the interveiwee states early on, that software development is not introspective, is horse-hockey. We think about it all the time. We invent new, clever ways to diagram software, capture requirements, interview users, validate functionality and come with all sorts of certifications. More than, say accounting, we are process focused people. Maybe our processes suck, but we spend a lot of time, energy, certification exams, etc, on those processes.
Tip of my hat: He does mention starting small and iterating. I think that's the best way to build software.
Leave the gun, take the cannoli -- Clemenza, The Godfather
Well no one really teaches software design do they.
You can go to school to learn to be a bridge builder and come out of it knowing all the exact specifications to build a bridge and probably design a fairly good bridge, or with a bit of creativity and some extra architectural skills a really cool bridge. Software design isn't really taught in this manor, sure your taught how all the bridge building tools work, and even a lot of the engineering specifications. But I have yet to see the software design school that covered more than a class or two into truly how to design software. Then again, we've been building bridges for thousands of years, and designing software systems for a few decades. It does take time for these things to really get figured out.
The first reason is the complexity of the systems. There are essentially extremely many interacting parts that cause all sorts of emergent phenomena. This is a field that we don't really know how to handle very well - good quantitative models of complex systems simply don't exist. We need more study of systems theory and chaos theory to build some form of predictive models on which in turn can base planning.
The second problem is a sort of 'freedom of thought' culture that sees coding as the vaguely mathematical expression of ideas. As there are a huge number of degrees of freedom most attempts to regulate it in practice have been abandoned. So instead of just the problem of describing a mathematical problem, you throw individual human preferences, thinking and biases into the equation.
Of course it can't continue that way forever and we need to move to a meta level of coding. We do have well-constructed systems, such as the software on-board an airplane - but they are laughably primitive (because you have to account for all possible states the software can get in to). And we have advanced software, but it is laughably unreliable.
Still, there is reason to be hopeful. We're not inventing the wheel every time any more (just the horse carriage and so on). Modern programming systems have extensive class libraries that are on average a more stable foundation than making a custom implementation from scratch every time.
Ultimately however we need to know how to handle complex systems and how to enforce convergence/stability to systems where the huge number of possible combination of states can be analyzed. And sad as it is, if we want it to be reliable, the liberal individual approach to coding has to be abandoned in favour of a much more strict industrial way of thinking. Right now we have handcraft and we need industrial precision, standardization and quality.
Software is still a Science and not an Engineering practice.
As long as the design can also be the implementor and estimates based on actual analysis are optional, Software will NEVER be an engineering study.
These aren't changing quickly for what I believe are a few reasons:
IEEE has not created a Professional Engineer - Software and noone really wants them to.
Companies don't like to be told they have to hire something that sounds expensive to build something they cannot see.
Companies will NEVER open their software to outside inspection the way construction companies must open their buildings because of the concept (flawed, I think) of Intellectual Property.
If a company had to have their software inspected by a Software P.E. before using it in production or selling it to end users - If Software P.E.'s had to adhere to standards which included concrete estimates and testing - If companies were not allowed to use anyone they could find that has seen a computer to write their software... commercial software development would be much further ahead.
Do I believe any of this should happen? No. Why? Because I want it to continue to fail. I do not believe software development should be put on a pedestal or only performed by "experts". I believe shoot from the hip software projects allow open source software projects to exist and to succeed in the market.
Open source works without accurate estimates because the contributors can flock to good projects and don't have to adhere to a labour budget. Company employees can't get wind of the cool software project and leave the crappy one's - corporate structure and budget's won't let them.
I don't believe companies with more than 120-150 people are stable - once they breach this range empire building occurs and massive uncontrollable monsters result. If a company truly needs more than 150 people it should split into two and partner on the project at hand. I believe this is a human condition - humans work best in tribes where they can personally know all of the members.
All of this might be completely and utterly wrong. But it's my hypothesis.
You are checking your backups, aren't you?
>If you were told to write a program to add two numbers together, and that was all, you'd have a pretty easy time rejecting "hacky" options, too - because you know all of the requirements.
...
Can the sum of the numbers exceed MAXINT? MAXINT on what platform? Should overflows set a flag, throw an exception, or be reported in-band?
Can the numbers be floating point? If one is much smaller than the other, how large a fraction of the smaller number's precision are you willing to lose?
Can the numbers be multiple-precision? Is the multiple-precision library you're using compatible with the rest of your system? Does it fit into your memory requirements?
Do the numbers come from some real-world source that need to be checked for sanity, for example the output of a sensor?
In what form does the output need to be? Encoded in ASN.1? BCD? IBM "unpacked decimal"? String? If output is variable length, is there a maximum size that must be enforced to prevent buffer overflows? If the maximum size is exceeded, should the result be truncated, or should the function throw an exception, or
The sickening thing being that I've probably left out some important issues. Like thread safety.
Look up the bug history of "IEFBR14", originally a single return instruction that required four or five revisions to meet shifting requirements.