Too Much Focus on the Beginning of Software Lifecycle?

rfreedman asks: "Most of the buzz on the web about software development tools, languages, and practices seems to concentrate on getting software developed as quickly as possible. Take, for example, the current huge hype about Ruby on Rails, and how it allows the creation of a CRUD web-database application x-times more quickly than every other environment. It seems to me that this concentration on initial construction of software ignores the issue of total cost of ownership. Most people who develop software also have to maintain it, and have to support changes to it over long periods of time. As has been discussed many times over the years, maintenance is the most expensive part of the software development life-cycle. I think that the software development community would be better served by discussions of how to build more robust, flexible, and maintainable software (thereby driving down TCO), than by the endless discussions that we currently see about how to build it quickly. What do you think?"

Comments

Good point

[Duhavid]

Now, how to convince the PHB's and the bean counters?

Justified

[Umbral Blot]

The emphasis on fast devlopment is justified, at least from a business perspective, because first to market gives a huge advantage in software, not to mention the network effect. Sure the ability to maintain and upgrade software is somewhat important, but it doesn't matter so much if it takes a long time if you are already dominating the market. Similiarily start-ups don't care about these issues since they plan on being bought out before they matter. Yes these attitudes create serious problems and lead to poorly made software, but what can you do about it? (besides using open source)

Re:Justified

[humblecoder]

Actually, being first to market really isn't that big of a deal. There are tons of products which were first to market which have slided away into oblivion because they were rushed out the door. Because they were rushed, they ended up sucking the big one. Then, another company comes out with a better, more polished version of the same product and everyone forgets about the first, sucky product.

The dotcom bust is littered with companies whose business model was, "be first or else", and nobody seems to remember them.

No.

[hsmith]

You have large maintnence costs because no one properly plans up front for the long term. People want to see something and they want to see something fast. No one sits down to write the proper documents, no one sits down to plan ahead, they think for the short term only. Which leads to long term problems down the road. At least, that is what I see as major issues on things i have worked on.

people don't want to make the initial investment to plan ahead, so they end up spending much more in development costs because no one decided where the product should go.

Re:No.

[RingDev]

Here! Here!

The problem isn't high level languages or agile development. The problem is a lack of foresight, planning, and documentation.

We have two major systems at the company I work for. One, that the related staff spent 2 years documenting their business practices and exactly what they wanted the application to do. They took that huge document to a couple of consulting companies and said "we want this." 2 years later the consulting company that won the bid turned over a nearly perfect application.

The other app is an extension of a 3rd party application. This app started development with little to no documentation, no project manager, and no one who had actually worked with the 3rd party application. Flying blind into a short deadline has left us playing redevelopment games for almost 2 years! We finally managed to get the critical personnel together to get the system and processes documented, and after 3 months of business process documentation we should be set to re-write the business layer of the application in 6 months.

Had the user group worked with the 3rd party app prior to launch, and documented their processes, the entire application could have been developed in under 9 months. But with out that work on the front side we will have over two and a half years wrapped up in development from 3 people.

One more project like this, and I'm moving to management. As much as I love coding, I hate inept managers even more.

-Rick

Re:No.

[Kadin2048]

That's a great anecdote, and I don't think that you can hit that point hard enough.

If somebody came to you and said "hey, I've got this great new way to build a bridge! Instead of making up plans, we'll just start building it! We'll build it out of popsicle sticks first, and then we'll go in and add some steel beams, and toss some pavement on top of that," you'd say they were insane. Nobody does stuff like that in the real world -- yet that's exactly what a lot of poorly-managed 'agile' software projects are doing. They're getting short-term prototyping gains but at the cost of maintainability and probably stability as well.

Rather than figuring anything out ahead of time -- actually answering the hard questions like "what do we want this software to do and how do we want to do it?" they just start making something. It's like just giving some steelworkers a pile of rebar on either side of the river and telling them to build towards the center and figure the details out later.

I think one of the biggest problems in software development, and it's really endemic, is an underappreciation of the pre-development work: requirements analysis, specification development, even simple stuff like the clear division of job roles and responsibilities. If you get that stuff done, the actual coding ought to be an academic exercise, not a seat-of-the-pants experiment.

Part of the problem lies with managers who don't understand software, and just take any opportunity to compress schedules and make themselves look better, and another problem is with "programmer culture," where people think the ideal way to solve a complicated problem is just to put a half-dozen developers in a room for a weekend with a few gallons of Mountain Dew, an Amex card, and the number of the nearest Domino's Pizza. (Although to be fair, this attitude seems to be less common among developers who have SOs/families, or are used to 40-hour workweeks.)

While concentrating on "getting it out the door" may solve the problem in the short run, it's almost certainly not going to give you neat, easily-maintainable, well-documented code. And when you're looking at the long-term maintainance costs, it sometimes can be better to not have anything at all, than to have a lot of spaghetti code that somebody is just going to have to rewrite down the road, when they can't figure it all out.

Re:No.

[mcrbids]

If somebody came to you and said "hey, I've got this great new way to build a bridge! Instead of making up plans, we'll just start building it! We'll build it out of popsicle sticks first, and then we'll go in and add some steel beams, and toss some pavement on top of that," you'd say they were insane. Nobody does stuff like that in the real world -- yet that's exactly what a lot of poorly-managed 'agile' software projects are doing. They're getting short-term prototyping gains but at the cost of maintainability and probably stability as well.


A complex software project doesn't compare well to a bridge. It's more like a city. Nobody goes and says "Let's build a city!", lays out plans, prototypes and discusses what business go where.

That's just stupid; nobody does a city like that.

A bridge is a simple item, an artifact of intelligence, it's an item with almsot irreducible complexity. A city, however, is a highly complex, interactive social organism with bazillions of interactions, many of which you can't easily forsee. It has a very small level of irreducible complexity. Lots of software has much in common with this.

Cities ARE built in a "agile development" fashion. People spec out just enough to get started, to get them by for the next few months/years, slam together a some houses, and maybe a store or two. People like living there, then somebody comes along and thinks "We outta have.... a Newspaper!" and they build a building and buy paper and start writing articles and printing and all that jazz. It's a little different than software because most software has a lifespan of perhaps 10-20 years, cities have lifespans in the hundreds of years.

Changes in this city are incremental; they're small. They happen everyday. Somebody does an extension on their house. The lady down the street gives birth to a son. The corner grocery starts selling sandwiches. The empty lot down the street becomes a movie theatre. And so, the agile development model continues.

You're dead wrong - the bridge is a small, incremental example of EXACTLY how well-done agile software develops!

I agree

[The OPTiCIAN]

> What do you think?

Spot on. It's funny to watch people do demonstrations of how quickly Ruby on Rails can be used to build something because it's exactly the same sort of thing that was used to promote WebObjects ten years ago and I know from experience what rubbish it is. For all but the most simplistic applications you have to abandon the mapping of form elements to the database because you need to do validation. If you start off with a RAD approach to problems to these pages and add validation as an afterthought they quickly degenerate into a horrible mess. There will be less of a problem in this respect with ruby on rails because the data layer is so primitive so there are some knots you can't even contemplate being tied into, but - um - what was the point of Ruby on Rails again? :)

I imagine that doing major schema refactors on Ruby on Rails apps would be a nightmare because there's no easy way to check that you've fixed all the breakages. Whereas if you use EOF or Cayenne and get a culture in your software where developers avoid using key-paths except in agreed spots it's quite easy - you make the change, fix the areas where you find compile problems and then its done.

Something I would be interested to see would be some sort of business logic layer that could emulate a JDBC adaptor. Then you could write your application against that and bind to it as though it were a schema, but in the background it would in fact have business logic behind it. This would allow a separation between business logic and presentation but still allow you to quickly bind applications up as you do in the RAD webapp tools.

Re:It's a disposable culture.

[wtansill]

Maintenance? If you can lower the cost of creation enough, it's cheaper to just start from scratch every couple of years. It's the same phenomenon seen in blenders and automobiles.

Ehhhhh -- no. Maybe some trivial applicationm can be thrown away and reengineered every few years, but business generally does not have that luxury. There are data conversions to consider, as well as the fact that, depending on the system in question, an entire "ecosystem" has likely grown up around the core application. You cannot just throw the application away without ripping out and reworking all those other systems that depend on it in some form or fashion. Yes, I know -- Software as a Service to the rescue -- well defined interfaces between applications, etc. Except that in the main, that does not happen, so you are better off having an extensible, maintainable system. Now if I could only sell that idea to the PHB...

Re:i think you answered your own question

[Anonymous Coward]

As Douglass Adams said: the problem with things that can't possibly go wrong is that when they do go wrong, there's no way to fix them. When you accept the "easy" models promoted by some of the higher level languages, you might take the framework your using as far as it goes and realize that it doesn't go far enough. At that point you're stuck. For example, you may have written a powerhouse GUI application for a platform like .NET or Java and now that you've invested millions in development, you realize that there's nothing more you can do to optimize performance on that platform: your application is a memory hog and takes forever to load. It's still a good application, but your choice of development tools has put a hard limit on how far you can go. That's a tough spot to be in.

I think you got it backwards

[humblecoder]

I think the problem isn't that there is too MUCH focus on the beginning of the software lifecycle. The problem is that there is too LITTLE focus on the beginning of the software lifecycle.

The beginning of the software lifecycle is supposed to consist of analysis and design - both of which can lead to the construction of a superior product if done right. The issue is that many of these "quick start" languages and frameworks make is easy for a programmer to dive right into the coding phase without considering the overall design of the system. Thus, they skip the beginning steps in the software lifecycle.