Business Software Needs A Revolution

An anonymous reader writes "According to a Businessweek Online article, today's high-end business software is bloated, buggy, and too expensive - no surprise to those of us who have paid our bills by adding pointless features to some piece of software arbitrarily priced at $100k. Evidently, firms are now re-evaluating their software purchases, and finding that they're not working out the way the sales guys told them they would."

Comments

Market forces control software quality

[dtolton]

There are some good points made in this article. Working at a
software company, there is quite frequently an incredible amount
of pressure to get new features in as quickly as possible.

However I don't think that phenomenon is ever going to go away.
To a certain extent there is market pressure to add new features
to your product, people always want the new bells and whistles.
There has been a tremendous market pressure over the last decade
to add bells and whisltes over bullet proofing your code.
Perhaps there will be some pressure now towards bullet proofing
your code, but until customers stop demanding more features and
start demanding quality code, software won't change.

There are some companies out there (M$ being the prime example)
that don't add much in the way of new functionality, but rather
repackage things, move buttons and menus around and make the new
incompatible with the old. At the same time they only fix
certain bugs, but leave others alone. Yet people buy their crap
at record rates.

I think most developers would love to see a move towards
software quality rather than software features, but until the
market dictates that as a priority it just won't happen.

Re:Market forces control software quality

[Daniel Dvorkin]

Yep. Companies rush buggy bloatware out the door because, within certain limits, that's what sells. Granted, there is a limit; apparently Oracle exceeded it with 11i, and even fairly small bugs can be big problems if they get publicized and the company responsible handles PR poorly. (It's hardware, not software, but the floating-point bug in the original Pentium comes to mind here.) But those limits are, all in all, extraordinarily loose.

PHB's love to buy "all-in-one" and "easy-to-use" solutions that can be used by morons, instead of hiring people who know what they're doing to assemble a solution out of reliable, well-tested components (which often are used from that scaaary command line.) Often they're seduced by the idea that such products will keep them from having to hire those weird, long-haired, jeans-and-t-shirts people to administer everything, because companies make absurd promises about how easy the products are to use. In the end, of course, these products end up costing more, because things fall apart at the worst possible time and someone (or a bunch of someones) has to be brought in now to fix things. But the PHB's never learn.

Re:Market forces control software quality

[killthiskid]

Market forces are definately part of the big picture, but companies are self-preserving entities. The goal of any business is to make money and that does not translate into quality software. I found this interesting:

For starters, give up the "not-built-here" dogma that has kept some software makers from working with new, easy-to-use programming building blocks made by Microsoft, Sun Microsystems, and IBM. That reluctance also has made some companies slow to adopt standardized programming technologies like the Extensible Markup Language, which makes it easier for different kinds of software to work together.

It amazes me when I work with software 'solutions' that have cost millions of dollars that have no interface in or out of them other than the specific stuff provided by the vendor. I'd kill for direct access to the underlying DB or a nice clear way of moving data in and out, or a great way to make custom GUI... but the company is more concerned with ensuring that we are locked in FOREVER than with providing the tools we could use to make their software more friendly to our over all IT enviroment.

I think we will see the service model slay many of these large monolithic enviroments, but the transition is going to be very painful... the infrastructure and data investment is going to be a large, expensive hurdle to overcome.

Re:Market forces control software quality

[Smallpond]

-- Companies rush buggy bloatware out the door because, within certain limits, that's what sells.

To be more precise, that's what Sales and Marketing says sells. I think if you ask most customers, they'ld prefer software that works and solves their problem. Generally, Sales blames any lost sale on the feature that the other guy has and you don't. Thats why the emphasis is on adding features over quality, its to take away the sales excuse, not to satisfy customer demand.

Re:Market forces control software quality

[GreyPoopon]

I think if you ask most customers, they'ld prefer software that works and solves their problem.

And herein lies a significant portion of the problem. Customers frequently don't really know what they want or need. Before software gets leaner and better, there's a number of things that need to happen.

• Corporate Complexity needs to be eliminated. Many corporations have built an incredibly complex structure over the years and are unwilling to redesign their business processes to make the best use of software products. I'm sorry, but you'll never convince me that the processes that have slowly built up (layer upon layer) over the last 100 years are anywhere close to optimal in today's market.
• The government needs to simplify their regulations. You'd be shocked at how much of the fluff in business software is simply to satisfy local government regulations. The complexity becomes staggering when you apply it to international corporations.
• Management needs to get in touch with the people in their organization who will be entering data. One of the biggest complaints about business software is the complexity and sheer volume of data that needs to be entered. This is almost always because management wants this data to be collected. If management is convinced that their sales force needs to collect 500 data points on each prospective customer, no software package on earth is going to make the data entry process easy or trivial.
• And darn it, get rid of the "we want this because we had it in our 'old' system" philosophy. Consultants are constantly adding legacy items to new systems because the client insists that they must have something they had in their old system.

Re:Market forces control software quality

[Ra5pu7in]

Would a carpenter build a house knowing the roof would fall in ...? No, because there are laws that make him responsible for structural issues even after the house is sold. However, many a carpenter has been known for using the lowest quality materials possible because the customers want to pay less for the house. Buying lumber that is completely cured, re-using nails and laying on over-heated tar may not produce a roof that will fall in, but it will reduce the price of the home -- with the pitfall of the roof leaking a few years into its life. The business consumer needs to get work done and has been conditioned to accept that "no software is bug-free" out the door. He runs into a bug and most likely gripes the first few times, then avoids it. Imagine what would have happened to Microsoft if the market had refused to buy any computer with Windows 98 on it until all the known bugs were handled. Quite a different scenario than what we have. As long as the consumer wants the cheapest possible software that will do the job he needs, companies will push their coders to get it out the door (making money).

Re:Market forces control software quality

[vsprintf]

Customers frequently don't really know what they want or need.

Customers know exactly what they want even if it may not be exactly what we think they need, and that's a big difference. Software types are far too willing to let their assumptions and preferences produce a solution that is what the customer "needs".

Corporate Complexity needs to be eliminated. Many corporations have built an incredibly complex structure over the years and are unwilling to redesign their business processes to make the best use of software products. I'm sorry, but you'll never convince me that the processes that have slowly built up (layer upon layer) over the last 100 years are anywhere close to optimal in today's market.

It is the individual business rules that make each company different, give each an identity, and give each a possible niche. Why should we have cookie-cutter companies just so we can install cookie-cutter software? I write software, and I believe it is the software that needs to adapt to individual companies, not the other way around. And any company that has been around for 100 years probably has some real business acumen, even if it doesn't fit well with your favorite COTS.

Re:Market forces control software quality

[SN74S181]

An important part of capitalism is for companies like this to fail.

This just in-

[IWantMoreSpamPlease]

Marketing speak does not translate to real world performance.

Seems to me that if I was spending 100K plus on a software package (or system) I would test it first to make sure it fit my needs, as opposed to listening to a marketing drone...

Re:This just in-

[Mindwarp]

Unfortunately (and I've seen this happen up-close-and-personal) the sparkly new features are quite often the thing that'll sell your package to the PHB sitting opposite you in the demonstration room.

It's very easy to run your demo and say - lookit the pretty colours! This is what our software does that our competitors doesn't! It's much harder to wow your audience with "This demo machine has been running for two weeks without a single crash!" The uptime statistics for these large corporate packages often end up as a dry piece of paper which ends up landing on a SysAdmin's desk and causing him to cry.

Remember that Chicago song "Give 'em the old Razzle Dazzle!"

Salesmen Lie

[Bonker]

They lie to their developers when they say 'All we need is this one feature to make customer 'X' happy'.

They lie to their customers when they say 'And this feature our developers just put in will make your life easier'.

The hell of it is that when developers put in 60-80 hour weeks coding bloat features, the salesmen are the ones who get bonuses for making a big sale.

The problem here is not so hard to see.

Re:Salesmen Lie

[salesgeek]

The part of the problem you are seeing is what is not so hard to see.

You are right, that some salespersons (it's not just the male kind) lie. On the other hand, there are many salespersons who pride themselves for their integrity - and are painfully honest at all times. The problem is that buyers lie, too and developers? Let's just say that honesty is a human problem, not a "salesperson" problem.

The real problem is what your salesperson has to do to keep the lights on at your firm: he or she has to find someone who needs what you do, then create a deal that is profitable for you and is good for the customer. Now imagine what happens when the customer does not disclose information or misleads the salesperson? What about managers who insulate salespeople from developers and engineers? The hell of it really is that doing good business is hard: everyone has to do their part, sales, marketing, finance, production, accounting and operations with integrity and accuracy or there is a liar or two created and possibly a bad deal done.

So far as bonus goes, I can assure you for every 60-80 week you put in coding, there is a salesperson who puts in 80-100 hours on the road, away from family presenting your software, writing proposals and getting completely and utterly rejected by 200 people to find one that wants whatever your company makes. At the end of the day, you make the product. Your salesperson finds a buyer and pays your paycheck. The dotcom folks learned the hard way: you go out of business if you don't sell anything or if you don't have anything to sell.

$G

I've always wondered

[nate1138]

I've wondered for a while what the point was. For the price of some of these packages, you can hire 2 developers (or more!) for a year and get them to code an application that does EXACTLY what you want. As long as you stick with a fairly standard architecture and document it well it should be just as effective. Using components that have already been developed (such as various jakarta subprojects) can really speed projects like these and make them worthwhile. Most importantly, it is custom tailored for your business.

Re:I've always wondered

[Mr. No Skills]

There are certainly some purchased products that could be developed or reverse engineered in a year. But, when you scope up past a department solution, or into more complex transactions, this is simply not true. As others have pointed out, you only have to look at the amount of time other projects have taken to see there are limits to this.

Enterprise products are purchased because:

1) It takes man years just to produce requirements for complex applications that developers can use.
2) It can take 2 FTEs just to do the DBA work on a relatively complex enteprise app, let alone all the developers, testers, documentation people, and trainers it takes for the rest of the app.
3) Most companies are not in the business of producing software and can't manage projects like this. They are in some other business and need to focus their energies in those directions.
4) With enterprise apps you pay for things other than the app. You pay for the time of not having to develop, of course. You also pay for something that is secure, audits its transactions, allows flexible reporting, maybe works across multiple time zones, maybe with multiple languages, maybe allows different database technology, and maybe integrates with other stuff you have. You want training. You want support. You want the ability to upgrade into new versions or onto new OS/HW platforms. And you want to yell at something when your expensive app doesn't work, as opposed to someone yelling at you for all the money they've sunk into their home-grown solution.

I think your point is valid for a lot of projects. And I think a lot of companies overspend on solutions because they don't understand their needs or overinflate their risk. But there are a lot of complicated systems that 2 people could never pull off without so much time that the technology moves before you get the app out the door.

So what's new?

[binaryDigit]

is bloated, buggy, and too expensive

Software has been following this general trend for years now (except the too expensive part). I know this is like the "when I was your age I had to walk 50 miles in the snow up hill to school at 4am" kinda whining, but I'm going to do it anyway.

Fact is, other than watching video files and ripping cd's, why is it that you need an OS that requires more RAM than you had HD space years ago for. If you map computing oomph (mips, ram, hd, video speed/resolutions) and software functionality (say on the y axis), you'd end up with an incredibly dissapointingly near flat line. With as much horse power that we have today, we should be able to create nearly bug free software because of all the majorly powerful development tools that put all this power to good use. Instead, we have majorly bloated development tools (Rational Rose et al) and environments that focus on letting people make pretty but ill conceived ui's and make a half hearted stab at helping to improve code.

Bah, humbug.

Little off topic...

[DragonMagic]

A bit off topic, but related. I got tired of trying to find a really decent store suite for e-commerce. Most of the ones which did what I needed cost hundreds to thousands of dollars, and would require additional licenses as more features or bugs got fixed and made a new version release. Going from version 3 to version 4 would be a $200 upgrade, or buying the package to include coupons would be $150.

The free or low-cost solutions did little, were hard to use, and were buggy as well.

So people really had two options. Pay plenty for good software that they would have to continue paying if they wanted to keep up to date in modules and features, or pay little or nothing and get software they would have to invest additional money into making work how they need.

So, like those before me in the free software world, it made me start my own software suite for e-commerce, to be released free under the GNU GPL license, because I would already need it for my site, so why not give it to those already in my position.

I think this is where business software will be going. A small company or a programmer will find that there is a need for a software package or suite. There will be two options, expensive lock-in software or cheap hobbling software. They will probably decide in this economy it will be easier to either build off FSF-approved-licensed software to make it work how they need, or just build their own.

I'd love to see this option work out well. An alternative to Peachtree/QuickBooks for all platforms that is XML based. Linux-based POS software for stores. Inventory management and shipping database applications.

Why spend $100,000 on such a suite when you can just build off a free project, or start your own using the knowledge you already have with the suite you couldn't get working how you wanted, open it up and reap benefits from servicing contracts and support to other companies in similar situations.

Or am I just dreaming?

When will management get it?

[prgrmr]

Analysts estimate business-software customers spend $5 installing and fixing their software for every $1 they spend on software.

The management mindset of "do it right now", as opposed to "do it right" is costing them more in both the short- and long-term. Until prevailing attitudes are changed on the part of those making the purchasing decisions, software makers will still have little motivation to change.

My employer is looking at a $1 million+ project for HR automation. $40k of that is for the unix server, the rest for software and "services". And this was, supposedly, the best software available. The vendor also recommended a .75 FTE for a Unix Admin for on-going support. This is about 5 times the need of any of our other Unix servers, and makes me wonder how much care & feeding the system will require just because of the buggy application. From my observations, the numbers quoted in the article for fixing software don't look high at all, and may in fact be too low.

maybe you and i would check, but unfortunately....

[feepcreature]

Seems to me that if I was spending 100K plus on a software package (or system) I would test it first to make sure it fit my needs, as opposed to listening to a marketing drone...

I'm sure you would, and I certainly would. Unfortunately in the corporate world these decisions are too often made, for "non-technical reasons" [1] by people who lack this apparently simple insight. I've seen too many inappropriate purchases made, of over-priced, under-functional software and systems that looked like it did what the purchaser thought he wanted, but in real life failed to do what the company acutally needed. Or had prohibitive costs. Or...

And I don't believe that my experience is particularly unusual.

[1] don't ask :-(

Simpler Business Software?

[xdroop]

It will never happen. And I will tell you why.

It is because software is supposed to bend to the will of the user, not the other way around. And that is why software is so feature-laden, so mandatorilly configurable.

If you wrote yourself a business app without configuration, you are dictating to your customers exactly how they will do the business they intend to use your software to do. That's great if your customer either does not do business in this area yet, or if they already do business the same way you expect them to.

But guess what! 99% of the target customers out there do or will want to do business differently!

For example? Let's pick something we can all agree on: source code management. Now how are we going to do business? Every shop will have a subtly different answer.

And that's the problem. Customers frequently don't know how they do business, and forcing them to articulate their current processes leads to them facing this unpleasant truth. Sometimes they tell you the wrong thing. Sometimes they deliberately tell you the wrong thing.

Sometimes management gets wind of all these neat metrics that the new system will be able to measure, and those get tacked on to the requirements sheet.

Sometimes there comes a requirement to seamlessly interface will all these legacy systems. Oh, and seamlessly sort, classify, access, and audit all the legacy data too.

You get the point.

The comparison with the auto industry is similarly bogus. The auto industry has a sharply restricted list of top level suppliers (Ford, GM, BMW), has the infrastructure to provide all routine supplies (like computer units, replacement hoods, etc), has a universal interoperability standard (the road), and a standardized operator interface.

Until the requirements for Business Software becomes simple and universal, the software fulfilling those requirements will never be either simple or universal.

Enterprise Software, Sales and Marketing Hype

[salesgeek]

There's enough blame to go around to everyone in the enterprise software rip-off game:

* Buyers for not even applying common sense to outrageous claims

* Software companies for overselling and underdelivering

* The press for pandering to the software companies for ad $$$ a the risk of their readers

* Consultants and IT managers for using buggy implementations as job security

Shame on all of us.

Because of tolerance

[www.sorehands.com]

The situation with the number of bugs is because people tolerate it!

The Sprint PCS Vission support person told me to powercycle my phone when I was having connection problems. This was to flush the buffers and cache. They said that this is a common problem with all software and said that even Microsoft Servers have to be rebooted every few days.

If we hold companies' feet to the fire and and be demanding, they may change when we start demanding refunds for buggy software.

No bugs is good bugs!

Why does software sucK?

[jafac]

It's that same question ALL OVER AGAIN.

Software sucks because there is no demand for quality software - hence software vendors do not need to implement internal engineering processes which could ensure that the software is good (to a degree).

The problem happens when you get into the Enterprise "space", where companies are accustomed to spending huge amounts of money for products that are engineered to be bulletproof.
Then they settle for products which were coded the same way consumer companies coded their freeware.

Here's how to solve this problem:
Company X claims they sell "unbreakable" software.
Their software breaks.
They get their asses sued off, or handed to them by their competitors. **IF** their competitors use an engineering process to write their software, and IF they can show that process, and prove it to their customers that they use it, and show, with numbers, how it helps.
In other words, act like a REAL software company, instead of just another dotcom trying to make a quick buck before another dotcom is tricked into buying them.

What do I mean by "engineering process?"
http://www.sei.cmu.edu/cmm/cmm.html
(o r comparable methods)

Yes, it costs a LOT more to do these things. But they work, and should be a great selling point for people buying software. Unfortunately, the main reason we had a "computer revolution" in the 80's and 90's is because software was so cheap to produce - because it was being written, compiled and shipped. Not engineered.
If there was a demand for quality software, then it would be worth it for a software vendor to use these processes to ensure quality. And it would be profitable, because they could charge a lot more. Some vendors DO this, and get their asses handed to them by other vendors who do not. Which brings me back to the original problem. STUPID CUSTOMERS.

So, stop whining about sucky software.
Stop spending money on software in the "Enterprise" price range, without knowing that it is, in fact, Enterprise quality.
Stop listening to Gartner Group and other ANALists. Stop reading lame trade rags. Their corruption is what allowed this market to degenerate and devolve to the state it's in today. Their job was to educate responsibly, and they failed miserably. But they did make a lot of money from SUCKERS along the way.

Been there, done that

[f1f2f3]

I've been a software Product Manager at some of the biggest software producing companies in the world since 1995. I'm the guy an awful lot of you coders seem to dislike so much. You know, the one always asking for just one more feature to be squeezed into the release, and, oh, can we get it two months early?

There is great truth in this article, and a great lie. Software companies publish buggy, bloated product all the time, but not because it's fun and not because we marketing weenies think it's such a good idea. It's because that's what the market wants. The idea that customers are asking us to stop is a load of crap created by journalists looking to write about the latest backlash.

Sure, as the article points out, Oracle 11i was bug-ridden, but how many millions did Oracle make off it? Claiming a 11% drop in revenue in 2002 is just a tad misleading--who *didn't* see a drop in revenue in 2002? Didn't some bubble burst or something? Bottom line, customer's bought the software, bugs and all. And you can bet an awful lot of them were screaming at Oracle to get the software out as soon as possible. So where's the motivation to do it any different?

Every customer will tell you he wants just one more feature, or just one critical (to him!) bug fixed, and then he'll be happy. Bunk. Fix the bug, add the feature, and get ready for the next demand. And since what Customer A wants isn't always what Customer B wants, we get lots and lots of features, many of them aimed at a very small subset of users. Add to that all the customers screaming since the customers want it *right* *now*, and we ship software with way too many bugs, and lots of silly features. Which customer pay for.

Self-created problem

[Tablizer]

Feature-creep is often caused by "the customer is always right" syndrome. The benefits of adding the feature are considered, but not the total cost it. It is sort of like cocaine, or a greasy burger: it might not kill you today, but eventually it will, or at least make you disfunctional in the longer run. But people ignore the downsides because they grow subtley.

Further, a lot of the tweaking is for vanity purposes IMO. I could build a system that could generate complex business applications mostly just by filling in values in a "data dictionary" set of tables that describe fields and relationships between them, plus a few event handlers. However, the interface would be so boring and predictable that it would be ignored. It would lack a WoW factor that people seem to want. Managers like to stand out from the crowd. They *do* pick a system simply because it looks cool.

Finally, too many places fail to design their databases well. They let slop creap in over time, resulting in bad, poorly normalized tables. Don't let your schemas slide. Don't duplicate columns and data if you don't need to, and don't make a bunch of tiny one-to-zero-or-one tables.

If only it was simple to fix.

[nomadicGeek]

It goes back to the old adage that in theory there is no difference between theory and practice, but in practice there is.

I donâ(TM)t think it is simple to fix. Business processes tend to get complicated. Companies must try to meet the needs of many different customers. The end result never looks like something that was designed by one person but rather a conglomeration of the attempts of many people to make everyone happy. You canâ(TM)t just tell your customers to deal with it this way because the programmer doesnâ(TM)t want to spoil the elegance of his system.

You also have to take into account that you rarely if ever get to start from scratch and make everything simple, homogeneous, and elegant. You always end up having to get this file from the old Unisys machine, access this database on the AS/400, ftp this file from another old system, etc. Deal with this Perl script, that COBOL code, that piece of JAVA.

These âoepackagesâ that you implement end up being about 75% code from the vendor and about 25% glue and hacks implemented on-site to meet requirements. So the complexity arises from having to meet all of these needs when a software system is implemented. Each new project adds a few more goofy requirements.

When you start integrating with legacy systems and code from all of the place it is very difficult to adequately test. If you sit down and look at everything it is impossible to develop enough test cases to cover the full range of possibilities. They are practically infinite. Every machine, OS, OS patch level, language, library, communication protocol, file format, database, etc. throws another opportunity for a failure.

You also never have a stable target. Businesses merge, spinoff, develop new products and services, move, layoff, downsize, rightsize⦠At the drop of a hat. When you think about it, we are kicking butt to have anything working.

Re:Better to buy or bring in house.

[Tintagel]

I don't want to seem mean but:

You admit that you lack accounting knowledge, but you wanted to code an accounts payable application yourself? Using open source software won't magically give you accounting domain knowledge. At least GreatPlains offered a framework for setting up an accounting system, even if your management mishandled the implementation.

As you now hack up some PHP+mysql app, you've hit the core problem mentioned by many, many others: users don't know what they want. ("The department that need(s) automation dont know what to automate and cannot come to conclusions.") But accounts payable is a *standard business function*. It's been implemented thousands of times before by companies like yours. Your staff don't need to know what to automate; in this task, they're hamstrung by 1) being accountants, not software or process designers, and 2) being used to the manual way they do their jobs.

So call up your suppliers, your trade association, the other companies in your building, whatever...someone who's done A.P. before. Ask who they used for implementation. Then call up vendors and ask how they handle the key requirements and pitfalls that you've heard. Stand on the shoulders of giants :-)

(Or, if you insist on coding something yourself, consider this: if you're automating a department that is 100% manual today, there is bound to be some very simple feature (in your eyes) that absolutely blows the staff away. "You mean we can list all the accounts 30 days past due in Chicago? With one click? Wow, that used to take Marge half a day to work out.")

(Oh, and your management needs to take charge and decide if they're going with an external vendor or an in-house solution. With the best will in the world, it sounds like you're torn between two completely incompatible implementations right now.)