Slashdot Mirror
How Do You Prove Software Testing Saves Money?
cdman52 writes "I work at a small software development company. We have one app that is used by a few hundred clients and was initially developed by a few undergrads about 10 years ago. The app is collection of about 25 developers preferences and ideas. Testing wasn't an initial concern since it was created as an internal application, I guess. Anyway, the app is now large and used frequently. Part of my duties are to fix bugs users find, I'm on a team with a few other people and at least once every 2-3 months I see some bug I fixed come back, and I can only assume it's because we don't have a formal test suite. The owner doesn't want to invest time or money in getting one set up, but I'm sure that in the long run it would save time and money. Can anyone offer suggestions for how to convince the owner that setting up a test suite is in his own best interest?"
Can anyone offer suggestions for how to convince the owner that setting up a test suite is in his own best interest?
Are you sure that it will be?
but I'm sure that in the long run it would save time and money
How much time and how much money?
What you need here is hard numbers to back up your opinion. Specifically, and I hate to say the word, but you need metrics.
- How much do these bugs cost (these are a mixruee of hard numbers (dev time, etc) and "fuzzy" numbers (customer relations and so forth)?
- How much is setting up this test suite going to cost? How much is it going to cost to use it?
- How effective do you think it will be (look at existing bugs.. would they have been caught)?
That last one is especially important. Testing is good, but it doesn't solve all problems and there are other options. Really take a good look at whether previous bugs would have been found in your proposed system.
Once you have this information, the math then becomes pretty simple.
If your program is modular-ish .. it might be possible to do a trial. This is probably a good idea anyway. There are different ways to do testing, some of which are incompatible with certain development mindsets. Best to find that out early on a small piece vice the entire code base.
Just start small, with "failing tests" for each new bug. Bug fixed, test passes. Keep expanding the test coverage.
http://stephan.sugarmotor.org
We've had to justify creating auto test suites where I work.
Over the last decade our product has grown from one code-base into three strands, each with separate customer foci, and we've had a healthy amount of staff turnover so that there are still brilliant, creative and skilled people working on it but some of the original knowledge has left us.
We found* numbers to justify that automated testing of existing features, applied later will protect against regressive changes. Even where there are complicated features which were not modular in design, or which lack good interfaces, the tests have saved us massive amounts of time testing by hand. The real win is hidden under something we didn't realise until later: creating the tests have forced us to really document what the features are and how they work**, sometimes from a unit-test level, sometimes at the interface level and sometimes in a top-to-bottom vertical slice. Once you have a record of what your software does, in a computer which is skilled at remembering exactly and repeating exactly what some former staff member told it a couple of years ago, you have a decent reason to be confident that your bug fixes won't cause more harm than good.
*: ballpark figures / educated guesses / made up.
**: We favour working code over comprehensive documentation, until our agile team is reassigned to other projects or leaves the company.