Are Regression Tests an Industry Standard?

Sludge asks: "I just finished leading a team through a software project. It was the first of it's type for our company: financial transactions were involved, and it was therefore very fault intolerant. In order to complete this, a set of regression tests were written. For example, if the amount of money collected doesn't match up to our order table, we get notified via our cellphone's text messaging as soon as the cronjob picks it up. Lots of other implementation-specific tests exist as well. My question is, how common is this for the software industry? My company had never heard of this before I came along. Is it the norm? (When you answer, also say whether or not your company does risk management.)"

Comments

Re:Regression test tools?

[renehollan]

We did it the old fashioned way: designing unit tests for each function and state transition for stateful interactions. The results were matched against what was supposed to happen, and stimuli for errors were included as well as stimuli for non-error responses. You can scale this technique up to subsystem and system testing and adapt it for client/server and p2p protocols.

You catch doc. errors this way because when the test reveals an error it means one of three things: (a) you coded the test wrong; (b) you found an implementation error; (c) the implementation is right, but you coded to the docs, which were wrong.

Now, if you use various 4GL tools (dunno if Rational Rose lets you do this), they might be able to automate the tests for you.

Good luck.

industry standard nomenclature

[Pauly]

In order to complete this, a set of regression tests were written. For example, if the amount of money collected doesn't match up to our order table, we get notified via our cellphone's text messaging as soon as the cronjob picks it up.

What you describe isn't regression testing. Regression testing "is a quality control measure to ensure that the newly-modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity." More accurately what you've done is paranoid programming. Really, these two things are orthogonal.

My question is, how common is this for the software industry? My company had never heard of this before I came along. Is it the norm? (When you answer, also say whether or not your company does risk management.)

This depends. Every company I've worked for has claimed to be concerned with mitigating risks both in the testing phases and post-release phases of the software development lifecycle. However, the amount and kind of testing and programming actually done have varied wildly and always ends up being determined by the industry for which the software is being built. In your case, money is the biggest factor. Organizations such as banks and other financial institutions are highly risk-averse due to the responsibilities and legal concerns related to handling others' money. It follows that these organizations regularly conduct formal testing of their code as well as "program paranoid" to mitigate screw-ups. In start-up's I've worked at in the past, this wasn't nearly the case since it was more important to get a product out the door and this sort of testing/coding always went out the window with looming deadlines.

So to answer your question, yes, regression testing (and paranoid programming) are highly common in the IT industry and their respective importance is a function of the risk aversion of the intended users/customers. My advice is to always practice good, paranoid, professional programming augmented by formal testing procedures. Vary the time spent on each to achieve the appropriate balance.

Frankly, the best way to enlighten yourself on this matter is to educate yourself in the ways of Extreme Programming. The horribly trendy name aside, this is the truly the only management fad I've seen in 10+ years that holds any merit.