Slashdot Mirror
Suggestions for Performing Regression Testing?
gmletzkojr asks: "The company that I am currently working at develops a fairly complex industrial controller, complete with embedded software, a GUI on the controller, and a Windows app to connect over Ethernet. On previous versions of a similar project, we have performed testing manually - ie, monkey presses button, and sees that light turns on, widget turns, GUI updates, etc. However, this is extremely time consuming (previous complete regression testing took ~3 weeks) and is error prone in itself. How do you perform complete system regression testing? Do you use shrink-wrapped packages, or build your own? How do you test features that are easy for humans to observe, but not as easy for software to detect (ie, the light came on, the GUI updated when I pressed the external input, etc)?"
Looks like the free AutoIt would help.
Also see AutoIt Script home page.
The more expensive programmer will in the end catch more bugs with the regression test suite, so your code has less bugs when you ship. However it is expensive.
I think this can be true if you write your regression tests afterward. But if you're doing test-driven development, so that your app is designed from the beginning for testability, then I think the costs are pretty reasonable. In my experience, the time testing is more than made up for by the time and stress saved on debugging.
On a recent 9-month project, a team of four developers I know built a web application with no external testing, just self-written automated tests and product managers playing with the app. After six months in production, they've had a total of two reported bugs and no scheduled downtime.