Slashdot Mirror
Retrofitting XP-style Testing onto a Large Project?
Mr Pleonastic submits this query for your consideration: "I work for a small startup (ok, me and another guy comprise the entire development team) that has somehow managed to survive the bust, attract a number of customers, and build up about 300K lines of functionality. Up to now we've made it by being smart and conscientious hackers, but I'm increasingly embarrassed by our shortcomings in testing. I like the XP approach to making enduring, automated test suites, but most of what I read about XP focuses on obvious stuff and changing your programmer culture at the outset. Does anyone have experience with, or advice for, retrofitting it onto a fairly mature project? What do your test suites look like, anyway? The bugs I fear most are of the 'If the user does X and then Y, the result blows away our assumptions' variety, not the 'Oops! My function returned the wrong value' variety (which happens of course). How do you write good test code for the former, without spending even longer debugging the test code? Is XP just for small, new projects?"
Finding conditions that are outside your assumptions is not something you can do with a unit test. I have found that trying to simulate user creativity (stupidity?) with unit tests is an exersize in futility. Use your unit tests to make sure your methods do what they're supposed to do.
To find all those tricky combinations of use cases that blow away all your assumptions, just stick to the Fail-Fast principle. If you find anything that goes even slightly wrong, complain. Loudly! Throw an exception, pop up a dialog, whatever you need to make sure that everyone knows an error just occured. This will do two things:
1) You'll find a lot more errors in your code. You'll also be motivated to fix them quicker because the app will be unusable until you do.
2) You'll reduce the likelihood of generating bad data. The only thing worse than your program doing something wrong and crashing is doing something wrong and NOT crashing. Users will usually forgive you if your software crashes. If you start giving them bad data, they'll lose confidence in your app and never trust it again.
I have a similar situation, I have a bunch of code that "mostly works" and I'd love to have unit and acceptance tests.
But it's really hard to add it later. I mean REALLY HARD. The tests are tedious and boring and after 2-3 I get tired and the tests have errors.
If you follow the XP test-first technique, you code comes out MUCH different. You have low coupling, you have "testable" code where the pieces are interchangeable (so you can easily use mock printer objects or non-RDBMS backends, etc), and generally it's really elegant code with little extra work.
And you don't get bored writing test-first because every time you write a test, you then write the code that passes the test and it's really a feeling of accomplishment! And you don't get "lost in the big picture" because you are focusing only on passing that one little test.
The same is true for acceptance tests. I use HttpUnit to automate web apps, and although I'm not quite as religious about testing the interface, it's great for "add record, query record, delete record" stuff, to make sure it doesn't blow up when the end-user does something basic. For instance I had some code once that worked wonderfully, except login was broken. Since I was testing while logged in and never thought to log out and log back in, I never caught it in my manual tests. Automated tests can catch the stuff you forget.
So I'd recommend requiring tests on all NEW code (you'll see a big difference between the old and new code I bet, in terms of simplicity and low coupling).
And whenever you refactor the old code, start by writing tests that the old code passes.
But it will really be tough to retrofit ALL your old code with tests. I'd even say it's not worth it because your tests will not be good.
And remember: EVERY LINE OF CODE MUST EXIST TO PASS A TEST. That should be your goal on new code.