Slashdot Mirror
Writing Unit Tests for Existing Code?
out-of-order asks: "I recently became a member of a large software organization which has placed me in the role of preparing the Unit Test effort for a component of software. Problem is that everything that I've read about Unit Testing pertains to 'test-driven' design, writing test cases first, etc. What if the opposite situation is true? This organization was writing code before I walked in the door and now I need to test it. What methodology is there for writing test cases for code that already exists?"
Before you actually fix it.
I don't see TestFirst as a Test Strategy, but as a design technique. Writing Tests first forces you to think differently about what you want to write.
This forces you to write testable code - writing tests afterwards does not force you to do that.
Of course, having the tests available later proves valuable for testing your application, but the tests main purpose is to lead you to a testable design
You'll most likely experience severe difficulties in adding Unit Tests to previously untested code. It might be easier to add acceptance tests (e.g. high-level scripts that utilize the application), especially if you want to cover more than small partitions of the application quickly
Write the tests as if the code _hadn't_ been written. Get the requirements and then write the tests from them.
Then if they fail the tests you'll have to discover if the requirements are wrong or if it's the code that's at fault. But at least you'll have something to start from - and you'll probably find some bugs they missed.
My Journal
You have to write tests as you change features. Lets say you have a simple change, tweaking a short method to do something different. First you write a test for its exisiting functionality, make sure it passes. Then add a test for the new functionality, run it and watch it fail. Make your change and make the test pass. This would also be the point where you can do some refactoring or clean up, or extend the test to catch boundy conditions.
1 177052/002-8698615-6720004?v=glance
With legacy code, you just have to start writing tests with the code as you go, writing tests for functionality that you need to understand or review. If you try and take x number of weeks to write test cases, your doomed to fall behind and have obsolete tests when you are dumb.
Also, see Working Effectively With Legacy Code by Michael Feathers --> http://www.amazon.com/exec/obidos/tg/detail/-/013
robsanheim.com
I spent several years managing a test team in a Fortune 100 company, and I have seen this situation many times (it's probably the norm, rather than the exception, in industry today).
Let the documented requirements for the code (or product) be your guide. Use those requirements to develop test cases, then design one or more tests that hit all of the test cases.
If there are no documented requirements, then you should ask yourself why you are working there. This situation usually leads to many arguments about what the code/product is really suppose to do, and you'll just become frustrated while you waste lots of time. It's not worth it.
Such tools can make after-the-fact testing quite a bit easier.
:-)
We used automated regression testing scripts in the mainframe environment I worked in 12 years ago, and that made some aspects of unit testing relatively easy.
Unisys had a tool (TTS1100) which allowed us to record each online transaction entry and computer response and then play it back later, and that made it possible to perform the exact same tests dozens or hundreds of times if needed. We used to run them after each set of changes was applied to make sure nothing broke.
One could also record a single occurrence of a lengthy interactive sequence and then add things like variables and looping structures into the recorded script to automate the handling of various test cases using different values.
Such a tool makes after-the-fact test design a little bit easier because you can sit down and methodically address each and every variation of each and every input field on a given screen.
Of course, the nature of the software you're using might make that sort of thing more difficult, or perhaps even easier.
I've never been able to do up-front unit test design -- specifications can change rather quickly when doing in-house software development, and the overall environment is a lot more dynamic than a typical "software house" environment would be where one always has formal detailed product specs to code to. We're often writing code based on an e-mail or on a couple of phone conversations.
Mainframe/UNIX Bit Twiddler and long time Windows/Linux Hobbyist.
The Theorem Theorem: If If, Then Then.
Unit testing is a method you use to achive something. Is the current component very buggy and you need to rewrite it or do you need to extend production quality software without breaking existing functionality?
If you are testing a component try to figure out if it is possible to in some schematic way. If you can figure out a way for the "business" people to write the tests for you that will take a lot of knowledge off your shoulders.
If it is an existing component maybe you could explore if it is possible to make some mechanism that "records" actions to the component and then later be able to "replay" them and check if the results are the same with your new or changed component as the production quality one.
I recently started as a contractor on a J2EE project that has lots of problems. The application has a classic backend with lots of ugly EJB anti-patterns and everything. The frontend is a VB client that communicates via. a simple webservice.
In a couple of days I was able to make a regression test engine that can save the xml-communication that our business-clever testers make to the server and then at any given time later run the same requests to the server and check if the responens match the originally recorded ones.
It works wonders and I now have free hands to clear out a lot of the technical mess while always having proof that I havn't broken anything.
Think about what your goals are. Then find the best tool to get there.
Gimme a fucking break.
Every testing job I've ever had we've had ZERO documentation. NADA. ZIP.
How do we survive? WE TEST. We put down the book (like we had one to begin with) and we test. Surely you have a server somewhere running dev-level code (at least) and you start poking around. Sure, its less than ideal, but you deal with it. And you bitch about how crappy it is and how it goes against all the principals of so-called 'real world' methodologies.
The thing is, this is how the real world does it.
Sure, in a perfect world, everyone has their shit in order. But in a perfect world we're not all competing against code monkeys working for 1/10th of what we make and that live in a 3rd world country.
Religion is for people afraid of going to hell.
Trying to write a test case for all the code you have will be very difficult, very long and to be honest not buy you a lot.
:)
A few open source projects have found themselves in the same situation as you, and they seem to work by 3 rules:
1) If you change any code at all which doesn't have a test, add a test
2) If you find a bug, make sure you add a test that fails before, and works now
3) If you are ever wandering around trying to understand some code, then feel free to write some tests
One thing I will say is to try very hard to keep your tests organised. Keeping them in a very similar directory structure to the actual code is helpful. Without this it's very hard to tell what has and hasn't got a test.
Combination - fun iPhone puzzling
I inherited a 1000 class Java based toolkit from my predecessor, which had exactly zero unit tests. Over the last two years, we've made a sustained effort to employ Test-Driven Development and add more tests to ensure that everything works as advertised. As of today the toolkit has over 830 tests, with line coverage of 61% and class coverage of 96%. We've still got a long way to go, but were much better off than we were. Here's how we got there...
1) A lot of people are going to tell you that you need to write your tests from scratch. That you should assume that your code is broken and work out the expected results by hand and create the test assertions accordingly. I disagree. If you're testing old code, it's much more useful to use the test to ensure that it does whatever it did before, instead of ensuring that it's "correct". I prefer to treat the code as though it is correct, and build the tests around it. Even if the assumption is occasionaly wrong, you can make the tests much quicker this way. That allows you to refactor and extend your system with confidence, knowing that you haven't broken anything. Remember, TDD isn't really about quality assurance, it's about design and evolving design through refactoring. More tests == more refactoring == better system.
2) You're probably not going to get a lot of extra time to sit around and write tests. You need to captialize on the time that you have and turn problems into oppertunities to add tests. Whenever you find a bug, make a test that reproduces it. If you need to add supporting stub or mock objects, consider making them reusable so that future tests will be easier to write.
3) If you need to add new functionality to the system, just follow the standard TDD steps of Test->Code->Refactor, and make sure that you add tests for anything that might be affected by the change.
4) I'm assuming that you already have a continous integration build that runs the tests, but if you don't, make one. Now. Also consider adding other metrics to the build like code coverage (we use Emma), findbugs, and jdepend. These will help you track your progress and can be very useful if you have to defend your methdology to people who view TDD as a waste of time (The Code Coverage to Open Bugs ratio gets them every time).
5) In general, you need to look for oppertunities to write tests. Don't understand how a module works? Write a test for it. Found a JDK bug? Reproduce it with a test. Performance too slow? Use timestamps to ensure that the performance of a alrorithm is in a reasonable range.
You've probably got a long road ahead, but it's worth the work. Keep at it, and good luck.
Guys, my legacy code doesn't have functions, just VB subroutines that modify global variables. Any idea how to make unit tests for this? And by the way, the functions aren't cohesive, each one is 100's of lines and does different sometimes unrelated things.
Often there aren't detailed enough requirements.
With requirements at a typical business level, you could have X totally different systems that meet them (and most better given hindsight). And often that level is as much as you're going to get when the original team has left.
Anyway recreating requirements at a detailed technical level could be a waste of time - because some module could be required to do something stupid by another module. Once you fix things all round, this requirement will be thrown out.
At one of my workplaces I made major changes in behaviour of some modules - e.g. instead of N^2 it's just N. And some things I just threw out because they were redundant.
I suppose, you could rewrite the requirements (after figuring things out), and then rewrite the code. But that's quite different from _getting_ the requirements.
It was good thing since QA categorically refused to "test" the software by trying to break it, they would only tested to see if it could work if the customer did everything right.
What kind of testing is that? You have to assume that the customer won't do everything right if you're going to find bugs. Just because you're using automated code testing, it doesn't mean that the unit tests themselves have been written correctly or all the code works perfectly together. A good QA team needs to have attitude that everything will be tested and everyone else can kiss ass.
A recent article in print about automated unit tests for legacy code was
"Managing That Millstone"1 c/sdm0501c.html
By Michael Feathers
Software Development
January 2005
http://www.sdmagazine.com/documents/s=9472/sdm050
It included suggestions for how to inject unit tests into code which isn't loosely coupled, some tips on how to refactor to get loosely coupled interfaces, & what you can do when neither of those approaches will work. It was a valuable & enjoyable read for me, at least.
gene
My suggestion that random banging on the keyboard, pushing buttons, and unexpectedly closing windows would be a good thing was not appreciated because there was no way to write it up as a test plan, or describe it as a repeatable bug.
In the video game industry, that's called button smashing. Programmers hated it because it meant that their input code didn't consider multiple buttons being pressed at the same time, and, worst, it was usually time dependent. Nintendo is very good at finding button smashing bugs.
You are so screwed. Writing tests for untested code is a thankless job. You are going to find so many bugs, and everyone is going to get really pissed off about that new hire that is rocking the boat complaining about "quality problems".
You are in a no win situation. They will tell you your tests are too picky, that no one will use it like that. Unit testing is thankless, you can't argue. Given that there was no test plan, I bet there isn't even a spec! Where there is smoke there is fire.
I'd start looking for a new job right away.
"This mission is too important to allow you to jeopardize it." -- HAL
...By Michael Feathers. The scenario that you may find is "we can't refactor until we have unit tests and we can't have unit tests until we refactor". The book has some strategies for getting around that paradox. You may find, however, that some code is essentially un-testable as written.