Slashdot Mirror
Why We Refactored JUnit
Bill Venners writes "In this article, three programmers tell the story of how their frustration with JUnit's API led to the creation of Artima SuiteRunner, a free, open source test toolkit and JUnit runner. These programmers simply wanted to create a small add-on tool to JUnit, but found JUnit's design non-intuitive and API documention poor. After spending time reading through JUnit's source code and attempting to guess at the API contracts, they gave up and rewrote it."
I could refactor my unit.
why does this sound like something of the beginning of a fantasy story about programming: "in the beginning was Iluvitar, the master of the JU API........then melkor, wishing to expand it...."
xao
xao
http://TheHillforum.hopto.org
I find this quite amusing. I recently attended a talk in Montreal given by Kent Beck, one of the X-programming icons. One of the key elements of XP, as I understood his talk, is cutting down what he calls excess 'inventory' which include things such as excess documentation, architecture, etc. By keeping these iventory elements to a minimum, you get quicker feedback, because you have quicker results, which feeds back into the process and allows you to respond quickly to changing requirments.
Well, it also happens that JUnit was developed al la XP by Kent and one other guy (didn't pay attention to who, go look it up!) For a while I was thinking that this XP thing might actually be something, but after skimming through the first 5 chapeters of 'Planning XP' coupled with the statements concerning the JUnit API, I'm starting to think that XP really is just one big hot air balloon.
In his defense though, he did say that refactoring often was a GOOD idea. It's just that he didn't say that you should wait for someone else to do it for you.
My 2 cents.
All your base are belong to us!
My big challenges are (1) existing code that wasn't designed for unit testing, (2) testing with databases, (3) testing that involves user interaction and (4) testing that involves multi-tasking. I like unit testing, but it is hard to apply to every task.
Is intuitive design necessarily good in programming.?
Recursion is intuitive but not necessarily efficient in terms of performance.
At what point do we decide which is better intuitive design or efficient design.
I'm sure you've also heard of a guy named Fowler... Martin Fowler is well known in the community for exactly two reasons.
The primary one is his book Refactoring. It describes his experience as a consultant refactoring medium sized software projects, and gives a lot of advice on methods for refactoring.
Apparently Fowler decided that refactoring is a good thing. Not just that, he decided that since refactoring is a good thing, and so should be the basis for programming, since most of the results of programming are bad. By that I mean just that most programs suck, not that they are evil or anything.
At that point he joined forces with Beck and formed his second reason for being well known, XP. As far as I can tell, the philosophy of XP is, "Software usually ends up sucking and in need of refactoring after it has been extended too far. Why wait for it to be extended too far? Just make it suck in the first place and refactor all the time!"
It does have a sort of perverse logic to it, but when I say that XP is bad, I don't just mean that it sucks. As well as sucking, XP really is evil.
We made SuiteRunner a JUnit runner as an afterthought, when it dawned on us that no JUnit user would use SuiteRunner unless it added value to their existing JUnit test suites.
One of the things we did originally was make "test" an abstract notion. Reporters can indicate a test is starting, succeeded, failed, or aborted. When it came time to turn SuiteRunner into a JUnit runner, we were able to just report JUnit test results as another kind of test. We felt the ease with which we could turn SuiteRunner into a JUnit runner was a validation of our abstract test notion.
I am a spec writer, I don't play one on TV. I authored the EAI for VRML97 and the various programming APIs for X3D. These are ISO specs, so they come with a lot of weight behind them from the conformance perspective.
As someone who is involved in specification writing of APIs I can tell you that a "compile test" is wholly insufficient for checking class/method signatures. (for this reply, I'm using method as being interchangable for any message passing system - C functions, Java methods, RPC calls whatever)
The first and major problem is not that the methods exist - everyone can cut and paste the spec document - but ensuring that nothing else exists. The bane of every spec writer is the company/individual that decides that they don't like the spec and then proceeds to extend it with their own methods within the same class - often overloading existing method calls. It is these extensions that a spec writer dreads because the (often) clueless end user makes use of them and then wonders WTF their code won't run on another "conformant" implementations of that spec.
Checking signatures is there for one thing and one thing only - making sure the implementors don't embrace and extend the specification in a way that is not permitted by the spec. Creating derived interfaces with new method signatures is fine, adding new method signatures to the specification-defined collection is not. A good conformance test suite will look at everything about the signatures to make sure everything is there, and nothing that should not be is not there.
Life is complete only for brief intervals in between toys or projects -- John Dalton
I believe that this quote is true for commercial software. The word processors, email clients and web browsers in the world should not be designed at the beginning of a project. The cost of doing that far outweighs the benifits. But...
When you are designing life critical applications (fly-by-wire, ABS, pacemaker, cancer radiation machine...etc) it is not acceptable to be redesigning while coding.
If you cannot model the way a system must react at all times, under any input, then what makes you think that your software should be responsible for someones life.
More and more software is being designed for life critical applications. If XP is being used to develop them, I'm worried. You cannot stumble onto 100% correct software. In a large system it takes a lot of money - an exponential curve, the last 3 bugs will likely take more money than the first 100 - time and a good design.
To help answer the grandparent's question: compile != matches API.
Can you write a test-compile program that checks whether the proper public/private/proctected access modifier keywords are set up right on each method? That the method was specified for 'short foo' but the implementor used 'int foo'? Or like Mithrandir said, that no additional methods have been added (such as an overloaded version)?
Basically, compile-compatability is not the same as using the precise APIs in the spec.
Ah, you do not understand the zen of XP yet. First you write the test. Then you write the code. Then you correct the code so the test passes. The sun is warm; the grass is green.
I interviewed Martin Fowler, author of Refactoring, last year. In this interview I asked him to define refactoring. He said:
We didn't refactor JUnit's code, because we didn't start with JUnit's code and make changes. But we did do what I consider a "refactor" of the design. We started in effect with JUnit's API, and made changes. We started with knowledge of what we liked and didn't like about JUnit's API, and refactored that.
Where you can see the refactoring is not in the code, but in the JavaDoc. Compare JUnit's JavaDoc with SuiteRunner's JavaDoc. I'm guessing it was version 3.7 of JUnit that I had in front of me when I decided I would start over. It may have been 3.6.
nonono ... you correct the test so the code passes ...