Slashdot Mirror
Why We Refactored JUnit
Bill Venners writes "In this article, three programmers tell the story of how their frustration with JUnit's API led to the creation of Artima SuiteRunner, a free, open source test toolkit and JUnit runner. These programmers simply wanted to create a small add-on tool to JUnit, but found JUnit's design non-intuitive and API documention poor. After spending time reading through JUnit's source code and attempting to guess at the API contracts, they gave up and rewrote it."
They are ... see ReactOS
Since until now JUnit really was the only game in town for java test-writing, I'm so impressed to see these guys put out something that's still compatible with it!! Even if it was frustration with JUnit that gave them the inspiration. All of us have a bunch of JUnit tests for our code (ok, well SOME of us do), and it's nice to have the option to try out another framework without having to refactor our tests.
Really, in the world of open source, and free software way too little attention is paid to compatibility. Why not be compatible with your "competition"? It's not like you're competing for customers or market share, or any of that crap. We're all on the same team!
It's called XMLUnit (Google it in 2 seconds). Works great.
Code refactoring is modifying code while preserving the same overall behavior; I suppose if you keep at it eventually all the original code will be gone. Design refactoring is presumably modifying design while preserving the same end results....
The other guy was Erich Gamma, of Design Patterns fame.
That said, I don't know if artima has really contributed anything new here. Your IDE likely has a JUnit test runner built into it already (IntelliJ, JBuilder, and NetBeans all do). Ant also already has decent junit test and report targets, which basically include all of the capabilities artima has implemented. Another stand alone test runner is probably not all that useful for day to day development.
At that point [Fowler] joined forces with Beck and formed his second reason for being well known, XP.
Martin Fowler did not invent XP. It originated in the work of Ward Cunningham and Kent Beck, then was refined by Kent Beck with help from Ron Jeffries and other members of the original XP team. Martin Fowler is an active part of that community, so he co-authored one of the XP books.
I am a spec writer, I don't play one on TV. I authored the EAI for VRML97 and the various programming APIs for X3D. These are ISO specs, so they come with a lot of weight behind them from the conformance perspective.
As someone who is involved in specification writing of APIs I can tell you that a "compile test" is wholly insufficient for checking class/method signatures. (for this reply, I'm using method as being interchangable for any message passing system - C functions, Java methods, RPC calls whatever)
The first and major problem is not that the methods exist - everyone can cut and paste the spec document - but ensuring that nothing else exists. The bane of every spec writer is the company/individual that decides that they don't like the spec and then proceeds to extend it with their own methods within the same class - often overloading existing method calls. It is these extensions that a spec writer dreads because the (often) clueless end user makes use of them and then wonders WTF their code won't run on another "conformant" implementations of that spec.
Checking signatures is there for one thing and one thing only - making sure the implementors don't embrace and extend the specification in a way that is not permitted by the spec. Creating derived interfaces with new method signatures is fine, adding new method signatures to the specification-defined collection is not. A good conformance test suite will look at everything about the signatures to make sure everything is there, and nothing that should not be is not there.
Life is complete only for brief intervals in between toys or projects -- John Dalton
I also found that JUnit's documentation was poor, but it wasn't just that. It didn't seem to be designed as a user-friendly API, where the user is a programmer trying to use it as an API. As I wrote in the article, I really liked using JUnit as an application, once I got it going. It was when I went in to integrate my signature test tool with JUnit as an API that I really ran into trouble. The API documentation was very sparse and frustrating. I started looking at the code, which was confusing not just because of poor documentation, but of non-intuitive (to me, anyway) organization.
I was able to figure out what the code itself was doing, but then I was left guessing at what the contracts of the types and the methods were. When you decipher code, you understand one implementation of an interface, not the abstract contract of the interface. I got so mad at it that I opened IntelliJ and started writing my own testing toolkit. I thought it would be fairly easy, but I was wrong. I underestimated how much functionality JUnit actually has in it. And although I think we did come up with a much simpler public API, it took a lot of work to simplify it. So I ended up with more appreciation of all the work that went into JUnit.
I suspect that a lot of JUnit's bulkiness comes from the fact that JUnit started with Java 1.0 or 1.1 and evolved over time in wide-spread public use. Because it was so popular, any design blemishes couldn't be erased in later releases. Eventually enough such blemishes accumlate and it makes sense to start over.
That assumes that the rate of the cost of change rises over time. XP rejects that assumption, and the XP practices are designed to keep the rate of the cost of change consistent.
how to invest, a novice's guide
It's Larry Rosen's Open Software License (OSL), not ours. I think it is a great license. I liked that they mention trademarks, for example. The main thing I didn't really like about it was the requirement that derivative works also be released under the OSL. I would have been OK with derivative works being proprietary. But I liked the OSL so much, compared to the many other existing open source licenses, that I went with the OSL.
The reason I have people register is to capture email addresses of users. Even though SuiteRunner is free and open source, I consider it a product, and I want to be able to contact users. If you don't want to register, just go to sourceforge and look up the suiterunner project. You can download it there anonymously. Or if you register but don't want to be emailed ever, just uncheck the two newsletter checkboxes.
A contract for a method in an interface doesn't say what the method does, it draws a fence around the full range of possible things that method could do. So for example, Collection.add method has the contract:
That leaves room for an implementation class, such as HashSet, to reject objects passed to add that already exist in the set. The Collection.add contract is abstract. It draws a fence around the full range of legal behavior for the add method. HashSet.add is a concrete implementation that is like a single point inside the Collection.add fence.
Imagine stumbling upon the code to HashSet.add without the benefit of the detailed contract-oriented API documentation of Collection.add. You want to implement an implementation of Collection.add in a different way, but all you know is the single point represented by the HashSet implementation. You have to guess where the fence is.
That was the real trouble I had trying to integrate with JUnit as an API. I spent time attempting to decipher the JUnit source code and understood fairly well what it was doing. I made educated guesses at the contracts, but given the non-intuitive (to me) nature of the JUnit API, I wasn't confident in my guesses. I could have integrated my tool and tested it with JUnit 3.6 or 3.7, but had little faith that JUnit 3.8 would not prove my guesses wrong and break my tool.
As I mentioned elsewhere in this topic, I ask for registration so that I know who the users are. Even though SuiteRunner is free and open source, I consider it a product. If you check the Artima Newsletter checkbox, you'll be notified of updates to SuiteRunner as well as new articles on Artima.com.
Also, for several reasons we decided to create a web-based discussion forum for SuiteRunner user support rather than using a mailing list. The web forum (I fear this will add weight to your conspiracy theory, but the forum is based on Jive.) requires that you have an account, so by getting an account when you download SuiteRunner, you are ready to post to the SuiteRunner users forum.
On top of that, shouldn't I get points for requiring such a minute amount of information to register? Many times I have been confronted with a long scrolling page full of edit boxes I'm required to fill in. At Artima.com, all I ask for is a name, which you can fake, a nickname, which you can make up, and an email address that I confirm. The name and nickname shows up on forum posts. If you watch a forum topic you get notified by email when someone posts a reply, which is why I confirm the email address even if you opt out of the newsletter.
Interesting that you found the site "flashy." I was going for simple. If you look carefully, you'll see the site is primarily text with a bare minimum of graphics.
Nevertheless, although I incorporated Artima Software a couple years ago, it is still just one person (me). So what you really need to ask yourself is not whether you trust a faceless corporation with your email address, but whether you trust me with it. I assure you I am very concerned about privacy.
I have another one,
it is geared towards tests with multithreaded environment.
here is the project page
http://www.michaelmoser.org/jmtunit/
The library sets up a thread pool (n threads) and runs a number of identical test sequences; each test sequence is a virtual user (m users). All threads are waiting on one input queue for test requests. The threading modell is the same as in most of the popular application servers
The programming interface is very similar to JUnit - there is a JMTUnit.TestCase with some additions, there is JMTUnit.TestResult.
The tool does print out performance statistics
Unit testing databases can be simplified by using DBUnit.
With DBUnit, you can describe your database in xml, and each time the tests are run, dbunit creates a new test database starting from the xml files. So you always test on a clean database. Check it out!
Kent Beck - co-author of JUnit - recently published "Test-Driven Development", a book about how you can change your coding habits by creating the tests up front.
One of the most useful sections is a complete walk-through of the creation of unit-testing tool for Python - it's probably the best way of understanding the internals of the JUnit framework without trying to reconstruct every single line of code...
It's all very well in practice, but it will never work in theory.
That's a good question. Several of the other XP practices work in your favor in that case.
First, if you've been doing the simplest thing that could possibly work, your code is already pretty simple -- it doesn't have superfluous stuff to work around. Second, if you're testing continually, you have a safety net to notify you if you accidentally break the essential behavior. Third, if you've been refactoring continually, the code will be decoupled and much easier to change. Aggressive testing helps this too. Finally, if you're using the planning game to work on half-day features, you're dealing with manageable chunks of code that very rarely touch more than a couple of individual units.
There are times when you have to schedule a couple of days for larger refactorings, but they're not common. If you have intelligent developers who keep up the good practices, the code ends up really clean, really easy to extend, and surprisingly well-designed.
how to invest, a novice's guide