Slashdot Mirror
Test Driven Development Examples?
esnyder queries: "I find the pragmatic/agile/XP hype about test driven development compelling, but find it hard to see how to test first (or even unit test at all) in some situations. I would like to explore some extended examples of it in a moderate to large scale real world codebase to improve my test design skills. Can anyone recommend some F/OSS software projects that consistently use test driven development processes that I could check out? Preferable over 50K lines of code, but I'd welcome pointers to anything that people think would be helpful."
Take a look at Zope3 it has some 5,000 unit tests:
http://cvs.zope.org/Zope3/
Free Talk Live: Talk Radio where YOU ar
The Python programming language and all of its standard library modules make extensive use of unit tests and TDD.
It would have been helpful if you'd mentioned the language you're using, and the types of applications (since of course both of those make a huge difference).
If you're using Java for web development, I'd suggest reviewing the Struts Applications Project on SourceForge.
It's a collection of documentation and applications using Struts that are really "done right" -- with documentation, sensible and scalable design, fully implemented testing (unit level AND on the HTTP level). I'm currently roaming through the AppFuse source in there -- it's basically designed to give you a complete setup to start building your app on, with common functions already built-in.
From the site:
AppFuse
An application for starting your Struts-based applications. Checkout from CVS and execute "ant -Dapp.name=yourApp -Ddb.name=database" to instantly be up and running with a Tomcat/MySQL app. Uses Ant, XDoclet, Hibernate, JUnit, Cactus, StrutsTestCase, Canoo's WebTest, Struts Menu, Display Tag Library, JSTL and Struts (including Validator and Tiles). Features include CMA, Remember Me, Self Registration, Password Hint. The fuse to start your apps.
There are only 10 types of people: those who understand decimal, those who don't, and, uh, 8 other types I forget.
I'm far from convinced that TDD is actually a good approach. Although it's pretty obvious that without testing the code is often trivially buggy, and unit testing is the cheapest way to perform testing. For instance this kind of thing is all too easy to do with TDD.
For unit tests you want to write your code, and then look at the best set of unit tests to do complete code coverage. For an OSS e3xample of that you can look at Vstr string library and the code coverage for that project.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
Creating mock objects is much simpler than creating a "text X server", although admitedly a wm is slightly harder than normal as you can't take the easy route of running a seperate version on your main display.
Howewver taking a quick look at blackbox, textPropertyToString() is the only thing in Util.cc that couldn't trivially be unit tested and at least all of i18n.cc and Timer.cc. That's 3% with basically no changes.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
SCons is a next-generation build tool, or make tool, written in Python with strong cross-platform support, integrated autoconf-like functionality, a lot of stable features, and a growing user community. We're currently at 14K+ lines of non-comment, non-blank source code, and 32K+ lines of non-comment, non-blank test code.
We use a combination of two different testing methodologies: 1) Individual modules all have PyUnit unit tests (similar to JUnit, but in Python, of course). 2) The SCons application itself is tested using a custom testing module that manages creation of temporary directories and files, execution of the application, and checking against expected results. This custom module is actually a wrapper around a generic "test any script/command" infrastructure module that could be easily used to test other scripts and/or commands. (The command under test could be implemented in any language, not just in Python.)
I use the Aegis change management system to manage the SCons development and testing cycle. Aegis' primary value add (for me) is its management of the test cases and the testing methodology it enforces. By default, all Aegis changes must have one or more modified or new tests. The new/modified tests must not only pass when run against the new code, but must (by default) fail when run against the old code. This helps guarantee that your tests are good, and that your code isn't passing because you made a mistake in your test and forgot to call the new feature.
By testing in this fashion from day one, we've built up a very strong regression test base--284 test scripts at last count, each script containing multiple individual tests. This test base has become crucial to our ability to refactor (and refactor and refactor...) the internals as we add more features. Sometimes it takes longer, of course, to make a rewrite satisfy all of the regression tests, but when you're done, you can be pretty sure you haven't broken anything. And if you did break something, then you have to add or modify a test when you fix it, and that becomes another part of the regression test base.
The key to getting going with this kind of test-driven development (in my opinion) is making writing and executing the tests as simple as possible (but no simpler!). If writing a test is too difficult, then a lot of developers will simply avoid it. But if you can get them over the initial hump by making it easy to write tests, it gets downright addictive because you get all of this positive feedback when your tests show you that your new code works.
We'd be glad to have you check out the testing infrastructure we've developed for SCons, either for code you can actually use, or simply as a source of ideas. Feel free to contact the SCons development team if you have any questions.
#1: My biggest problems always revolve around the user interface.
But what would you rather debug: 1) A problem in the user interface you know has to be in the user interface because you have confidence that it's not below it or 2) the whole thing all the way down all of the method calls? If you have unit tests in the rest of the code, debugging the UI is much easier.
#2: I spent many hours developing tests for things that ended up rarely/never choking my tests....
Making tests for situations that work is still valuable. If you make such a test and it fails later on, you'll know you had a regression in the code somewhere. This is very handy to have when refactoring as well.
#3: Whenever I re-factor, unless I'm Mr. Perfect and all my interfaces are perfect from the get-go...
Interfaces can be refactored too, and of course you'll break tests and have to rewrite things because now you're expecting a new set of behaviour.
I've found that this has the side benefit of forcing people to really think about changes they make to their code and not just hack away at it. If people know they have to write tests too they seem to put more thought into it ahead of time.
Anyway, that's just my personal experience with unit testing.
----- rL
And check out the Boost unit test libraries. http://boost.org/libs/test/doc/index.html
Try the book called "Test Driven Development: By Example" by Kent Beck!
.. using the framework he is writing to write the framework! Here he shows you how to bootstrap a new testing framework, which you probably won't ever need to do, but it was cool. He starts with a single test that prints "1" or "0" on the screen for pass/fail, and builds on that.
It's a neat little book. The first third demonstrates writing a Money class in Java using test-driven development. It's kinda like you're sitting next to him and he is demonstrating. In that example he shows a lot of what comes up in an average TDD session. If you're new to it, you might not realize how fluid it is, and how tiny the steps can be. Even if you *think* you know what TDD is about from reading a description, this will really show you how it works.
Then the next chapter is really cool.. here he writes a unit test framework in Python
Final third of the book is patterns, example, theory, philosphy, somewhat interesting.
This is a book you'll only read once or twice, honestly, but you will benefit greatly from it. In my opinion.
Software Development Magazine has been running a series for about the past year called The Craftsman written by Robert C. Martin. It focuses on a young apprentice writing a java app with his mentors. Nothing is done that doesn't involve a test first.
While the series itself is kind of slow, it is a good introduction to TDD, and I really enjoy Robert's writing style. Might be able to lead you to some more examples.
Random Musings
It's not *exactly* test driven, but gcc has a very extensive test suite, and adds regression tests for every new bug that is uncovered. The project certainly passes your 50kloc specification!
http://gcc.gnu.org/install/test.html
I bought this book, and it was really useful in the sense of basic training, but it does not tackle any of the difficult questions, such as mock objects, user interfaces, etc.
I am left still wanting a nice non-trivial working example to look at.
Most writers regard truth as their most valuable possession, and therefore are most economical in its use - Mark Twain
First, IMO, html is the UI ... and the best way to test that isn't with a unit test (that I've seen). You want all your UI functions to just glue sql/data commands to the UI. Then you can "easily" test the sql/data commands with a test database and a bunch of function calls ... then you'll know that any bugs you find in the html part must be the fault of the UI code (which should be fairly small).
As far as the testing DB, I'd recommend having testing data outside the DB and use something to init. the database just before you run your unit tests. This should be easier to keep upto date, and solves problems of testing alterations to the DB.
Having an external representation of the DB that can be reloaded should solve this problem.
The only DB thing I've seen is for Java that was on testdriven.com.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B