Slashdot Mirror
Test Driven Development Examples?
esnyder queries: "I find the pragmatic/agile/XP hype about test driven development compelling, but find it hard to see how to test first (or even unit test at all) in some situations. I would like to explore some extended examples of it in a moderate to large scale real world codebase to improve my test design skills. Can anyone recommend some F/OSS software projects that consistently use test driven development processes that I could check out? Preferable over 50K lines of code, but I'd welcome pointers to anything that people think would be helpful."
Nothing reality tests the usability of a proposed API then writing unit tests against it.
For those who develop in Java, may I propose JUnit? If you want to test the GUI of a web server, then try HTTPUnit although the value of writing unit tests to this is less since GUI is usually subject to a lot of changes over time.
Take a look at Zope3 it has some 5,000 unit tests:
http://cvs.zope.org/Zope3/
Free Talk Live: Talk Radio where YOU ar
Where I work we're coding a dating site. Currently, it's up to about 180K lines of scripts written in PHP. However, not a lot of it was written with unit testing (and test-driven development) in mind. I'm a big believer in unit testing but it's been hard to get the other members on board as they complain how hard it is to write unit tests for web pages. Bah.
So, most of the library functions that I wrote (stuff like except an integer, return a text string from a list) have been unit tested by myself, and every time I change a function or a class, I try and write a unit test for it.
Seriously, you just need to dive right in and think about how you make your code easy to test. I use the SimpleTest testing framework (it's PHP), and I always feel good when my array of tests all run correctly when I make a major change to the code that impacts a huge portion of the site.
If test-driven development has done one thing, it's forced me to carefully examine my code to create a way to make sure it is actually working according to the business logic we've been asked to implement.
None of the projects I have looked at appear to have test first design. Some (especially CPAN items) have good testing of the functionality in place, but these are all modules inside a programming language. I have never seen a full on stand alone project with such testing built into the core.
I find that odd really, considering that the point behind "write tests first" is to create an executable specification for functionality. If a new piece is desired, surely it would be easier for someone else to pick up the module with the tests and write code to pass the tests than it is to try to pick up the flow of a project from out of date documents, etc, and just contribute. I look forward to those who know of such projects, because I plan to examine them for help on how to implement these techniques myself. (My current project uses regression tests driven by an external test application... I have had this same question for some time, especially in regards to web applications).
Sig under construction since 1998.
The Python programming language and all of its standard library modules make extensive use of unit tests and TDD.
It would have been helpful if you'd mentioned the language you're using, and the types of applications (since of course both of those make a huge difference).
If you're using Java for web development, I'd suggest reviewing the Struts Applications Project on SourceForge.
It's a collection of documentation and applications using Struts that are really "done right" -- with documentation, sensible and scalable design, fully implemented testing (unit level AND on the HTTP level). I'm currently roaming through the AppFuse source in there -- it's basically designed to give you a complete setup to start building your app on, with common functions already built-in.
From the site:
AppFuse
An application for starting your Struts-based applications. Checkout from CVS and execute "ant -Dapp.name=yourApp -Ddb.name=database" to instantly be up and running with a Tomcat/MySQL app. Uses Ant, XDoclet, Hibernate, JUnit, Cactus, StrutsTestCase, Canoo's WebTest, Struts Menu, Display Tag Library, JSTL and Struts (including Validator and Tiles). Features include CMA, Remember Me, Self Registration, Password Hint. The fuse to start your apps.
There are only 10 types of people: those who understand decimal, those who don't, and, uh, 8 other types I forget.
I'm far from convinced that TDD is actually a good approach. Although it's pretty obvious that without testing the code is often trivially buggy, and unit testing is the cheapest way to perform testing. For instance this kind of thing is all too easy to do with TDD.
For unit tests you want to write your code, and then look at the best set of unit tests to do complete code coverage. For an OSS e3xample of that you can look at Vstr string library and the code coverage for that project.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
Unit testing my code has saved me hundreds of hours of debugging time, unit tests are also great documentation. In a big system where you can't keep track of all the code and how it works in your head, unit tests act as little documentation beacons showing how some piece of code works.
Free Talk Live: Talk Radio where YOU ar
I wrote a series of tests for code I have written for what I would term a trivial application. Took me about 3 months to develop. The series of tests for about 50% of the code and missed 50%. The vast majority of the bugs came from the second half of the code.
Second. The tests themselves act as documentation. I am anoyed that I pick up generic routines like a CSV file reader and there is subroutines but no way to figure out how to use them. A test suite acts as sample code as well.
This process is balance, you don't test every detail but you do codify some tests. When you find a bug you codify a test and then you NEVER reintroduce the same bug. I find that in general I reintroduce bugs about once for every month of coding. Since this application only gets worked on in random blocks of a week here and there I am not focussed on the whole application and I simple forget. Use the computer to validate that you have not made a mistake.
These sort of tests saves my reputation, does not frustrate my users.
Sorry to let you know but, you didn't write good unit tests and probably did waste your time. I've found very close to 100% of the bugs in Vstr a network IO string library using unit tests. That includes a couple of ones that would have been damn hard to track down otherwise.
However it's been over a year since 1.0.0 which had a unit test for every function and every function option, to the last release which had over 99% code coverage found a couple of weird corner case issues (not just bugs, but optimizations that could never be reached for some reason). And going from 98% coverage to 99% coverage took a significant time investment, and required significant thinking about how the test should be written.
As with much software development, it's easy to write simple tests that don't show much and aren't very useful. It's much harder to write tests that find bugs (and you have to appraoch writing the tests with a very different mindset to how you approach writting the code you are testing. This is not even close to being "Like picking lint from your belly-button."
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
I'm sure a lot of people have grounds to object, but I suggest, depending on the nature of the project, it's a good idea to be intelligently selective about what does and does not need to be unit tested. Consider:
::sigh::
Do write a test case if:
+ a failure could introduce data propagation issues
+ it performs some intricate mathematical or logical function whose result must be precise
+ you're writing test cases to hunt down bugs that you know are in you're code; keep those test cases
+ you're uncomfortable about the quality of your code
+ an error might kill someone, or otherwise be Really Bad
Don't bother writing a test case if:
- you can use a guard clause or assertion instead
- a failure in the code will otherwise be immediately obvious
- the code generates massive amounts of data which need not be mathematically precise (i.e. graphical output)
- you don't feel it
I should probably write more test cases, according to my own guidelines.
Upstairs Dog, Downstairs People.
My experience has been that "unit testing" slows development to a painfully slow crawl.
This is a common miconception about test-driven development. Sure, when you look at the speed at which new features go in, "development" seems to be slowing to a crawl.
But this is a very short-sighted and limited view of the whole project lifecycle. I've found that having great unit tests for my code actually help in the long run for adding new features. And that's not just adding the features to the code base, that's all the way to the end -- making sure they work properly.
Nevermind the benefit they give while refactoring in an agile development process: the unit tests become a sort contract you can guarantee will work for objects that use your code, no matter the internals of a function. For refactoring this is invaluable.
In short, you're not alone. Very few developers think long term like this and it's a shame. But once you drink the unit testing/test driven design (TDD) kool-aid I guarantee you'll never go back. I wouldn't think of writing code without unit tests any more and quite often it's TDD.
----- rL
The first wow, that's cool moment I had with test driven development was from an article on Object Mentor called The Bowling Game. It also highlights pair programming a bit.
The unique thing about the article was that it was presented as a discussion between two developers pair programming doing agile test driven development of the game. It was like watching over their shoulders.
If you want to get an idea of what extreme programming is like, I suggest reading this article AND writing the code and tests along with it, either in Java (and JUnit) or C#/VB.NET (and NUnit) or another language with a xUnit unit testing framework. Most object oriented languages have them now so you don't have to roll your own framework.
----- rL
Where I work, unit tests are a requirement, not a luxury. WIthout fail, the projects with the most unit tests have the fewest bugs, and the projects with the fewest unit tests have the most bugs. I'm not exagerating.
My question to you is, if you have no unit tests, how can you be sure that the code works at all? Manual testing? Do you cover all the code paths with manual testing? It must take forever, or your application is rather trivial.
SCons is a next-generation build tool, or make tool, written in Python with strong cross-platform support, integrated autoconf-like functionality, a lot of stable features, and a growing user community. We're currently at 14K+ lines of non-comment, non-blank source code, and 32K+ lines of non-comment, non-blank test code.
We use a combination of two different testing methodologies: 1) Individual modules all have PyUnit unit tests (similar to JUnit, but in Python, of course). 2) The SCons application itself is tested using a custom testing module that manages creation of temporary directories and files, execution of the application, and checking against expected results. This custom module is actually a wrapper around a generic "test any script/command" infrastructure module that could be easily used to test other scripts and/or commands. (The command under test could be implemented in any language, not just in Python.)
I use the Aegis change management system to manage the SCons development and testing cycle. Aegis' primary value add (for me) is its management of the test cases and the testing methodology it enforces. By default, all Aegis changes must have one or more modified or new tests. The new/modified tests must not only pass when run against the new code, but must (by default) fail when run against the old code. This helps guarantee that your tests are good, and that your code isn't passing because you made a mistake in your test and forgot to call the new feature.
By testing in this fashion from day one, we've built up a very strong regression test base--284 test scripts at last count, each script containing multiple individual tests. This test base has become crucial to our ability to refactor (and refactor and refactor...) the internals as we add more features. Sometimes it takes longer, of course, to make a rewrite satisfy all of the regression tests, but when you're done, you can be pretty sure you haven't broken anything. And if you did break something, then you have to add or modify a test when you fix it, and that becomes another part of the regression test base.
The key to getting going with this kind of test-driven development (in my opinion) is making writing and executing the tests as simple as possible (but no simpler!). If writing a test is too difficult, then a lot of developers will simply avoid it. But if you can get them over the initial hump by making it easy to write tests, it gets downright addictive because you get all of this positive feedback when your tests show you that your new code works.
We'd be glad to have you check out the testing infrastructure we've developed for SCons, either for code you can actually use, or simply as a source of ideas. Feel free to contact the SCons development team if you have any questions.
#1: My biggest problems always revolve around the user interface.
But what would you rather debug: 1) A problem in the user interface you know has to be in the user interface because you have confidence that it's not below it or 2) the whole thing all the way down all of the method calls? If you have unit tests in the rest of the code, debugging the UI is much easier.
#2: I spent many hours developing tests for things that ended up rarely/never choking my tests....
Making tests for situations that work is still valuable. If you make such a test and it fails later on, you'll know you had a regression in the code somewhere. This is very handy to have when refactoring as well.
#3: Whenever I re-factor, unless I'm Mr. Perfect and all my interfaces are perfect from the get-go...
Interfaces can be refactored too, and of course you'll break tests and have to rewrite things because now you're expecting a new set of behaviour.
I've found that this has the side benefit of forcing people to really think about changes they make to their code and not just hack away at it. If people know they have to write tests too they seem to put more thought into it ahead of time.
Anyway, that's just my personal experience with unit testing.
----- rL
Try the book called "Test Driven Development: By Example" by Kent Beck!
.. using the framework he is writing to write the framework! Here he shows you how to bootstrap a new testing framework, which you probably won't ever need to do, but it was cool. He starts with a single test that prints "1" or "0" on the screen for pass/fail, and builds on that.
It's a neat little book. The first third demonstrates writing a Money class in Java using test-driven development. It's kinda like you're sitting next to him and he is demonstrating. In that example he shows a lot of what comes up in an average TDD session. If you're new to it, you might not realize how fluid it is, and how tiny the steps can be. Even if you *think* you know what TDD is about from reading a description, this will really show you how it works.
Then the next chapter is really cool.. here he writes a unit test framework in Python
Final third of the book is patterns, example, theory, philosphy, somewhat interesting.
This is a book you'll only read once or twice, honestly, but you will benefit greatly from it. In my opinion.