Slashdot Mirror
The Art of Unit Testing
FrazzledDad writes "'We let the tests we wrote do more harm than good.' That snippet from the preface of Roy Osherove's The Art of Unit Testing with Examples in .NET (AOUT hereafter) is the wrap up of a frank description of a failed project Osherove was part of. The goal of AOUT is teaching you great approaches to unit testing so you won't run into similar failures on your own projects." Keep reading for the rest of FrazzledDad's review.
The Art of Unit Testing with Examples in .NET
author
Roy Osherove
pages
296
publisher
Manning
rating
9/10
reviewer
FrazzledDad
ISBN
1933988274
summary
Soup-to-nuts unit testing with examples in .NET
AOUT is a well-written, concise book walking readers through many different aspects of unit testing. Osherove's book has something for all readers, regardless of their experience with unit testing. While the book's primary focus is .NET, the concepts apply to many different platforms, and Osherove also covers a few Java tools as well.
Osherove has a long history of advocating testing in the .NET space. He's blogged about it extensively, speaks at many international conferences, and leads a large number of Agile and testing classes. He's also the chief architect at TypeMock, an isolation framework that's a tool you may make use of in your testing efforts – and he's very up front about his involvement with that tool when discussing isolation techniques in the book. He does a very good job of not pushing his specific tool and also covers several others, leaving me feeling there wasn't any bias toward his product whatsoever.
AOUT does a number of different things really, really well. First off, it focuses solely on unit testing. Early on Osherove lays out the differences between unit and integration tests, but he quickly moves past that and stays with unit tests the rest of the book. Secondly, Osherove avoids pushing any particular methodology (Test Driven Development, Behavior Driven Development, etc.) and just stays on critical concepts around unit testing.
I particularly appreciated that latter point. While I'm a proponent of *DD, it was nice to read through the book without having to filter out any particular dogma biases. I think that mindset makes this book much more approachable and useful to a broader audience – dive in to unit testing and learn the fundamentals before moving on to the next step.
I also enjoyed that Osherove carries one example project through the entire book. He takes readers through a journey as he builds a log analyzer and uses that application to drive discussion of specific testing techniques. There are other examples used in the book, but they're all specific to certain situations; the brunt of his discussion remains on the one project which helps keep readers focused in the concepts Osherove's laying out.
The book's first two chapters are the obligatory introduction to unit testing frameworks and concepts. Osherove quickly moves through discussions of "good" unit tests, offers up a few paragraphs on TDD, and lays out a few bits around unit test frameworks in general. After that he's straight in to his "Core Techniques" section where he discusses stubs, mocks, and isolation frameworks. The third part, "The Test Code" covers hierarchies and pillars of good testing. The book finishes with "Design and Process" which hits on getting testing solidly integrated into your organization, plus has a great section on trying to deal with testing legacy systems. There are a couple handy appendices covering design issues and tooling.
Osherove uses his "Core Techniques" section to clearly lay out the differences between stubs and mocks, plus he covers using isolation frameworks such as Rhino.Mocks or TypeMock to assist with implementing these concepts. I enjoyed reading this section because too many folks confuse the concepts of stubbing and mocking. They're not interchangeable, and Osherove does a great job emphasizing where you should use stubs and mocks to deal with dependencies and interactions, respectively.
The walkthrough of splitting out a dependency and using a stub is a perfect example of why this book's so valuable: Osherove clearly steps through pulling the dependency out to an interface, then shows you different methods of using a stub for testing via injection by constructors, properties, or method parameters. He's also very clear about the drawbacks of each approach, something I find critical in any design-related discussion – let me know what things might cause me grief later on!
While the discussion on mocking, stubbing, and isolation was informative and well-written, I got the most out of chapters 6 ("Test hierarchies and organization") and 7 ("The pillars of good tests"). The hierarchy discussion in particular caused me to re-think how I've been organizing an evolving suite of Selenium-based UI tests. I was already making use of DRY and refactoring out common functionality into factory and helper methods; however, Osherove's discussion led to me re-evaluating the overall structure, resulting in some careful use of base class and inheritance. His concrete examples of building out a usable test API for your environment also changed how I was handling namespaces and general naming.
If you're in an organization that's new to testing, or if you're trying to deal with getting testing around legacy software, then the last two chapters of the book are must-read sections. Changing cultures inside organizations is never easy, and Osherove shows a number of different tools you can use when trying to drive the adoption of testing in your organizations. My own experience has shown you'll need to use combinations of many of these including finding champions, getting management buy off, and most importantly learning how to deal with the folks who become roadblocks.
The Art of Unit Testing does a lot of things really well. I didn't feel the book did anything poorly, and I happily include it in my list of top software engineering/craftsmanship books I've read. All software developers, regardless of their experience with unit testing, stand to learn something from it.
You can purchase The Art of Unit Testing with Examples in .NET from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
Osherove has a long history of advocating testing in the .NET space. He's blogged about it extensively, speaks at many international conferences, and leads a large number of Agile and testing classes. He's also the chief architect at TypeMock, an isolation framework that's a tool you may make use of in your testing efforts – and he's very up front about his involvement with that tool when discussing isolation techniques in the book. He does a very good job of not pushing his specific tool and also covers several others, leaving me feeling there wasn't any bias toward his product whatsoever.
AOUT does a number of different things really, really well. First off, it focuses solely on unit testing. Early on Osherove lays out the differences between unit and integration tests, but he quickly moves past that and stays with unit tests the rest of the book. Secondly, Osherove avoids pushing any particular methodology (Test Driven Development, Behavior Driven Development, etc.) and just stays on critical concepts around unit testing.
I particularly appreciated that latter point. While I'm a proponent of *DD, it was nice to read through the book without having to filter out any particular dogma biases. I think that mindset makes this book much more approachable and useful to a broader audience – dive in to unit testing and learn the fundamentals before moving on to the next step.
I also enjoyed that Osherove carries one example project through the entire book. He takes readers through a journey as he builds a log analyzer and uses that application to drive discussion of specific testing techniques. There are other examples used in the book, but they're all specific to certain situations; the brunt of his discussion remains on the one project which helps keep readers focused in the concepts Osherove's laying out.
The book's first two chapters are the obligatory introduction to unit testing frameworks and concepts. Osherove quickly moves through discussions of "good" unit tests, offers up a few paragraphs on TDD, and lays out a few bits around unit test frameworks in general. After that he's straight in to his "Core Techniques" section where he discusses stubs, mocks, and isolation frameworks. The third part, "The Test Code" covers hierarchies and pillars of good testing. The book finishes with "Design and Process" which hits on getting testing solidly integrated into your organization, plus has a great section on trying to deal with testing legacy systems. There are a couple handy appendices covering design issues and tooling.
Osherove uses his "Core Techniques" section to clearly lay out the differences between stubs and mocks, plus he covers using isolation frameworks such as Rhino.Mocks or TypeMock to assist with implementing these concepts. I enjoyed reading this section because too many folks confuse the concepts of stubbing and mocking. They're not interchangeable, and Osherove does a great job emphasizing where you should use stubs and mocks to deal with dependencies and interactions, respectively.
The walkthrough of splitting out a dependency and using a stub is a perfect example of why this book's so valuable: Osherove clearly steps through pulling the dependency out to an interface, then shows you different methods of using a stub for testing via injection by constructors, properties, or method parameters. He's also very clear about the drawbacks of each approach, something I find critical in any design-related discussion – let me know what things might cause me grief later on!
While the discussion on mocking, stubbing, and isolation was informative and well-written, I got the most out of chapters 6 ("Test hierarchies and organization") and 7 ("The pillars of good tests"). The hierarchy discussion in particular caused me to re-think how I've been organizing an evolving suite of Selenium-based UI tests. I was already making use of DRY and refactoring out common functionality into factory and helper methods; however, Osherove's discussion led to me re-evaluating the overall structure, resulting in some careful use of base class and inheritance. His concrete examples of building out a usable test API for your environment also changed how I was handling namespaces and general naming.
If you're in an organization that's new to testing, or if you're trying to deal with getting testing around legacy software, then the last two chapters of the book are must-read sections. Changing cultures inside organizations is never easy, and Osherove shows a number of different tools you can use when trying to drive the adoption of testing in your organizations. My own experience has shown you'll need to use combinations of many of these including finding champions, getting management buy off, and most importantly learning how to deal with the folks who become roadblocks.
The Art of Unit Testing does a lot of things really well. I didn't feel the book did anything poorly, and I happily include it in my list of top software engineering/craftsmanship books I've read. All software developers, regardless of their experience with unit testing, stand to learn something from it.
You can purchase The Art of Unit Testing with Examples in .NET from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
For anyone familiar with the basics of unit testing but struggling to implement it in real world scenarios, I'd strongly recommend xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros.
The idea is not only that automated testing is good, but that testable code is fundamentally better because it needs to be loosely coupled. I still struggle to follow TDD in many scenarios, especially where I'm closely interacting with system APIs, but just reading xUnit Test Patterns has given me tons of ideas that improved my code.
LOAD "SIG",8,1
When I was a young man in the program, they tested the unit by having us march shoeless through 2 miles of uphill, mine-ridden, barbed-wire-laced snow! The unit got tested, and tested HARD! The program didn't allow for no pansy-ass pussy-footers. And did the unit in the program pass its tests? By God it did! You youngsters got it easy just havin to do some stupid vocabulary test to test your unit in the program. Plugging in words. HAH! Try plugging in the gaping hole left by the bark of an exploding tree!
The only part that is an "art" is working out how to successfully isolate the component that you're trying to test. For simple components at lower layers (typically data CRUD) it's not so hard. Once you find you're having to jump through hoops to set up your stubs, it gets harder to "fake" them successfully and becomes a more error prone and time consuming process. It can also be difficult if there's security in the way. The very checks you've put in to prevent security violations now have to be worked around or bypassed for your unit tests. There's also a danger of becoming too confident in your code because it passes the test when run against stub data. You may find there's a bug specific to the interfaces you've stubbed. (For example a bug in a vendor's database driver, or a bug in your data access framework that doesn't show up against your stub).
All of those distracting side issues and complications aside, we are dealing with fundamental engineering principles. Build a component, test a component. Nothing could be simpler, in principle. So it's disappointing when developers get so caught up in the side issues that they resist unit testing. There does come a point where working around obstacles makes unit testing hard and you have to way benefit against cost and ask yourself how realistic the test is. But you don't go into a project assuming every component is too hard to unit test. That's just lazy and self-defeating. It comes down to the simple fact that many programmers aren't very good at breaking down a problem. In industries where their work was more transparent, they wouldn't last long. In software development where your code is abstract and the fruit of your work takes a long time to get to production, bad developers remain.
These posts express my own personal views, not those of my employer
I read this book recently and found it tiring. Much of it reads like a blog, and like many books, the author randomly switches stances. He'll refer to the reader as "the reader", "you", "we", and in the third person. This is the kind of book where it's hard to keep a clear idea of what the author is talking about, because he doesn't have a clear idea of what he's trying to communicate.
When I think of tiring books like this, I cannot avoid always remembering Steve McConnel's Code Complete (first edition; I haven't looked at the second edition yet). Reading that book is like having your autonomy assaulted, because the author constantly tries to get you to accept the things he's claiming, via whatever means necessary, rather than presenting them along with rational arguments, and letting you decide when to apply them. I'm not saying Osherove's book is that bad, just that it has that same unenjoyable aspect that makes it a chore to read and get useful information from.
I recently also read Kent Beck's Test-Driven Development and highly recommend it, if you simply want to learn about unit testing and test-driven development. It's concise and enjoyable to read. Unfortunately it doesn't cover as many details, and I don't have any good alternatives to books like Osherove's (and I've read many at my local large university library).
Unit tests should be trivial for the majority of classes. Good OO design will cause your many of your classes to be single purpose and simplistic therefore the unit tests will also be simplistic. That's the point of OOD (or even modular design)--breaking down complex problems into many simpler problems*.
Maybe you should consider that unit testing is not just for validating the current set of objects but also validating that future revisions do not break compatibility. In other words it makes regression testing possible or easier with automation.
Writing the unit tests also serve to prove to your teammates you've thought about boundary conditions and logic errors. When you're forced to think them in a structured way then you're in a better position to catch code bugs while writing the unit tests. Many times you'll find them before even executing the test code.
Note: If anyone responds with something along the lines of "complex problems cannot always be simplified" I will literally punch you--repeatedly.
Camping on quad since 1996.
Does your speaker have anything concrete with citations to back his assertions up, or is he happily dismissing one of the few genuine advances in software engineering in the last decade?
we found that the code developed using a test-driven development practice showed, during functional verification and regression tests, approximately 40% fewer defects than a baseline prior product developed in a more traditional fashion. The productivity of the team was not impacted by the additional focus on producing automated test cases. This test suite will aid in future enhancements and maintenance of this code.
-- http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.129.7992&rep=rep1&type=pdf
A Spring 2003 experiment examines the claims that test-driven development or test-first programming improves software quality and programmer confidence. The results indicate support for these claims
-- http://portal.acm.org/citation.cfm?id=949421
Experimental results, subject to external validity concerns, tend to indicate that TDD programmers produce higher quality code because they passed 18% more functional black-box test cases.
-- http://qwer.org/tdd-study
We observed a significant increase in quality of the code (greater than two times) for projects developed using TDD compared to similar projects developed in the same organization in a non-TDD fashion.
-- http://portal.acm.org/citation.cfm?id=1159733.1159787
My apologies for the rough and ready citations, I only picked the ones I could find on the first fucking page of Google search results.