Test Coverage Leading You Astray?

An anonymous reader writes "Are your test coverage measurements leading you astray? Test coverage tools bring valuable depth to unit testing, but they're often misused. This article takes a closer look at what the numbers on the coverage report really mean, as well as what they don't. It then suggests three ways you can use your coverage to ensure code quality early and often."

Comments

code flow is also important

[joebebel]

code flow is just as important as code coverage. If code in section 1 is executed in unit test 1, and code in section 2 is executed in unit test 3, there needs to be a unit test which executes both. All combinations have to be handled, if sections of code have side effects on other sections.

Unit tests are too simplistic for many apps

[BadAnalogyGuy]

The idea that you can input some values and expect useful output from a function is nice in theory. Perhaps in some very limited mathematics oriented programs where the inputs must lead to a nice answer, but real world applications may end up manipulating more than just the input data.

Can you test that the LCD has refreshed at the inputted rate? Can you verify that the input data was correctly injected into the database just be checking the output of the function?

Functions lie like dogs. You can test the output of functions until you're blue in the face, but until you take a holistic view of the application and what it does, unit tests are more a salve for management's mind than a boon to developers.

Re:Non-object oriented test tools?

[yermoungder]

"I'd love to hear from anyone who can recommend test coverage tools for C..."

See http://www.polyspace.com/

Re:Non-object oriented test tools?

[GejTOO]

Here is a list of testing frameworks for several languages.

http://c2.com/cgi/wiki?TestingFramework

G.

The Emperor has very few clothes

[ribuck]

Test coverage measurement is a really dilute quality assurance tool. It can show you parts of your code that are untested, but it doesn't say anything about whether the other parts of your code are tested.

Just executing a line of code or a branch (whilst running a test) does not imply that you are testing that code.

Re:DO-178B - MCDC

[Anonymous Coward]

Note that DO-178B requires MCDC (Modified Condition Decision Coverage) for level A software (check DO178B page 74).
MCDC requires that "every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to independently affect that decision's outcome. A condition is shown to independently affect a decision's outcome by varying just that condition while holding fixed all other possible conditions" (Miller and Chilenski).