Slashdot Mirror
Test Coverage Leading You Astray?
An anonymous reader writes "Are your test coverage measurements leading you astray? Test coverage tools bring valuable depth to unit testing, but they're often misused. This article takes a closer look at what the numbers on the coverage report really mean, as well as what they don't. It then suggests three ways you can use your coverage to ensure code quality early and often."
Who needs testing? Doesn't everyone's code work perfectly on the first ru
Segmentation fault
Bloody cricket.
Real Daleks don't climb stairs - they level the building.
Three types of code coverage are required for safety critical airline applications:
1) Line Coverage - Has every line been tested
2) Branch Coverage - Has every branch been tested
3) Boolean Coverage - Is EVERY possibility on a truth table for each logical operator explicitly defined
These tests alone don't certify that the code is ready for an airplane and that it is indeed "bug free." My software engineering professor said it best when he stated, you can only prove the existence of bugs, you cannot prove the non-existence of bugs. These guidelines as adopted by the FAA for the certification of safety critical code, don't prove the non-existence of bugs, but they do go a long way towards proving the existence many bugs and provide a MINIMUM standard to which code must be exercised before being allowed into an airplane.
Software Engineering is a science, methodology has been pioneered to help us ENGINEER the software we develop to be as defect free as we know how to make it. As in other disciplines of engineering, there will always be things not yet quantified. Take architecture for example, an architect would design a bridge to withstand an earthquake of a specific magnitude, winds of a specific speed. Does that mean the bridge is safe? What if the materials used weren't rated for the temperature range needed for the locale, etc...
As much as we do to ensure quality, there is no silver bullet. The company I interned at which will remain nameless made a multi-function navigational display for airforce one. It rebooted during a touch and go at 40 degrees farenheit. Wasn't it tested you ask? Of course it was, it was tested at -40 degrees and 140 degrees, but the timing on one of the buses was off at 40 and the hardware watchdog took it into a reboot at a very critical time. It was DO 178B Level A certified, had 100% code coverage of course, but there will always be bugs. Don't trust tools to tell you otherwise, because you can never prove the non-existence of bugs.
(For those who don't know, a touch and go is where the plane starts landing and takes off again)