Slashdot Mirror
Seeking Input for Software Verification Policies?
e8johan asks: "I currently work at a company in the automotive industry. It is importat to ensure quality and for our products we use extensive testing before, during and after the assembly. Now I have been asked to help in the creation of a company instuction for ensuring that all 'in-house' developed software works properly. I have written a short summary describing the essentials such as: "do not ignore compiler warnings", "always describe what the code is intended to do in your comments", "do not assume that all users run in the same environment", etc. I now feel that I need more input, has anyone done anything like this before? How do you test your software?"
First you should understand the difference between testing and verification. With testing, you rule out a certain number of bugs, but you can never guarantee the absence of bugs. The latter is only possible with software verification, where you prove that the software contains no bugs and fulfills its specification.
Unfortunately, software verification is very expensive. Also, software verification is still a subject of research, and AFAIK it has not been tried for large software projects so far. (My current research project aims at verifying a complete operating-system kernel -- check out the VFiasco project.)
So in most cases, verification is too expensive, and you need to ensure robustness by other means like a good programming standard and testing. (I believe that is what you are looking for, despite the title of your question including the word ``verification.'')
If your shop uses C, I recommend the following book: Les Hatton, Safer C . It explains what a good programming standard should contain, and advocates automatic enforcability. (It also explains what the standard should not contain -- in particular, issues of style such as indentation and variable naming.) One example of such a programming standard for C are the Guidelines for the Use of the C Language in Vehicle Based Software, which should be particularly relevant to you as you also work in the automotive industry.
Testing is another important aspect. In safety-related environments such as yours, a high test coverage for your code is desirable. This requirements suggests automated testing on the unit level. Automated unit testing is practical only if the code is designed to be testable, and there are a number of strategies which provide this property, such as avoiding dependency cycles (for C and C++, John Lakos' Large-Scale C++ Software Design still is the ultimate guide), or creating the unit test before the program (the Extreme Programming way). I also liked Testing Object-Oriented Systems: Models, Patterns, and Tools by Robert Binder - a huge and thorough guide to all imaginable kinds of testing, especially for object-oriented systems.
But if you haven't already, read the Practice of Programming. It has some good thoughts about testing applications that should help you out. And it's a pretty decent book, anyway.
Here's a few random things that occur to me (most are probably repeats of other posts, though):
Automate as much as you can (testing, code generation, anything that's possible to automate - do it).
Use strongly typed languages, and strongly typed interfaces.
Use any and all warning flags, if available
Whenver you find a bug, add a test that detects that bug to your test suite. Then, and only then, fix the bug.
This can hinder the finding of bugs. Too often I have seen people read the comments, say "oh, that's what it does" and go on without actually looking at the code.
A much better idea: write clear code, with good variable names, well abstracted and well named procedures / functions / methods, so that the code itself is the "documentation."
Save the comments for out-of-band information like citations, etc.
-- MarkusQ