Secure Programming Exams Launched

An anonymous reader writes "The SANS Software Security Institute, in conjunction with organizations such as Siemens, Symantec, Juniper, OWASP, and Virginia Tech, has announced a program for testing whether programmers know how to write secure code. The Secure Programming Skills Assessment is split into separate language families (C/C++, Java/J2EE, Perl/PHP, and ASP/.NET). Director of research Alan Paller says 'This assessment and certification program will help programmers learn what they don't know, and help organizations identify programmers who have solid security skills.' The pilot exam will be held in Washington DC in August, followed by a global rollout."

Comments

Not impressed

[Anonymous Coward]

I took the sample "C/C++" exam, making a point to respond quickly and not to consult any references (something I'd never do when actually programming) and I have to say I was singularly unimpressed. The emphasis wasn't on trying to teach good programming practices but rather on trying to fool the person taking the exam with tricky questions that have little if any relevance in actual programming.

For example, one question was basically making the point that sizeof is a compile time thing that cannot be used to, say, determine the size of a string buffer passed through a char * parameter. But rather than present the question in a straightforward fashion, they has this business of dividing one sizeof value by another - completely unnecessary and confusing. I got the question right but I could just as easily missed it because of the unnecessary complexity and still understood the underlying concept.

One I did miss (I got 8 out of 10 right, I think) was one about how to declare a pointer to a array of constant strings. The question was phrased so awkwardly that I got confused as to what theyactually wanted and gave the wrong answer even though I know how to get the effect they want.

I was also unimpressed that there was no feedback given as to why a given answer was correct or incorrect. No feedback is arguably fine if the intent is purely to assess skill levels, but the stated intent is that this "will help programmers learn what they don't know". You can't do that unless you also explain why a given answer is wrong. Perhaps this is merely an artifact of the sample exams, but if so the samples aren't very interesting.

If this sample exam is in any way representative I think there's a lot more work to be done before this will actually be useful.