Are All Bugs Shallow? Questioning Linus's Law

root777 writes to point out a provocative blog piece by a Microsoft program manager, questioning one of the almost unquestioned tenets of open source development: that given enough eyeballs, all bugs are shallow. Are they? Shawn Hernan looks at DARPA's Sardonix experiment and the Coverity static-analysis bug discovery program in open source projects to conclude that perhaps not enough eyeballs are in evidence. Is he wrong? Why? "Most members of the periphery [those outside the core developer group] do not have the necessary debugging skills ... the vast numbers of 'eyeballs' apparently do not exist. ... [C]ode review is hardly all that makes software more secure. Getting software right is very, very difficult. ... Code review alone is not sufficient. Testing is not sufficient. Tools are not sufficient. Features are not sufficient. None of the things we do in isolation are sufficient. To get software truly correct, especially to get it secure, you have to address all phases of the software development lifecycle, and integrate security into the day-to-day activities."

Comments

Re:Bugs are an error in the...

[rcamans]

There are many assumptions in Linus statement, several hidden, and several just wrong. (I love linux).
Code review works only if you have sufficient info, time, and training. So you need the supporting documentation for a piece of software to review it. The flow diagrams, what it is supposed to do, the design docs, the api docs, etc. Such docs could be released along with the code, or pointers to the docs. But currently that is not common practice.

Code review is a complex and painful task. So only people who have a lot of time on their hands, and strong motivation, take on reviewing others' code. Motivation currently tends to be either paid to do it, or a bug that is bugging you. Otherwise, you are unlikely to do code review. A larger number of reviewers does not automatically occur just because the code is available.

Also, most of the people with all the necessary skills, training, knowledge, and experience are actually busy writing or maintaining their own code, and do not have the free time, or insanity, to spend reviewing others' code.

It just ain't goin to happen. Get over it.