Are All Bugs Shallow? Questioning Linus's Law

root777 writes to point out a provocative blog piece by a Microsoft program manager, questioning one of the almost unquestioned tenets of open source development: that given enough eyeballs, all bugs are shallow. Are they? Shawn Hernan looks at DARPA's Sardonix experiment and the Coverity static-analysis bug discovery program in open source projects to conclude that perhaps not enough eyeballs are in evidence. Is he wrong? Why? "Most members of the periphery [those outside the core developer group] do not have the necessary debugging skills ... the vast numbers of 'eyeballs' apparently do not exist. ... [C]ode review is hardly all that makes software more secure. Getting software right is very, very difficult. ... Code review alone is not sufficient. Testing is not sufficient. Tools are not sufficient. Features are not sufficient. None of the things we do in isolation are sufficient. To get software truly correct, especially to get it secure, you have to address all phases of the software development lifecycle, and integrate security into the day-to-day activities."

Comments

Re:Bugs are an error in the...

[rtb61]

What the essay fails to capture is the nature of the functioning of the eyeballs in practice, between open source and closed source. In closed source, the eyeballs only look at what they are paid to look at, if the code is just barely good enough to sell, then out it goes and nobody looks at that code again until the complaints start rolling in and then and only the do they fix it, well, sort of fix it, they of course only fix it just barely enough to silence the noisiest of complaints and the only if there are real consequences for failing to do so. Don't think so then try this http://social.technet.microsoft.com/Search/en-GB?query=this%20is%20a%20know%20fault&ac=8 and a huge number of them have never been fixed.

Open source follows a completely different series of routes;
1) People looking for faults because they get a kick out of finding them and fixing them.
2) Tweaks to functions that indirectly remove bugs by simply replacing them with better code.
3) Discoveries in user interactions, less of a complaint because there is no force in pushing the fix.
5) Governments and government departments directly pursuing more secure code.
6) Corporations seeking to build a public reputation by demonstrating coding expertise.

So in the case of open source software there are many 'different' kinds of eyes, so those eyes all working from different perspectives do in reality make bugs very shallow. In the closed source proprietary world the bugs are buried in the depths of the code, hiding in the dark, basically because of profits versus workmanship issues, which means no light is shone on them because only one set of eyes looking from a single 'shallow' perspective looks at them.

There is of course one other set of eyes looking at code, the saboteurs both private and government, looking for faults to exploit. Hard with open source because it can rapidly turn around and bite you on the arse if you use it (if you protect against it everybody notices). Closed source (mostly but a lot of less than honourable eyes lend up looking at it), of course can be targeted as long as you, well, use open source code yourself whilst promoting closed source to everybody else (hmm, kind of reminds me of all those mainland China computer companies, odd that, isn't it).