Bug Hunting Open-Source vs. Proprietary Software

PreacherTom writes "An analysis comparing the top 50 open-source software projects to proprietary software from over 100 different companies was conducted by Coverity, working in conjunction with the Department of Homeland Security and Stanford University. The study found that no open source project had fewer software defects than proprietary code. In fact, the analysis demonstrated that proprietary code is, on average, more than five times less buggy. On the other hand, the open-source software was found to be of greater average overall quality. Not surprisingly, dissenting opinions already exist, claiming Coverity's scope was inappropriate to their conclusions."

Comments

Re:So how did they test the proprietary software?

[msh104]

they tested it by using a program that systemattically scans code for common errors.

I don't know if the closed source statistics are online somewhere, but these are the open source statistics.
http://scan.coverity.com/

and if you ask me the "Defect Reports / KLOC" is pretty low, and such software would normally be considered "good" software.

Misquoting TFA

[Harmonious Botch]

While I appreciate that PreacherTom was good enogh to bring this to us, the sentence "...no open source project had fewer software defects than proprietary code." just does not match TFA.

TFA says that no open source project is as good as the BEST of proprietary, but it also says that the AVERAGE open source is better than the AVERAGE proprietary.

Re:Why is this surprising?

[tb3]

Are you nuts? Or are you just trying to see how many vapid over-generalizations you can jam into a single comment?

Propriety software traditionally undergoes a formalized, designed testing process. It's not perfect, but it's an ordered approach to boundary testing, design level implementation of quality, and more.

Says who? QA and testing covers the entire gamut, from formalized unit-testing at every level, to 'throw it at the beta testers and hope nothing breaks'. it's got nothing to do with 'proprietary' (not 'propriety') vs open source.

Open source software must rely on after-the-fact testing in the form of "this broke when I tried to do this".

Where on Earth did you get that? Are you completely oblivious to all the testing methodologies and systems developed by the open source community? Here's a few for you to research: JUnit, Test::Unit, and Selenium.

Commercial software has a strong QA engineering component. Open Source software relies primarily on a black box testing approach.

Again with the generalizations! Commercial software development is, by definition, proprietary, so you don't know how they do it! They might tell you they have a 'strong QA engineering component' (whatever that means) but they could be full of shit!