Slashdot Mirror
Bug Hunting Open-Source vs. Proprietary Software
PreacherTom writes "An analysis comparing the top 50 open-source software projects to proprietary software from over 100 different companies was conducted by Coverity, working in conjunction with the Department of Homeland Security and Stanford University. The study found that no open source project had fewer software defects than proprietary code. In fact, the analysis demonstrated that proprietary code is, on average, more than five times less buggy. On the other hand, the open-source software was found to be of greater average overall quality. Not surprisingly, dissenting opinions already exist, claiming Coverity's scope was inappropriate to their conclusions."
Knuth used to have this great offer where he'd send you a check for pi or e or something if you managed to find a bug in his code.
Well, what is a bug?
I doubt he'd send me a check if I told him that TeX doesn't have an easily accessible iconic user interface. No, his concept of a bug is a deviation from the specified functionality.
But what if that functionality is wrong or sucks?
Apple does really well at creating functionality that doesn't suck. They suffer from the same problems of deviations from the spec as much as anyone, but they manage to mold their spec around what users want. Microsoft, to some extent, does the same and they release products that conform to what users want (generally) because they change the spec as necessary when customers demand change.
If you are implementing towards a standard (like most OSS projects with any traction are wont to do), then you are necessarily restricted by what that spec says. If the spec says to do something inane, the standard-follower must implement it that way.
I don't really have a point here except to say that unless they say "this is what we mean by bug", there can be no way to really examine their results.
While I appreciate that PreacherTom was good enogh to bring this to us, the sentence "...no open source project had fewer software defects than proprietary code." just does not match TFA.
TFA says that no open source project is as good as the BEST of proprietary, but it also says that the AVERAGE open source is better than the AVERAGE proprietary.
No, *popular* open-source software is 5x as buggy as *safety-critical* closed software. The linked dissenting opinion is at least partly right; they're comparing apples to oranges.
Maybe they should try comparing open- and closed-source software that's actually trying to solve the same problem? That'd be a bit more valid of a comparison...
He's comparing "bugs" in a project such as Apache with "bugs" in the software controlling a jet engine on an airplane.
He refuses to accept that different projects have different requirements. When the project results in people dying if it fails, you spend a LOT more money and time finding all the "bugs".
When the worst that happens is that you don't see a web page, your money/time requirements are not so high.
Even so, from his finding, Open Source is, on average, better than the closed source projects (not counting the closed source projects that result in loss-of-life in the event of a failure).
He's an idiot for confusing the different requirements.
Says who? QA and testing covers the entire gamut, from formalized unit-testing at every level, to 'throw it at the beta testers and hope nothing breaks'. it's got nothing to do with 'proprietary' (not 'propriety') vs open source.
Where on Earth did you get that? Are you completely oblivious to all the testing methodologies and systems developed by the open source community? Here's a few for you to research: JUnit, Test::Unit, and Selenium.
Again with the generalizations! Commercial software development is, by definition, proprietary, so you don't know how they do it! They might tell you they have a 'strong QA engineering component' (whatever that means) but they could be full of shit!
www.lucernesys.comHorizon: Calendar-based personal finance