450 Million Lines of Code Can't Be Wrong: How Open Source Stacks Up

An anonymous reader writes "A new report details the analysis of more than 450 million lines of software through the Coverity Scan service, which began as the largest public-private sector research project focused on open source software integrity, and was initiated between Coverity and the U.S. Department of Homeland Security in 2006. Code quality for open source software continues to mirror that of proprietary software — and both continue to surpass the industry standard for software quality. Defect density (defects per 1,000 lines of software code) is a commonly used measurement for software quality. The analysis found an average defect density of .69 for open source software projects, and an average defect density of .68 for proprietary code."

Comments

Correction

[Sla$hPot]

"450 Million Lines of Code Can't Be Wrong"
should have been
"450 Million Lines of Code Can't ALL Be Wrong"

Some things can't be measured objectively

[Hentes]

Errors per lines of code may give you a hard number, but that number has nothing to do with the quality of code. It only takes one well-placed error to ruin a piece of software.

Re:Some things can't be measured objectively

[gorzek]

You are wrong, and here's why.

With no measurements at all, you cannot make informed judgments about the quality of your software. You can only guess. This means you would be unable to convince anyone (sane and intelligent) that your product has n bugs. "Because I say so" is not a metric.

With a poor measurement--such as one that ranks all defects equally--you have information, but now it's bad information. If you share the information but not the method(s) used to gather it, you can convince people you're right, because you have data about it. Never mind if you are stacking up Product A with 1 show-stopping bug against Product B with 50 cosmetic bugs or unhandled corner cases. By this bugcount-only metric, Product A looks better, and that's just stupid.

You need good measurements, and sometimes that includes measurements which cannot be quantitatively calculated without human intervention. A human programmer (or QA or other support person) who is familiar with a product will know just how severe a given bug is in terms of its impact. It is why, after all, bug tracking systems generally allow you to prioritize work by severity, fixing the worst bugs first.

Poor information is worse than no information because it can lead you to make the wrong decisions with confidence. With no information, at least you know you are shooting in the dark.

Re:Defects fixed for proprietary may differ.

[Cenan]

Propietary defects are ones that may cause financial harm. FOSS defects are ones that cause annoyance.

I know that our code has more defects than we'd consider fixing purely because the CBA isn't there.

I'm guessing you mean defects in propietary software only gets fixed if they have an impact on the bottom line? Otherwise that whole reply makes no sense.

Anyways, that is not much different from the OSS model. Whoever cares about the sub-system that has a bug, fixes it, and if nobody cares (or has the skills to fix it) it can go ignored for years. The selector for OSS is different, but the end result is the same: nobody gives a fuck about the end user unless it directly affects their day/paycheck/e-peen.

What else is in the "industry"?

[ZahrGnosis]

and both [proprietary and open-source software] continue to surpass the industry standard for software quality

... What else is there? And why is this unknown third type of code dragging down the "industry"?

Re:it contradicts the definition

[Anonymous Coward]

Wrong. There are quite a few organizations who have access to Windows source code, yet Windows is still proprietary software. Proprietary just means that you cannot freely share, not that you have no chance to get the source code.

Re:and all the children are above average

[ZorroXXX]

The selection of sample projects is biased. For proprietary software, the data is taken from projects that at least cares as much for code quality that they run some tools (e.g. at least Coverity) to analyse it. I would suspect that the industry standard is below that because there exists some companies that mostly only consider "get the product out the door". For open source the selection is probably also somewhat scewed, in that they have analysed relatively large, mature and highly successfull projects. I would assume those have higher quality than the average sourceforge/github project.