Coverity Report Finds OSS Bug Density Down Since 2006

eldavojohn writes "In 2008, static analysis company Coverity analyzed security issues in open source applications. Their recent study of 11.5 billion lines of open source code reveal that between 2006 and 2009 static analysis defect density is down in open source. The numbers say that open source defects have dropped from one in 3,333 lines of code to one in 4,000 lines of code. If you enter some basic information, you can get the complimentary report that has more analysis and puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby. While Coverity has developed automated error checking for Linux, their static analysis seems to be indifferent toward open source."

Comments

Three?

[Dhar]

"...puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby."

Counting, apparently, was low in quality.

Re:Three?

[akadruid]

and then you get so-called slashdotphiles, who think they can hear artifacts in the lossy story compression.

let's see how you fare in a double blind test

Re:Three?

[eldavojohn]

TFA says four.

So, not only are the /. summaries merely paragraphs copied from the article nowadays, they're paragraphs copied incorrectly.

So if my summary was "merely paragraphs copied from the article" then where did I get the 1 in 3,333 and 1 in 4,000 numbers from?

Also, if all I did was copy/paste the article, I'd be plagiarizing and -- not only that -- I would have copy/pasted the correct count of the projects in Rung 3 status. Instead I skimmed the report and was thinking "Rung 3" when I wrote that sentence the three was put in instead of the four. Doesn't make me any less wrong but I hate anonymous non-constructive criticism that's modded up. I apologize for my human error, obviously the human editor also missed it. Since you're anonymous, I can't assume you're human and beg you to relate to my plight of errors. I'm sure my error made the summary completely unreadable. I'm also certain that you've published hundreds of articles on Slashdot without so much as a single error in any of them.

You do know that the number of submissions I've had recently, almost all have had some flaw or error in them. Simply because I realize there's no reward for fact checking. And there's no penalty for getting an error published. So assuming the summary sells to eyeballs and there's no error large enough to get it rejected the next thing is timing. I've written submissions that have been beat out by a few minutes and I get marked "dupe" by firehose. So that pushes me from taking 10-15 minutes to create a summary to 2-3 minutes. Oh well, the worse penalty is if I respond to the article (like this) I'm modded down by righteous moderators. Doesn't really bother me.

If the editors aren't catching the errors and I've got no incentive to reduce the errors, do you think they're going to go away?

Survivorship bias

[vlm]

Survivorship bias

http://en.wikipedia.org/wiki/Survivorship_bias

The projects that were alive back then, and now, are obviously more mature, thus would have fewer bugs. Unless you believe in spontaneous generation of bugs at a constant rate in unchanged code (in my experience, actually not too unbelievable for old C++ compiled by the newest G++ due to specification drift)

Re:Umm yeah

[Disgruntled Goats]

At 4000 lines of code every second (e.g. 4GHz) you are looking at 33.2 years to check that much code.

GHz = 1 billion cycles per second. You're only about 6 orders of magnitude off.

Re:Umm yeah

[eldavojohn]

A: We know they didn't check the code by hand.

Of course not, do you know what static code analysis is? I repeatedly said that in the summary.

B: The methodology didn't classify defects (cosmetic, seucrity, minor, major. etc.)

From the report, which is linked to in the article and you obviously didn't care to read before criticizing:

NULL Pointer Deference
Resource Leak
Unintentional Ignored Expressions
Use Before Test (NULL)
Use After Free
Buffer Overflow (statically allocated)
Unsafe Use of Returned NULL
Uninitialized Values Read
Unsafe Use of Returned Negative
Type and Allocation Size Mismatch
Buffer Overflow (dynamically allocated)
Use Before Test (negative)

They then go on to discuss Function Length and Complexity Metrics.

C: The numbers aren't normalized nor broken by application size.

I don't understand how this is statistically relevant. The summary I gave lists by static code defect per line of code and looks at function length. Of course a project with 4 million lines of code would have more defects than one of 4 thousand lines of the code. The lines of code is the normalization!

D: The use of a bug reporting database needs to be measured in regards to a baseline filing\fix % not a total volume (as we need to correlate new lines of code being added)

Does it make any difference to the end user whether 90% of the project is new lines of code or 9% of the project is new lines of code?

It reads like something from the Onion.

You didn't read the report so you can't really speak.

Dear Lord journalism is dead...

Says the poster who didn't read or understand the report.