Coverity Report Finds OSS Bug Density Down Since 2006

← Back to Stories (view on slashdot.org)

Coverity Report Finds OSS Bug Density Down Since 2006

Posted by timothy on Wednesday September 23, 2009 @06:27AM from the but-3333-is-a-cool-number dept.

eldavojohn writes "In 2008, static analysis company Coverity analyzed security issues in open source applications. Their recent study of 11.5 billion lines of open source code reveal that between 2006 and 2009 static analysis defect density is down in open source. The numbers say that open source defects have dropped from one in 3,333 lines of code to one in 4,000 lines of code. If you enter some basic information, you can get the complimentary report that has more analysis and puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby. While Coverity has developed automated error checking for Linux, their static analysis seems to be indifferent toward open source."

79 comments

Coverity also finds that... by Anonymous Coward · 2009-09-23 06:28 · Score: -1, Offtopic

Rob Malda's penis is tinier than a toddler's.
1. Re:Coverity also finds that... by K.+S.+Kyosuke · 2009-09-23 07:55 · Score: 1
  
  Perhaps, but his smaller number of cells means less cancer-exploitable DNA code bugs.
  
  --
  Ezekiel 23:20
Three? by Dhar · 2009-09-23 06:30 · Score: 5, Funny

"...puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby."
Counting, apparently, was low in quality.
1. Re:Three? by Anonymous Coward · 2009-09-23 06:33 · Score: 1, Insightful
  
  TFA says four.
  So, not only are the /. summaries merely paragraphs copied from the article nowadays, they're paragraphs copied incorrectly.
2. Re:Three? by akadruid · 2009-09-23 06:38 · Score: 5, Funny
  
  and then you get so-called slashdotphiles, who think they can hear artifacts in the lossy story compression.
  let's see how you fare in a double blind test
  
  --
  "Those who cast the votes decide nothing; those who count the votes decide everything." (attrib. Joseph Stalin)
3. Re:Three? by eldavojohn · 2009-09-23 06:59 · Score: 5, Insightful
  
  TFA says four.
  So, not only are the /. summaries merely paragraphs copied from the article nowadays, they're paragraphs copied incorrectly.
  So if my summary was "merely paragraphs copied from the article" then where did I get the 1 in 3,333 and 1 in 4,000 numbers from?
  
  Also, if all I did was copy/paste the article, I'd be plagiarizing and -- not only that -- I would have copy/pasted the correct count of the projects in Rung 3 status. Instead I skimmed the report and was thinking "Rung 3" when I wrote that sentence the three was put in instead of the four. Doesn't make me any less wrong but I hate anonymous non-constructive criticism that's modded up. I apologize for my human error, obviously the human editor also missed it. Since you're anonymous, I can't assume you're human and beg you to relate to my plight of errors. I'm sure my error made the summary completely unreadable. I'm also certain that you've published hundreds of articles on Slashdot without so much as a single error in any of them.
  
  You do know that the number of submissions I've had recently, almost all have had some flaw or error in them. Simply because I realize there's no reward for fact checking. And there's no penalty for getting an error published. So assuming the summary sells to eyeballs and there's no error large enough to get it rejected the next thing is timing. I've written submissions that have been beat out by a few minutes and I get marked "dupe" by firehose. So that pushes me from taking 10-15 minutes to create a summary to 2-3 minutes. Oh well, the worse penalty is if I respond to the article (like this) I'm modded down by righteous moderators. Doesn't really bother me.
  
  If the editors aren't catching the errors and I've got no incentive to reduce the errors, do you think they're going to go away?
  
  --
  My work here is dung.
4. Re:Three? by Runaway1956 · 2009-09-23 07:09 · Score: 1
  
  "Coverity Report Finds OSS Bug Density Down Since 2006"
  Bad news for entomologists, huh?
  
  --
  "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
5. Re:Three? by Anonymous Coward · 2009-09-23 07:27 · Score: 1, Insightful
  
  You have an excuse. Mistakes happen.
  Mistakes like this is why we have editors. The post you replied to was somewhat out of line, though as a general rule I'd say they would have been more accurate than they were in this case. Most submissions ARE copied directly from TFA.
  The real issue is that this was a blatantly obvious, easy-to-catch mistake. We're not talking about to/too or their/there issues that a technically-oriented person may not pick up on at first glance; we're talking about something that the original writer probably would not catch [psychologically, it's hard to tell for yourself you made the mistake until enough time has passed for you to forget writing it] but for which the editor would, in any professional system, get reamed for missing.
6. Re:Three? by Anonymous Coward · 2009-09-23 08:13 · Score: 0
  
  Thanks for your three paragraph rebuttal.
7. Re:Three? by V!NCENT · 2009-09-23 09:28 · Score: 1
  
  What was NASA's 'bug guideline'? I remember seeing or reading it somewhere, I thought it was one bug in 10.000 lines of code.
  I could be absolutely wrong! But I just like to know...
  
  --
  Here be signatures
8. Re:Three? by zizzo · 2009-09-23 09:34 · Score: 1
  
  It's true but static analysis can fix this problem.
9. Re:Three? by someone1234 · 2009-09-23 09:44 · Score: 1
  
  And what is their actual bug count?
  
  --
  Patents Drive Free Software as Hurricanes Drive Construction Industry
10. Re:Three? by evanbd · 2009-09-23 09:51 · Score: 2, Insightful
  
  We're bitching about the slashdot editors, not you. It's their job to catch submitter mistakes. That is what an editor does. The really annoying thing is they're as likely to "edit" the summary to introduce mistakes as to remove them.
11. Re:Three? by Tubal-Cain · 2009-09-23 10:12 · Score: 1, Redundant
  
  0. Samba
  1. tor
  2. OpenPAM
  3. Ruby
12. Re:Three? by TheRaven64 · 2009-09-23 10:48 · Score: 1
  
  Not really, it's a meaningless statistic. Coverity has been publishing these reports for a few years. Every time they do, the relevant projects fix all of the bugs they find. The next year, some proportion of that code is the same code that already had these bugs fixed, so if the total number of bugs per line of code didn't go down it would be quite disappointing. On top of that, there are other static analysis tools, like clang, that are used by a lot of open source projects. Even if Coverity reports no bugs, that doesn't mean that there are no bugs, just no bugs that can be easily found by static analysis.
  
  --
  I am TheRaven on Soylent News
13. Re:Three? by Ihmhi · 2009-09-24 01:57 · Score: 2, Funny
  
  I have gold-plated Ethernet cables, so my Internets sound nice and crisp. You can really hear the richness in the lower kbps range.
  
  --
  Random Thoughts From A Diseased Mind (Not For Dummies)
14. Re:Three? by pha3r0 · 2009-09-24 04:32 · Score: 1
  
  Honestly here everyone. It's a human error and its bound to happen. Hell why even waste time bitching just read on, use your brain to figure out that the 3 was an oversight and spend all the time you do bitching on a more productive task.
  And I know we are all anal retentive nitpicks but by FAR /. has clearer more intelligible writing then any other news or new summary site.
  I find typos, not just grammar errors, in almost ever major new story I read. Give the volunteers a break and go chew on someone that's actually getting paid to turn out this trash.
15. Re:Three? by OhHellWithIt · 2009-09-24 04:48 · Score: 1
  
  Honestly here everyone. It's a human error and its bound to happen. Hell why even waste time bitching just read on, use your brain to figure out that the 3 was an oversight and spend all the time you do bitching on a more productive task.
  It is fair sport to pick nits with spelling, etc. It becomes tedious and unsporting when the nitpicking erupts into a flame war.
  
  --
  "Who controls the past controls the future. Who controls the present controls the past." -- George Orwell
Fewer but bigger by Anonymous Coward · 2009-09-23 06:31 · Score: 1, Interesting

Why would Samba and Linux have got so unstable over the years, then?
1. Re:Fewer but bigger by poetmatt · 2009-09-23 06:49 · Score: 0, Offtopic
  
  Maybe because grammar is tough for windows users?
2. Re:Fewer but bigger by Anonymous Coward · 2009-09-23 07:03 · Score: 0
  
  Grammar would tell you that you should capitalize proper nouns such as Windows. Way to fail, grammar nazi!
3. Re:Fewer but bigger by Anonymous Coward · 2009-09-23 07:24 · Score: 0
  
  Most computer users wouldn't consider windows a proper anything, let alone give them credit of being a noun.
4. Re:Fewer but bigger by Virak · 2009-09-23 10:13 · Score: 1
  
  Wait, Samba and Linux are unstable now? That's news to me. I can't remember the last time either crashed for me ever.
5. Re:Fewer but bigger by Anonymous Coward · 2009-09-23 14:42 · Score: 0
  
  YHBT.
  This place is infested with MS dronez and sockpuppets. All you'll find in Slashdot these days is FUD and market-speak.
  Get out and find a real tech blog.
6. Re:Fewer but bigger by Anonymous Coward · 2009-09-23 14:44 · Score: 0
  
  A proper noun would be Windows XP, windows could be a colloquial for the group of operating systems from Windows x(.xx), Windows 9x & Me, Windows NT and other Windows versions. Ala bsd, unix, linux etc.
  Seeing as you are unable to parse common grammar, please stop using the abused 'fail' family of memes.
7. Re:Fewer but bigger by jhol13 · 2009-09-23 15:30 · Score: 1
  
  Possibilities are endless, just a few here:
  1. Fixing bugs found by Coverity might give false sense of "goodness", especially as:
  2. Coverity does not catch all problems, e.g. timing or parallelism related. Dual cores are now abundant.
  3. A lot of hardware is flaky.
Three? by ifwm · 2009-09-23 06:32 · Score: 1

puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby

Hmmm...
In all seriousness, this seems to point to an increasing level of sophistication and maturity in OSS products and procedures, which can only be a good thing.
Oblig reference by StuartHankins · 2009-09-23 06:34 · Score: 4, Funny

"... and puts three projects at the top tier in quality of the 280 open source projects: Samba, tor, OpenPAM, and Ruby."

Our chief weapon is surprise...surprise and fear...fear and surprise....
Our two weapons are fear and surprise... and ruthless efficiency....
Our three weapons are fear, surprise, and ruthless efficiency...
and an almost fanatical devotion to the Pope....
Our four... no...
Amongst our weapons... Amongst our weaponry...
are such elements as fear, surprise...
I'll come in again.
Wonder when MS, IBM and others will publish? by MosesJones · 2009-09-23 06:34 · Score: 4, Interesting

The question of course is "Is 4000 good, average or bad?" can't be answered because closed source companies just aren't going to publish this sort of information.
So what we can say is that the quality of OSS is trending upwards, but we can't say whether this makes it better, equivalent or worse than close source competitors.
What are the odds on any of them taking up the challenge?

--
An Eye for an Eye will make the whole world blind - Gandhi
1. Re:Wonder when MS, IBM and others will publish? by Anonymous Coward · 2009-09-23 06:49 · Score: 2, Informative
  
  Actually the topic is the subject of research and the blog below quotes some book that says Microsoft is at 1/2000 lines of code.
  http://amartester.blogspot.com/2007/04/bugs-per-lines-of-code.html
  Of course, these studies try to assess the number of defects that have not been found yet... So the numbers are to take with a grain of salt, but apparently testing the software before delivery gets 90% of the bugs.
  The Coverity report is likely based on what the tool says, so you need a grain of salt for that too.
  The trend is probably what matters most. This stuff is really about improving your code, finding what's wrong, checking that you are making progress and trying hard enough.
2. Re:Wonder when MS, IBM and others will publish? by MathFox · 2009-09-23 07:07 · Score: 1
  
  What I heard from a Coverity employee doing a presentation is that the best closed source/commercial projects score as good as the best Open Source projects; bad commercial projects do as bad as bad Open Source projects.
  In other words, the variation in both categories is so big (more than a factor 10!) that one can not say either side is better with statistical relevance.
  
  --
  extern warranty;
  main()
  {
  (void)warranty;
  }
3. Re:Wonder when MS, IBM and others will publish? by Anonymous Coward · 2009-09-23 07:10 · Score: 1, Funny
  
  Actually, we did test our code here at Microsoft, we have 4200 defects by line of code, which is much better than the 4000 of open-source projects.
  wait a second...
4. Re:Wonder when MS, IBM and others will publish? by heritage727 · 2009-09-23 07:14 · Score: 1
  
  Actually the topic is the subject of research and the blog below quotes some book that says Microsoft is at 1/2000 lines of code.
  If some blog quotes some book that makes some claim about Microsoft being worse than Linux, that's good enough evidence for me!
5. Re:Wonder when MS, IBM and others will publish? by gmrath · 2009-09-23 08:34 · Score: 0, Offtopic
  
  Where's the "funny" for this one, mods?
6. Re:Wonder when MS, IBM and others will publish? by Strake · 2009-09-23 09:32 · Score: 1
  
  The question of course is "Is 4000 good, average or bad?" can't be answered because closed source companies just aren't going to publish this sort of information.
  This is part of the reason that OSS is better than closed-source competitors - the bugs are widely-known, and therefore can be more readily fixed.
  This is also part of the reason that the quality of OSS is trending upwards.
7. Re:Wonder when MS, IBM and others will publish? by bloodhawk · 2009-09-23 11:15 · Score: 1
  
  You can't even really say that quality of OSS is trending upwards. The same company using the same tools is doing the analysis, this brings a certain degree of bias as many of the flawes they point out the first time are fixed and therefore artificially lower the error count. The kind of analysis needed is one that covers stuff this code inspection tool doesn't cover to see how the error rates are really trending.
8. Re:Wonder when MS, IBM and others will publish? by Anti_Climax · 2009-09-23 12:11 · Score: 1
  
  They probably wouldn't make a good representative sample, but you could take the source code of projects that were formerly closed and subsequently opened to see how many errors they averaged. The ID engines come immediately to mind.
  
  --
  Even people that believe in pre-destiny look both ways before crossing the street.
9. Re:Wonder when MS, IBM and others will publish? by jc42 · 2009-09-23 14:51 · Score: 2, Interesting
  
  There can be some serious "methodology" problems in many of the definitions of "bugs", that can seriously confuse the bug counters.
  An example that I like to use is a project I worked on in the late 1990s. An important part of the package that I delivered included a directory of several hundred C source files, mostly small, with at least one bug in each. The project's leaders got some chuckles out of mentioning this at meetings, commenting that they had no intention of letting me fix any of the bugs, since they were an important contribution to the project. This produced much confusion among the higher ups, who took some time understanding what was going on and how to account for it.
  Some readers might have guessed what my task was: Building a regression-testing suite for the C compiler. The directory in question was for testing the diagnostics in the compiler. Each source file had one or more carefully designed "bugs". The makefile ran the C compiler on each, and sent the stderr output to a validator that verified that the compiler had successfully identified the bug and produced the right error message.
  We had a bit of fun confusing people by asking them whether these test files really contained "bugs" or not. According to the C standard, they certainly did. But according to the test procedure, these weren't bugs; they were tools for testing the compiler. If they were "fixed", the test scripts would no longer be able to validate the compiler's error messages.
  The higher-ups did finally understand the value of this, and agreed that although this batch of files were full of "bugs", they shouldn't be counted as such in the bug reports.
  I also sometimes listed my job as the project "bugger". It's always fun to construct new words by stripping prefixes off words that usually have them. But I wasn't sure what term was best for the task of making sure that a routine actually contains the bug that the specs say it should have. "Debugging" doesn't seem right when the job is making sure that the right bug is there.
  (Actually, I mostly thought that the project had a minor management problem, since any competent software development manager should understand the value of making sure that the software's error messages are correct and useful. But we all know how rarely this is actually done well. How often does your compiler point to the right place in the code when it produces an error message? And how often does the message describe the actual error?)
  
  --
  Those who do study history are doomed to stand helplessly by while everyone else repeats it.
Bug Density Down? by LeadLine · 2009-09-23 06:37 · Score: 1

... or less effective bug checking?
1 per 4000?! by spiffmastercow · 2009-09-23 06:40 · Score: 1

That's some good coding.. Makes me feel like a n00b. I'm not sure what my bug to code ratio is, but I'm sure its a lot higher than that.
1. Re:1 per 4000?! by jellomizer · 2009-09-23 07:09 · Score: 1
  
  It is not as much that 1 line out of 4000 is average for each programmer. It is just that they fix the bugs before release.
  
  --
  If something is so important that you feel the need to post it on the internet... It probably isn't that important.
2. Re:1 per 4000?! by jimicus · 2009-09-23 08:55 · Score: 1
  
  Remember the bug finding is automated. There are only some classes of bugs that can be automatically found.
3. Re:1 per 4000?! by TheRaven64 · 2009-09-23 10:52 · Score: 1
  
  And not just automated bug finding, but bug finding by static analysis. This is notoriously bad at finding bugs in programs that use shared libraries or indirection layers (e.g. code that calls other code via function pointers).
  
  --
  I am TheRaven on Soylent News
4. Re:1 per 4000?! by Anonymous Coward · 2009-09-23 14:24 · Score: 0
  
  "or indirection layers (e.g. code that calls other code via function pointers)." -- you mean like the code i write when i'm doing um, empirical studies in rate of hairline recession in upper-level IT admins? :-D
5. Re:1 per 4000?! by jc42 · 2009-09-23 14:57 · Score: 1
  
  They also decrease the bugs-per-line count in their coding standards. That's why you see lots of blank lines in the code, lines that contain just a single brace, etc. The more lines you can spread your code over, the fewer bug you have per line.
  If you don't like this observation, you shouldn't be measuring bugs-per-line. But nearly every company does just that.
  There was also the funny thing a few years ago, when MS was claiming that some percent of linux code was stolen from Windows. Someone did a grep, and found that the percent named matched almost exactly the count of lines that consisted of just "/*" or "*/".
  But we can't stop management from counting things that way. We've been pointing out how silly it is for decades, and it has had no discernable effect on published figures.
  
  --
  Those who do study history are doomed to stand helplessly by while everyone else repeats it.
6. Re:1 per 4000?! by TheRaven64 · 2009-09-23 22:08 · Score: 1
  
  Or any Objective-C code, or any code that uses a C object model like GObject...
  
  --
  I am TheRaven on Soylent News
7. Re:1 per 4000?! by jellomizer · 2009-09-24 04:16 · Score: 1
  
  I don't know.
  If someone uses retval for the return value they must be stealing my code.
  
  --
  If something is so important that you feel the need to post it on the internet... It probably isn't that important.
8. Re:1 per 4000?! by jc42 · 2009-09-24 04:49 · Score: 1
  
  Then there's the infamous case of the AT&T /bin/true program, which was a shell script that contained nothing but a blank line and a copyright notice. So if you include blank lines in your code, you're violating AT&T's copyright.
  I had fun once (around 1990) by "publishing" the entire text of one of these on a newsgroup, and publicly challenging AT&T's lawyers to take me to court over this blatant copyright violation. For some unexplained reason, I never heard from them.
  (If you google for "/bin/true copyright", with or without the quotes, you can find several copies of this script. Note that some of them include a version number and more than one copyright date. More recently, it has been distributed as /usr/bin/true, and it's a binary, so it runs much faster. I saw an early version of this that had a -c option which produced a copyright notice, presumably as a bit of geek humor. But I see that this option is no longer present in the current releases on several systems that I have available.)
  
  --
  Those who do study history are doomed to stand helplessly by while everyone else repeats it.
-1 Offtopic, why don't tags work now? by Anonymous Coward · 2009-09-23 06:42 · Score: 0

The stupid little arrow does nothing no matter which "index" view I'm looking at. Guess I first noticed about a week ago. On both Firefox/Linux and Firefox/Windows XP. Posted Anonymous Coward for obvious reasons.
1. Re:-1 Offtopic, why don't tags work now? by thePowerOfGrayskull · 2009-09-23 06:54 · Score: 1
  
  Posted Anonymous Coward for obvious reasons.
  Why? It's not obvious. The sequence of bits stored with your UID known as "karma" might take a hit because this is offtopic? Oh no. End of the world. Do you think your "karma" here matters for anything in the real world?
  Or maybe you're really Ballmer and don't want to get caught here.
  So explain, because it's not obvious at all.
2. Re:-1 Offtopic, why don't tags work now? by nomadic · 2009-09-23 07:00 · Score: 1
  
  The whole point of gaining karma is so you can flame people at +1 starting off and "spend" what you've gained.
3. Re:-1 Offtopic, why don't tags work now? by thePowerOfGrayskull · 2009-09-23 07:22 · Score: 1
  
  Well - crap. I've been doing this all wrong!
4. Re:-1 Offtopic, why don't tags work now? by nomadic · 2009-09-23 07:59 · Score: 1
  
  Yes, you have....you vile, disease-ridden half-wit.
5. Re:-1 Offtopic, why don't tags work now? by thePowerOfGrayskull · 2009-09-23 09:25 · Score: 1
  
  Now there's no reason to - oh! I get it! You pig-sticking goat-roper.
6. Re:-1 Offtopic, why don't tags work now? by MetalPhalanx · 2009-09-23 10:58 · Score: 1
  
  Totally OT in an OT thread, but your user name makes me lol every time I see a post from you.
Three kinds by Anonymous Coward · 2009-09-23 06:49 · Score: 0

There are three kinds of Slashdot story submitters...
Umm yeah by kenp2002 · 2009-09-23 06:54 · Score: -1, Troll

If they check 1 line of code every second it would take 133,101.85 years to check 11.5 billion lines of code. At 1000 lines of code every second you are looking at 133.10 years to check that much code. At 4000 lines of code every second (e.g. 4GHz) you are looking at 33.2 years to check that much code.
A: We know they didn't check the code by hand.
B: The methodology didn't classify defects (cosmetic, seucrity, minor, major. etc.)
C: The numbers aren't normalized nor broken by application size.
D: The use of a bug reporting database needs to be measured in regards to a baseline filing\fix % not a total volume (as we need to correlate new lines of code being added)
I'll only say this. Their methodology is not very good and thus any outcome they draw from is only as vaid as their methodology...
A 1% defect rate in an application, say the size of the Linux Kernel is a very different animal then say a 1% defect rate in say, VIM. In addition, 44 minor defects is very different then 6 majors. A minor defect could be as trival as a field not being aligned properly on a form.
You need to see a comparison such that:
App A
23 Minor
14 Major
3 Critical
and the accompaning resolution rates for the MMCs.
Usually it's
Minor : 95%
Major : 99%
Critical : 100%
Volume of defects is proportional to complexity, not lines of code.
That's like crying there is an epidemic of 5000 people dead every year... With a population of billions that is not even in the area of insignificant. 5000 people dead in a since town... that is a serious problem when the population is only 65000... SCOPE counts in data analysis.
This is garbage reporting... I don't care about the outcome, I don't agree or disagree but the methdology I see so far... I would fail this as an assignment in high school... If you are going to publish news about a report, part of the reporting is to indicate the methodology...
It reads like something from the Onion. "Gartner reported today that Gumby is more popular then Spongbob by using a computer!"
At the very least you need to throw in "Gartner reported today that Gumby is more popular then Spongebob by using a computer model that took people between the ages of 50 and 89 and had them fillout a survey. After normalizing the data and eliminating outliers Gartner consultants were able to determine that Gumby was more recognizable then Spongbob by a margin of over 90%."
Dear Lord journalism is dead...

--
-=[ Who Is John Galt? ]=-
1. Re:Umm yeah by Volante3192 · 2009-09-23 07:13 · Score: 3, Insightful
  
  If they check 1 line of code every second it would take 133,101.85 years to check 11.5 billion lines of code. At 1000 lines of code every second you are looking at 133.10 years to check that much code. At 4000 lines of code every second (e.g. 4GHz) you are looking at 33.2 years to check that much code.
  And if they were only using one system to do this, I'd imagine that would be a problem. I wonder, though, if you spread the processing across, oh, say, 512 processors, if you could get that time down under a month...
2. Re:Umm yeah by Trepidity · 2009-09-23 07:32 · Score: 2, Informative
  
  Isn't 4000 lines/code a second 4 kHz, not GHz, if we're using Hz to measure the frequency of line-processing?
  
  --
  10 PRINT CHR$(205.5+RND(1)); : GOTO 10
3. Re:Umm yeah by Anonymous Coward · 2009-09-23 07:32 · Score: 0
  
  280 programs across 280 machines.
  33.2 years split amongst 280 machines is roughly 44 days of continuous testing (given all programs being the same approx length, total code of 11.5 billion lines, and 1 machine per program at 4000 lines per second)
4. Re:Umm yeah by Tekfactory · 2009-09-23 07:38 · Score: 1
  
  11.5 billion lines of code, with one bug in every 4,000 lets say that's the top line number that their software kicks out almost 3 million bugs. So are there 3 million bugs in all of open source?
  At my last posting we used FxCop to analyze our VB.Net software and it found 3-4,000 errors, now what the tool looks for is wide ranging but all but two of these findings were Variable Naming conventions and best practices developed by the .Net development team.
  The 2 we fixed said Variables did not have Strong typing and another Security option turned on when the executeable was compiled.
  So what is Coverity's definition of a bug?
  Now if I had the definition of bug, are any of these bugs in common libraries? Were any of them copy pastes from other software that could be fixed on a one to many basis?
5. Re:Umm yeah by Disgruntled+Goats · 2009-09-23 07:43 · Score: 5, Informative
  
  At 4000 lines of code every second (e.g. 4GHz) you are looking at 33.2 years to check that much code.
  GHz = 1 billion cycles per second. You're only about 6 orders of magnitude off.
6. Re:Umm yeah by Tekfactory · 2009-09-23 07:45 · Score: 1
  
  So looking at the article, but not the study itself, people are submitting their projects to Coverity for static analysis, and 11,200 defects have been eliminated with the ehlp of the program.
  Article also says 60 million unique lines of code were scanned on a recurring basis from 280 projects.
7. Re:Umm yeah by eldavojohn · 2009-09-23 08:01 · Score: 5, Insightful
  
  A: We know they didn't check the code by hand.
  Of course not, do you know what static code analysis is? I repeatedly said that in the summary.
  
  B: The methodology didn't classify defects (cosmetic, seucrity, minor, major. etc.)
  From the report, which is linked to in the article and you obviously didn't care to read before criticizing:
  
  NULL Pointer Deference
  Resource Leak
  Unintentional Ignored Expressions
  Use Before Test (NULL)
  Use After Free
  Buffer Overflow (statically allocated)
  Unsafe Use of Returned NULL
  Uninitialized Values Read
  Unsafe Use of Returned Negative
  Type and Allocation Size Mismatch
  Buffer Overflow (dynamically allocated)
  Use Before Test (negative)
  They then go on to discuss Function Length and Complexity Metrics.
  
  C: The numbers aren't normalized nor broken by application size.
  I don't understand how this is statistically relevant. The summary I gave lists by static code defect per line of code and looks at function length. Of course a project with 4 million lines of code would have more defects than one of 4 thousand lines of the code. The lines of code is the normalization!
  
  D: The use of a bug reporting database needs to be measured in regards to a baseline filing\fix % not a total volume (as we need to correlate new lines of code being added)
  Does it make any difference to the end user whether 90% of the project is new lines of code or 9% of the project is new lines of code?
  
  It reads like something from the Onion.
  You didn't read the report so you can't really speak.
  
  Dear Lord journalism is dead...
  
  Says the poster who didn't read or understand the report.
  
  --
  My work here is dung.
8. Re:Umm yeah by RichardJenkins · 2009-09-24 00:46 · Score: 1
  
  How efficient is static analysis? perhaps it takes a million processor cycles to check one line of code, and the author takes it for granted that us plebs understand that.
9. Re:Umm yeah by Hatta · 2009-09-24 03:24 · Score: 1
  
  How do you design an algorithm to detect bugs? If there is such an algorithm, why isn't included in compilers so that they can produce bug free code?
  It seems to me that in order to determine whether something is a bug, you have to know what the programmers intent was. That requires intelligence.
  
  --
  Give me Classic Slashdot or give me death!
10. Re:Umm yeah by Volante3192 · 2009-09-24 04:15 · Score: 1
  
  http://scan.coverity.com/faq.html#static
  Some examples of the defects include:
  * leaked resources
  * references to pointers that could be NULL
  * references to pointers that are guaranteed to be NULL
  * use of uninitialized data
  * array overruns
  * unsafe use of signed values
  * use of resources that have been freed
  It won't fix misplaced decimals (sorry Mr. Bolton), but it'll probably catch if you try sticking an item into $array[11] when you've defined @array[10].
  (Yes, I know I used perl syntax which doesn't give a whit, but it's what I'm familiar with. It's the theory I'm trying to convey.)
Detection Accuracy? by Reason58 · 2009-09-23 06:59 · Score: 1

It seems logical that older security issues are more well-known and documented than newer ones. Is it possible that the results do not point to an improvement in coding quality so much as an inability to detect newer flaws as accurately older flaws?
Survivorship bias by vlm · 2009-09-23 07:04 · Score: 5, Interesting

Survivorship bias
http://en.wikipedia.org/wiki/Survivorship_bias
The projects that were alive back then, and now, are obviously more mature, thus would have fewer bugs. Unless you believe in spontaneous generation of bugs at a constant rate in unchanged code (in my experience, actually not too unbelievable for old C++ compiled by the newest G++ due to specification drift)

--
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
1. Re:Survivorship bias by Anonymous Coward · 2009-09-23 09:10 · Score: 1, Informative
  
  Old projects doesn't necessarily mean old code. Currently, on average each day the linux kernel adds 13K lines, deletes 5K lines, and changes 2.8K lines. Over a year, that works out to roughly 4.5M lines, 2M lines, and 1M lines.
  For a project with roughly 12M lines of code, that's a pretty significant amount of churn.
Trust Coverity by Anonymous Coward · 2009-09-23 07:14 · Score: -1, Troll

It's too bad you can't trust Coverity to actually analyse code correctly
Usability by deadkennedy · 2009-09-23 07:16 · Score: 1

I wonder if these code improvements lead to overall usability. That seems to be the biggest stumbling block for open source, not stability.
fp gn4A by Anonymous Coward · 2009-09-23 08:07 · Score: -1, Redundant

if you don't that has grown up do and doing what BSD's filesystem the hHard drive to words, don't get pRopaganda and guests. Some people Troubled OS. Now
FrosT pist by Anonymous Coward · 2009-09-23 09:02 · Score: -1, Flamebait

a child knows Perform kkeping you down. It was you join today! are a pathetic WASTE OF BITS AND the above is far
Well, since there has been progress... by Anonymous Coward · 2009-09-23 10:54 · Score: 0

... maybe complimentary is in order, too.
Fixing issues improves code... by MtHuurne · 2009-09-23 12:04 · Score: 1

From the press release:

Since 2006, more than 11,200 defects in open source programs have been eliminated as a result of using the Coverity Scan service.
While this is good for open source and demonstrates the value of static analysis, it is not surprising that if you fix the issues found, the number of issues remaining will go down.
1. Re:Fixing issues improves code... by chromatic · 2009-09-23 17:38 · Score: 2, Informative
  
  If you fix the issues, Coverity moves the project to a new rung and performs stricter analysis to find more types of errors.
  
  --
  how to invest, a novice's guide
1/4000LOC but WHERE? by gzipped_tar · 2009-09-23 14:23 · Score: 1

The bug-per-line count doesn't really give you a reasonable measure of product stability. A bug in the hotspot code is far likely to be triggered by the end user than one in the rest of the software is.
So why not correlate the bug distribution with profiling data? I don't think this should be too difficult. You don't even need to do the profiling yourselves; when you obtain the source code just ask for the data from developers.

--
Colorless green Cthulhu waits dreaming furiously.
Teaching to the test by ChrisDolan · 2009-09-24 01:28 · Score: 1

I love Coverity. I love other static analysis tools too -- I'm one of the lead developers for Perl::Critic, which performs static analysis on Perl code. They are enormously valuable tools.
However, I've seen many cases where people read the issue report from the tool and fix the symptom rather than the problem. The improvement from 1 in 3333 to 1 in 4000 is fantastic, but that means 1 *Coverity issue* in 4000, not 1 *bug* in 4000 lines.
My current closed source project has a Coverity count of 2 issues in 150,000 lines of code as of today. Does that means it's less buggy than Samba? No, it's just different. We've simply removed the majority of the easiest-to-automatically-detect bugs.