Slashdot Mirror
Firefox Analyzed for Bugs by Software
eldavojohn writes "In a brief article on CNet, a company named Coverity announced that Firefox is using software to detect flaws in Firefox's source code. Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects." An interesting tidbit from the article: "Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. The buggiest program is the Advanced Maryland Automatic Network Disk Archiver, or AMANDA, a Linux backup application first developed at the University of Maryland. Coverity found 108 bugs in its 88,950 lines of code, or about 1.214 bugs per thousand lines of code." We've covered this before, only now Firefox is actually licensing the Coverity software and using it directly.
I will definitely take another look at Coverity's products, if the Firefox team is finding value in it.
Why can't I mod "-1 Idiot"?
there are already multiple projects who have brought there bugs down to zero.
You mean "who have brought down the count of their bugs that this tool can detect down to zero." I'm sure they will have other bugs in code and design.
How does this tool compare to tools that do analysis by introspection on bytecode from languages like C# and Java. I use FxCop on C# code, and while it is very cool, using it is not newsworthy at all. Does this tool do more? Is is the news that it's used in a high-profile C++ program?
Integrating tools like this into your build process may be cutting-edge best-practice at present, but give it a while.
My Karma: ran over your Dogma
StrawberryFrog
> I hope these Coverity guys aren't pompous enough to think that their tool can find ALL bugs in a program with... magic...
I am sure that they know their tools limitations, but I am pretty sure that others will interpret
no outstanding bugs as if the application is secure or bugfree. Ethereal (now known as wireshark) has
a very low bug count, but I will not use it due to numerous past remote exploits coupled with
little interest in fixing bugs contra adding new features.
> Hmm, they should run their tool on its own source code, that would be fun.
I would be very surprised if they did not.
In this age of SarbOx and risk management there is a real competitive advantage to F/OSS over proprietary code to large companies: audit-ability. In previous roles I've had to attest under HIPPA::Security that proprietary code was "secure" -- how? All I could do was obtain a vendor statement that was as non-commital and burden-shifting as possible. Yet, with a true ability to audit the code my pharmaceutical company depended on it would tilt the balance between similar-featured Closed vs Open source solutions. Especially today.
Ok, maybe nobody really cares about the 'many eyes' theory anymore. Regardless, the "open the hood" theory still applies, perhaps more than ever.
-- @rjamestaylor on Ello
Uh. Coverity TOLD the amanda folk, bugs were fixed, now amanda's fixed all coverity bugs found ?! Score -1 FUD...
AMANDA could easily be the buggiest OSS program in existence, and it would still be OK. The reason? It just has to be less buggy than Netbackup, and more usable than Legato. Luckily for the AMANDA developers, this are very very difficult criteria to miss.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
And I'm assuming that they mean "Mozilla is using Coverity..." or "Firefox developers are using Coverity...". After all you don't hear about what Internet Explorer is doing, but rather what MS are doing with it.
Wouldn't it be great if the summary was clearer and neither of us had to make mental amendments?
It is not possible for a program to analyze another program and find all the bugs; see halting problem .
Wrong. It is quite possible to analyze a program and find all the bugs that violate the language constraints (null pointers, buffer overflows, etc.). That's what program verification is for. For some programs, you can't tell whether a bug condition will occur, so you treat that as a bug.
Automated program verification is a good idea that went away because C and C++ have such ambiguous semantics. It's hopeless for those languages. The "pointer equals array" concept alone makes it very tough, because the language has no idea how big an array is. Worst idea in the language, and the root cause of buffer overflows.
Good verifiers were written for Pascal (I headed one of those projects), a good one was written for Java (at DEC, just before DEC went under), and Microsoft is working on one for C#.
As an object oriented programmer, I always follow the general rule of having a function always give the same output for the same inputs. That is, you then don't have to worry about the 'state' of an object and you as a result have fewer paths to test and fix. This is why, IMO, global variables aren't such a good thing unless they are constant/rarely change.
This should be common knowledge to a good object oriented programmer, but I wonder how often it's employed in the 'C' discipline.
A function that always returns the same value given its inputs is part of functional programming, not object-oriented programming. Most OO code is littered with side-effects and state-dependent behaviour. If you like to program in such a way, you may find yourself much more comfortable with a functional programming language. Languages like Haskell even enforce this.
"Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs"
What a silly reason! How about gzip etc then?
"gzip sounds like a scam. It is not possible for a program to analyze any data and always compress it successfully"[1].
I could go on: "life sounds like a scam..."
But I suggest you wake up to the harsh imperfect real world some time and leave that sort of thinking to the run-of-the-mill "academics".
How you deciding whether Coverity is good or not should be like how you decide whether gzip is good or not. If Coverity doesn't find bugs better than even gcc then it probably useless to most people.
[1] On a related note, in my opinion programming can be viewed as a type of compression.
XMMS, a multimedia/mp3 player was tested as part of what the article calls a "$1.2 million, three-year grant [the Department of Homeland Security] awarded to a team consisting of Coverity, Stanford University and Symantec Corp" that was setup to "reinforce the quality of open-source programs supporting the U.S. infrastructure".
40 programs were tested. 40 open source programs. Not even all the programs installed by, or regularly used on, a default install of a particular distro or two; just 40 programs. I thought maybe these 40 were just the first 40 tested, but the original announcement of the award of the grant states that 40 programs would be tested.
And yet they didn't test BIND? ssh? Also, PostgreSQL is on the results list, but MySQL isn't? Did Homeland Security put this list together?! Using a dartboard and a list of open source applications, or what?!
This seems like a great software package, and I'm glad that Homeland Security acknowledges that "much of the critical infrastructure runs on open source", but I could think of a few other ways they could've spent $1.2 million, or at least a few other applications they should've tested before they got to XMMS.
Surely a program having very few bugs is reason for it to be stagnant, very little work needs doing in the way of bugfixing and the program already provides all the features it needs (i personally hate programs which suffer from feature bloat - pointless features added for the sake of it, that usually result in the core program not doing it's original task as well as it used to)
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
"Coverity was also run on the Windows source code. Unfortunately, the 32-bit integer iterator in Coverity was 1 count too small to store the count of the number of bugs found, and so Coverity's counter rolled-over, showing that Windows actually has -2,147,483,648 bugs. Microsoft employees were ecstatic at the results, and Steve Ballmer was said to be seen dancing in his office, yelling 'developers, developers, developers, developers!!'."
Is Capitalism Good for the Poor?