Slashdot Mirror
Firefox Analyzed for Bugs by Software
eldavojohn writes "In a brief article on CNet, a company named Coverity announced that Firefox is using software to detect flaws in Firefox's source code. Even more interesting is the DHS initiative for Coverity to use this same bug detection software on 40 open source projects." An interesting tidbit from the article: "Most of the 40 programs tested averaged less than one defect per thousand lines of code. The cleanest program was XMMS, a Unix-based multimedia application. It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code. The buggiest program is the Advanced Maryland Automatic Network Disk Archiver, or AMANDA, a Linux backup application first developed at the University of Maryland. Coverity found 108 bugs in its 88,950 lines of code, or about 1.214 bugs per thousand lines of code." We've covered this before, only now Firefox is actually licensing the Coverity software and using it directly.
That's .051 bugs per thousand lines of code for XMMS, an order of magnitude better.
"Firefox is using software to detect flaws in Firefox's source code."
Didn't they always use software of some sort, Bugzilla, etc? I assume they specifically mean "Firefox is using Coverity software to detect flaws in Firefox's source code."
VOTE!
If this is the same as most automated testing software I've seen, it detects many things which aren't truly bugs as bugs. Accuracy on automated testing tools I've been exposed to is around 40%.
I will definitely take another look at Coverity's products, if the Firefox team is finding value in it.
Why can't I mod "-1 Idiot"?
I hope these Coverity guys aren't pompous enough to think that their tool can find ALL bugs in a program with... magic...
Hmm, they should run their tool on its own source code, that would be fun.
if you look at the coverity site ( http://scan.coverity.com/ ) you will see that there are already multiple projects who have brought there bugs down to zero. samba being on of the earliest.
I find the AMANDA results interesting because AFAIK it hasn't recieved a code rewrite since the early 90's. I think an interesting study would be the to compare older projects with ones that have been rewritten from the ground up. Comparing the rate of new bugs introduced as opposed to those hidden in legacy code.
If an officer ever threatens to taze you, say you have a pacemaker.
"It had only six bugs in its 116,899 lines of code, or .51 bugs per thousands lines of code."
Sounds like someone needs to run this debugger on their calculator.
Or that job is left for the monkeys banging on the keyboards.
But what checks Coverity for bugs? Coverity? What if it has a bug-checking bug causing it to not see its own bug.
do {print "Mini-Geek Rules!\n";}
until ($TheEndOfTheWorld);
Amanda works on many unix and unixoid operating systems, it's not a "linux" backup system. It's used primarily for driving remote backups to big tape libraries, most /. reading linux users would never have systems large enough to justify its use. :-)
Amanda IS, however being very actively developed right now, lots of new features -> lots of new bugs. Other issue is that it's a componenty, plugin architecture, made of a few processes communicating over pipes and sockets. A failure in one component won't necessarily be a security risk or take the whole system down, it's extremely robust in normal operation in my experience, despite this "high bug count". Unlike XMMS, various contributed plugins (e.g. tape changer robot drivers) are redistributed in the source tarball but only used by very small numbers of people with outlandish hardware.
I suspect if you included various XMMS plugins in the XMMS count, things would be different...
None of that *really* excuses a high bug count - but what really pisses me off is coverity's "we've found X bugs, but we're not going to tell you what they are or substantiate our claims (some of amanda is quite old code, has a lot of strcpys, I know that some automated security checkers will treat a strcpy as a "bug" even if it's safe), just FUD your project in various public fora...
One has to wonder if these are coding/language bugs or logical bugs. Finding coding bugs is of course a valuable time saver, but the challenging and usually most costly bugs are of the logical sort, and invariably app specific.
In this age of SarbOx and risk management there is a real competitive advantage to F/OSS over proprietary code to large companies: audit-ability. In previous roles I've had to attest under HIPPA::Security that proprietary code was "secure" -- how? All I could do was obtain a vendor statement that was as non-commital and burden-shifting as possible. Yet, with a true ability to audit the code my pharmaceutical company depended on it would tilt the balance between similar-featured Closed vs Open source solutions. Especially today.
Ok, maybe nobody really cares about the 'many eyes' theory anymore. Regardless, the "open the hood" theory still applies, perhaps more than ever.
-- @rjamestaylor on Ello
Coverity segfaulted whilst auditing MS Vista.
Here are some links to show the bugs in the Bugzilla database which were turned up by Coverity.
Open Coverity Bugs
All Coverity Bugs
static analysis only gets one so far. generally it's the higher order logic that gets reviewed in code reviews, I would hope, though.
I have to admit I am not upto speed on the best C and C++ testing suites, but is Coverity's product the best for this kind of thing?
I know that Parasoft has a solution in their product Insure++, which gives runtime analysis, not the static code analysis that Coverity appears to give from my short look at their website.
Any one else with better knowledge on these types of products wish to comment further?
Looks like somebody failed troll academy ;)
Unless the program's domain is restricted to context-sensitive languages. In fact, it is impossible for a computer to try to decide anything more general than a context-sensitive language because anything bigger requires the resources of a Turing machine, which has infinite memory. Computers implementable in a finite amount of matter are equivalent to linear bounded automata, not Turing machines.
AMANDA could easily be the buggiest OSS program in existence, and it would still be OK. The reason? It just has to be less buggy than Netbackup, and more usable than Legato. Luckily for the AMANDA developers, this are very very difficult criteria to miss.
"People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
Wait! Is that the source code?!
the mods may say you posted flamebait, but to me it's a flame that warms my heart. rock on, brother! --chebucto
And thank god for that score. I don't mind my AMANDA backups being corrupted, but if my mp3 of "Hit Me Baby, One More Time" ever skipped a beat, I wouldn't know what to do...
Coverity's own site shows how many defects each product has fixed. the number of outstanding defects on AMANDA is now zero. zdnet reported the fixes back in April.
Those that follow amanda-hackers will know that there was less than a week between when coverity released the report on March 6th and it was announced that all bugs were fixed in AMANDA on March 12th.
Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs; see halting problem.
I would find heuristic analysis annoying. I'd get quite annoyed if the program says "fix this buffer overflow" 1000 times because I use "strcpy" somewhere - even though I'm very careful and only use it when I know it can't overflow.
I should write a program that searches for odd perfect numbers and terminates if it finds one. I wonder whether Coverity would say it is an infinite loop.
Coverity sounds like scare tactics to make money by claiming to do the impossible. They won't even disclose what their algorithm is. I would never trust them, especially on closed-source programs. Firefox doesn't have that risk, but they are wasting money.
Microsoft's PREfast is simpler but seems like a much more realistic solution: mark up your code to say how things are supposed to be used and the compiler can decidably sense problems. I'd just get tired of typing 2 underscores a million times.
Melissa
"Screw Sun, cross-platform will never work. Let's move on and steal the Java language." - Visual J++ Product Manager
And I'm assuming that they mean "Mozilla is using Coverity..." or "Firefox developers are using Coverity...". After all you don't hear about what Internet Explorer is doing, but rather what MS are doing with it.
Wouldn't it be great if the summary was clearer and neither of us had to make mental amendments?
zmanda and other amanda hackers have been actively developing AMANDA. While the comparison of bugs in new code and legacy code might be interesting, one wouldn't really see this by just counting projects.
I took a look at the bugs and funny enough, almost all of these would be immediately catched by the C# compiler or would be non issue (memory leaks). The remaining others (and much more) would be detected by FxCop. And then, there is still Spec# ! Like someone said before, the real hard bugs are logical. So really, one question : how is this newsworthy ?
The only thing that this teach me is that a language/platform that allow better typing, memory management and static analysis is far far more robust and productive in the end.
Intelligence shared is intelligence squared.
They built their tool on gcc, and never gave any source back to the community. "But they didn't distribute it", you say. Really?
All this ''helping open source'' is just their giant guilt trip.
Yawn. Microsoft has been doing this for years already.
Maybe if Firefox were written properly the first time around, blah blah blah.
The only thing that this teach me is that a language/platform that allow better typing, memory management and static analysis is far far more robust and productive in the end.
Haskell.
Funny selection of programs; I don't see rsync on the list. From the article: DHS wants to reinforce the quality of open-source programs supporting the U.S. infrastructure. So, XMMS (an MP3 player) is more important to the U.S. infrastructure than rsync?
I'm somewhat surprised to see amanda being badmouthed here by this tool. It was mentioned on the amanda-users list a few months back that the amanda tree had been checked by coverity, and the 2 bugs coverity found were promptly fixed.
Thats not to say that as new features are added, new bugs haven't been too, but to actually call amanda a truely buggy application does stretch this users belief a wee bit. I'm currently running a 20060424 dated snapshot of the 2.5.0 tree, with no hiccups at all.
--
Cheers, Gene
You say this as if it invalidates his point. Since (as you would obviously agree) no computer is more powerful than a Turing machine, if something is impossible for a Turing machine it is necessarily impossible for a computer as well. If anything, your quibble makes his argument stronger.
--MarkusQ
can they run these programs on the source of the program itself to look for bugs? Or would that be like the human brain being able to completely understand itself inside and out (aka. not possible)
Life is rarely fair. Cherish the moments when there is a right answer.
I agree... can't get enough of the Children of Bodom!
I just care for one thing that mozilla has been ignoring since FF 1.0.x...
Will this make Firefox any faster?
-----BEGIN PGP SIGNED MESSAGE-----
e ssage.php?msg_id=20590002
p XRN2Cf5Pp0KOrhNb6GHN9kgEBMMc4I JPQgkEmhT4RBXslkSAjMfYZ1Xg6DEuo9jwKt//RcScL5U2 Aueo2TiRlXoo0bHRujATgzP+U29nLIXIM4EdtEJo99vH2eg1HZ ka82hNtpN7u1 8bBkQP6sLEAMrHz/XGKkuqRR+dG9 Fn8UvI55iDW07pQVy2fC+YDJ7ZQnIwpfANUEdnWbRuVn xPrOwm2CMaRZsNZq8odK1pTZCxHsK+zjYL3wkmUvLCyr+Z8eA= =
Hash: RIPEMD160
Melissa wrote:
> Coverity sounds like a scam.
I used Coverity to scan for bugs in the development release of imap-uw
and in nfs userspace components a couple of months back. For instance:
http://sourceforge.net/mailarchive/m
Most turned out to be legitimate bugs that would likely remain unfixed
today were it not for my use of the static code analysis tool.
Mike
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iQEVAwUBRN4N3NtAhTFtyodpAQPL4gf9Gp
ShVXSYsAQk2kcPnz97
da
2AF791miIEpRY5vaKxnT09juoXKrgXoXx0+7
SFRnuEUcZr16frHR4L7C
eHyP
=fZMZ
-----END PGP SIGNATURE-----
(drop the space in ``message'' to follow link and verify sig)
props! No wonder I keep always using it, after trying all the other audio media players out there. It "just works".
"Coverity sounds like a scam. It is not possible for a program to analyze another program and find all the bugs"
What a silly reason! How about gzip etc then?
"gzip sounds like a scam. It is not possible for a program to analyze any data and always compress it successfully"[1].
I could go on: "life sounds like a scam..."
But I suggest you wake up to the harsh imperfect real world some time and leave that sort of thinking to the run-of-the-mill "academics".
How you deciding whether Coverity is good or not should be like how you decide whether gzip is good or not. If Coverity doesn't find bugs better than even gcc then it probably useless to most people.
[1] On a related note, in my opinion programming can be viewed as a type of compression.
So, while absolutely true that a proprietary vendor can run the code checker on their code as well as an open source project, there is a huge difference when it comes to the customer/user of said software: with Open Source the user has the freedom to run such a tool over the source code themselves.
Actually, I would argue that it isn't just a freedom, it's a necessity. Having the source open means that wrongdoers can use bug-seeking programs to find exploits (presumably they have already been doing so for a while). So just to even things out, the Open Source community should scan them as well. This issue shouldn't be ignored.
Of course, closed-source programs are also being scanned by exploit-seeking software (it's not too hard to e.g. search for all calls to strcpy in a binary). So this isn't a 'new' problem with open-source projects. But, with the advantages of Open Source come a few risks, which we should deal with, as mentioned in the previous paragraph.
We were using it since it was the Meta Compiler. I believe we had some interns from the project. They used our codebase to research their algorithms and we got free scanning. We may well be using the Coverity commercial code today.
Firefox is, once again, the most unstable program in common use.
The 1.5.0.4 version of Firefox was quite stable, if the Flashblock extension was installed. The 1.5.0.6 version is unstable again. The CPU-hogging bug is back!
This comment posted from a copy of Firefox that is constantly using 2.8% of the CPU, even when all pages have been loaded, and there is no active content. That's 2.8% on the way to 70% or more, making it necessary to close Firefox and reboot Windows XP.
There are some bugs found by Coverity left unfixed, but so far things have gotten worse since 1.5.0.4, not better.
The halting problem is not an issue for program verification. This claim is raised repeatedly by the clueless, and it just isn't an issue.
Yes, you can construct a program that's formally undecideable. It's a hard way to write a bad program. It takes some work, and the resulting program is unlikely to be useful.
Most crash-type and security-hole problems in programs are entirely decidable. This is because almost all subscript calculations are composed from addition, multiplication by constants, and logic operations. Those are totally decideable, and there are good decision algorithms for that problem. Only when multiplication of two variables (both non-constant) is introduced can formal undecidability appear. See Presburger arithmetic.
In fact, halting is decidable for all deterministic machines with finite memory. Either you repeat a previous state, or halt within a finite number of cycles. The decision process may be made arbitrarily hard, but that's not undecidability. True undecidability in the Turing sense requires infinite memory.
Most of the practical problems with program verification come from dealing with interactions between various parts of the program. Containing those interactions well enough that you can localize problems is constraining on the programmer. "Design by contract" languages like Eiffel try to do that, but they're not popular. Retrofitting design by contract into C and C++ has been discussed, but the proposed schemes all have holes you could drive a truck through. A big truck.
Although software work seldom uses proof of correctness techniques, there's a whole industry doing it for hardware. There was a machine-generated formal proof of correctness for the FPU in AMD's K7 processor. AMD thus avoided the "Pentium division bug".
After looking at some of the results from the Firefox sources, I see that "bugs" include unreferenced variables and dead code that never gets executed.
It looks like most of the real bugs consist of not checking return values, the worst being routines that act upon an object allocated by another routine without checking for null pointer.
Dan East
Better known as 318230.
The discussion on this bug which was eventually resolved as WONTFIX is quite interesting, IMHO.
I'll probably be modded down for this...
XMMS, a multimedia/mp3 player was tested as part of what the article calls a "$1.2 million, three-year grant [the Department of Homeland Security] awarded to a team consisting of Coverity, Stanford University and Symantec Corp" that was setup to "reinforce the quality of open-source programs supporting the U.S. infrastructure".
40 programs were tested. 40 open source programs. Not even all the programs installed by, or regularly used on, a default install of a particular distro or two; just 40 programs. I thought maybe these 40 were just the first 40 tested, but the original announcement of the award of the grant states that 40 programs would be tested.
And yet they didn't test BIND? ssh? Also, PostgreSQL is on the results list, but MySQL isn't? Did Homeland Security put this list together?! Using a dartboard and a list of open source applications, or what?!
This seems like a great software package, and I'm glad that Homeland Security acknowledges that "much of the critical infrastructure runs on open source", but I could think of a few other ways they could've spent $1.2 million, or at least a few other applications they should've tested before they got to XMMS.
Believe it!
The road to tyranny has always been paved with claims of necessity.
It does. The halting problem proof applies only to machines with infinite memory. There exists a trivial algorithm to determine whether a program halts when run on a linear bounded automaton: given a capacity of b bits and s states, run b*s*2^b steps and see if the machine has halted. This reduces an undecidable problem to an NP-hard problem. Further optimizations are possible, such as by detecting obvious cycles in the state of the machine and/or by recognizing parts of the system as regular or context-free.
But in practice, the question is not whether the program halts when run on a Turing machine but whether it halts when run on an LBA. A program for a Turing machine halts when run on an LBA if it halts normally or if it steps the cursor beyond the linear bounded portion of memory (which causes a special type of halt called a segmentation fault).
It seems silly to me that we're still looking for memory leak bugs, buffer overrun/strcpy-type stuff, and pointer dereferencing bugs. These problems have been fundamentally solved (or at least all but abstracted from the programmer) by managed code environments like Java, .NET, and others.
Why are we in the IT world still causing ourselves problems by using C/C++ in any situation except those which call for the strengths of C/C++ -- strengths which are quickly being matched by their managed counterparts.
Realtime? Embedded? Video game? Ok, use C/C++ (though even video games you could make an argument...). For everything else, there's managed code. No more memory pointer leaks (well, the hard-to-find kind caused by poor pointer management in C/C++), no more buffer overruns that aren't immediately fixable in one place, etc, etc.
C'mon, we, as the IT profession, have evolved past that. Why are we still trying to work around these already-solved problems?
I forgot to add: with many good libraries.
Intelligence shared is intelligence squared.
If I understand you correctly.
Suppose I have a 1Gb (=2^20) memory and some smallish number of states (say 2^8). Then we would need to run the program for 2^20 * 2^8 * (2^(2^20)) steps. This might take a while - if I were to start it running tonight on a nice teraflops machine (2^30 operations per second) it would be done in only about 2^(2^20) seconds. I'll start it running tonight and let you know when its done.
The CPU hogging bug only occurs when Firefox windows and tabs are kept open over a period of hours or days. (See the link for a description.)
This causes lots of severe problems for heavy browser users, like equipment buyers, for example. Buyers often visit several pages, then have to wait for information, and while they are waiting, they work on buying other items.
Hiring dozens of QA people to discover discover a broken error path that a developer could have fixed in 5 minutes is inefficant and slow. Instead, developers can find many bugs quickly by excerising the data paths of their own code. After (or even better, before) writing a new snippet of code, a motivated developer writes a unit test for it. The test is simple: given certain specific conditions, your code should return some predictable result.
Example- I need to write code that tell me which of two numbers between 1 and 10 is larger. I would need to write at least 6 simple tests that would guarantee this code will work in virtually every scenario. I would need to three tests to make sure it works when the input in valid, and another 3 tests to make sure it works when the input is invalid. Sample inputs:
After a while there will be dozens, hundreds of even thousands of these simple tests. They will not mean much alone, but chained together they can provide a very stable testing bed. By running all the tests every time you release another version, developers can be pretty sure that everything that worked in the past will still work. (see regression testing).
Anyway, I didn't mean to start a lecture on basic testing principles... This post was motivated by the claims in the article regarding code quality. I would take the claim that 'XMMS is the most bug free software' with a grain of salt; a big grain. The results of any testing procedures are only as good as the individual tests. Lets say that the above example was one of the tests that Coverity was using in XMMS. Maybe it passed, great! But what where to happen if a user input -13 instead of 12? or maybe -5,000,000,000 and 1? How about the letter A as an argument?
They don't even give us code coverage numbers, or a count of tests... just a claim. I would bet that the AMANDA project had 10 times as many tests per line of code as XMMS. That would make AMANDA the most bug free software. Oh, and on another note, i noticed some comments about IE testing. I don't use IE, but I am 100% certain that Microsoft has an internal testing framework that puts this one to shame.
There are 10 types of people in the world. Those who understand binary and those who do not.
For a company selling a software product, they seem stupidly protective of how much the damn thing is going to cost me to obtain. Try and find a price sheet on their website. It isn't there.
The less up-front anybody is about costs, the less worthwhile their product usually is. And the more variable the cost usually is (ie: as they figure out how much they can overcharge you). And no, I will not register with them for the "honor" of finding out more information. I'm guessing that it's something stupidly outrageous since the cost of running their application on a bunch of Open Source programs cost $1.2 million - which anyone with a single copy and a free weekend probably could have done for themselves.
They also don't disclose what their product actually does. So I'll join with the other voices here in calling for the need of an open-source alternative to this project - an alternative that has full disclosure about what the product is capable of and what it's going to cost you to use.
The program need to be suid or run by root (on "hostile" input") for a local exploit to be relevant.
Of course, amanda probably run as root on "hostile" input, so local exploits can be relevant.
Mozilla Firefox and many other GUI programs react to input as a stream of events. Each event handler could be considered a separate program under the LBA model. Treating things that fit outside a given model as "black boxes" (where a model will fail to find bugs) is still useful for reasoning about things inside the model.
Which is why these static analysis tools concentrate on factoring parts of the program (which is guaranteed to be context-sensitive) into regular and context-free parts that are more suitable for analysis. For instance, a lot of security vulnerabilities come from parser bugs, but luckily a lot of languages encountered by parsers are context-free.
s/bug/detected potential bug/g article.txt
you must have ment to say psudo-random, I'll forgive you this once.
psudo-random number generators always generate the same sequence (normally incrimenting the sequence on each call) based on the a seed value. often this value is a based on the current part of a second, or some arbitary chunk of memory (note, arbitary is not random), or (ive seen this one in practice) the users mouse movements. The seed value *IS* an input, and given the same input the same result always occers.
oninoshiko, pointing out the blatently obvious since 1981.
Memory leaks are a non issue in C#?
Seriously, Its a good browser but nothing drives me nuttier than the frenzied masses that are so blind in their arguments against IE, they compeletely turn their heads to the fact that Firefox has many bugs, compatibility and memory issues. Especially when you compare it to the current iterations of Opera, Safari 2 and yes even IE. IE can be a bigger culprit to spyware and such all theough thats going to change a LOT with IE7, but it sure as hell isnt as buggy in regards to memory bugs, CPU utilization etc.
Yes, because the CLR gobbles up so much memory that leaks in C# apps are barely noticeable in comparison.
"I will definitely take another look at Coverity's products, if the Firefox team is finding value in it."
Well Linus was "finding value" in Bitkeeper, and look how that turned out.
"Coverity was also run on the Windows source code. Unfortunately, the 32-bit integer iterator in Coverity was 1 count too small to store the count of the number of bugs found, and so Coverity's counter rolled-over, showing that Windows actually has -2,147,483,648 bugs. Microsoft employees were ecstatic at the results, and Steve Ballmer was said to be seen dancing in his office, yelling 'developers, developers, developers, developers!!'."
Is Capitalism Good for the Poor?
It's rare when someone turns down research money so they must be on to something.
Maybe, but i'm skeptical.
Classification is just a way to hide bullshit and prevent exposure of incompetance/mismanagement/squandering/etc/etc.
Here's a bug: The program does not terminate in a finite amount of time!
Can this bug be found in polynomial time by another program?
Well, the name Turing should ring a bell.
Generally speaking, this is an unsolvable problem (in *finite* time) on almost all programming languages. [It is trivial on non-Turing complete languages, but then those languages are limited to only certain classes of computations and aren't very useful].
So forget polynomial time.
From disputing his original claim:
You have been reduced to saying:
In other words, you have gone from claiming that it is possible to write a program that will find all bug in any program (which, the original poster and I agree, is impossible) to arguing that static analysis can find some bugs in some kinds of programs, which no one is disputing.
The problem with your "finite number of states" argument is that, if you are going to allow real-world constraints to intrude, you will run out of time long before you run out of storage space. Even a very fast machine, with a very small amount of memory, will never be able to get through even a vanishingly small proportion of its states before the universe is canceled due to low ratings.
For all intents and purposes, Turing's results apply to real computers, in so far as the actual real world limitations they face don't change the conclusions.
--MarkusQ
Multiplication can be defined entirely in terms of + and &, a subset of your "addition, multiplication by constants, and logic operations". Consider this function:
uint32_t multiply(uint32_t left, uint32_t right)
{
uint32_t result = 0;
uint32_t mask = 1;
for (unsigned i = 0; i < 32; i++)
{
uint32_t rightmask = right & mask;
uint32_t leftmask = 0;
for (unsigned j = 0; j < 32; j++)
{
leftmask = leftmask + leftmask;
leftmask = leftmask + rightmask;
}
leftmask = leftmask & left;
result = result + leftmask;
mask = mask + mask;
left = left + left;
}
return result;
}
The "for" loops can be completely unrolled because they only use constants and are never used by the code inside. The entire function then uses only the constants 0 and 1, and consists solely of the instructions "mov" (immediate and register-register), "add", "and", and "ret". Actually, on x86-32, you'd run out of registers, but it'd work exactly like this for x86-64.
"Screw Sun, cross-platform will never work. Let's move on and steal the Java language." - Visual J++ Product Manager
Unfricking believable! You obviously know nothing about this subject, so please, please keep your trap shut and let the grown-ups talk.
This is "insightful?" Speak of waiting for grownups! If the parent thinks he knows about the subject, perhaps he could enlighten the GPP (which is what the post directly above hod did, explaining the difference between this and bugzilla) rather than flaming as an AC?
How is any insight at all afforded that post? I'd have modded it flamebait. Something is broke here (besides me).
mcgrew's razor: Never attribute to stupidity that which can be explained by greedy self-interest
I am looking at analyzing code for some high-reliability applications. For many applications, it is very difficult to determine the correct outputs for a given set of inputs. Specifically, most of the mathematical proving falls victim to the fact that in the end some poor engineer is developing the specification. At high reliability levels, the engineer may not even know all the fault modes of the inputs, so cannot accurately specify the correct outputs for all fault cases.
Things get worse, if you assume you have statistical failures on each of the inputs. Then the accuracy of the outputs can become governed by how accurately the statistical model describes the type and likelihood of faults on the inputs. Eventually, you reach the conclusion that no matter what we build, somehow, somewhere, it will be vulnerable to a fault, and we are simply doing a fault minimization exercise. We can't prove correctness. We can just minimize the severity and impact of the faults.
lint/splint anyone? valgrind? this isn't fuckign news arhgfhaeghaehgh
When it was reported on digg I didn't mind because the digg users are morons but slashdot? oh wait..
I can very cheaply write a program which has no false positives, and almost as cheaply one that has no false negatives; the first will run in constant time, while the second is linear in the size of the program :-)
Xenu loves you!
Far be it for me ( Anonymous Coward ... and proud of it ) to even indirectly defend any of Coverity's self serving claims... BUT! static source code analysis tools do help find bugs. It might help some of YOU reading this ... 0.005% of the time. It might help in the world of open source where the "millions of eyeballs" might have missed a thing or two... 0.5% of the time. It might even help the brain-trust in Redmond... 0.00005% of the time. HOWEVER... for the millions of asses, sitting in front of millions of keyboards, typing in millions of lines of code around the world every day (for the banks, brokerages, healthcare, retail, etc. businesses that we all depend on)... static source code analysis WILL produce results worth a second look. And, of course, they will produce some noise.
So, if you don't like the signal to noise ratio produced by the static source code analysis tools of today... don't use 'em. Or, (with your PhD in hand) think you can eliminate all noise from the results... go for it. Or, you feel that a 1M+ line of source code review is trivial... carry on.
Coverity has scanned quite a bit more software than is listed on their web page. My product was scanned a couple of months ago. They found only a handful of bugs (fewer than 10), none of which were exploitable or even likely to be encountered by a user.
http://wiki.zmanda.com/index.php/Developer_documen tation
Amanda: Open Source Backup Software
http://www.coverity.com/news/nf_news_06_27_05_stor y_9.html
It crashes often and the user interface is a Human Interface Designer's worst nightmare.
IANAL but write like a drunk one.
An example from one of the submissions for the Coverity bugs in Mozilla where the coder who made the patch actually had the balls to set the arrogant committer prick right.
In a society that believes in nothing, fear becomes the only agenda ~ Bill Durodié