Slashdot Mirror
Software Code Quality Of Apache Analyzed
fruey writes "Following Reasoning's February analysis of the Linux TCP/IP stack (putting it ahead of many commercial implementations for it's low error density), they recently pitted Apache 2.1 source code against commercial web server offerings, although they don't say which. Apparently, Apache is close, but no cigar..."
2.1 is'nt even out yet! the latest is 2.0.46!
Cats: All your base are belong to us.
Captain: Take off every sig !!
Reasoning found 31 software defects in 58,944 lines of source code of the Apache http server V2.1 code.
so what are the calling a defect?
And don't most NDAs for when they do let you look forbid any competetive analysis?
Or am I just too far out of that line of work to know how these things work?
He tried to kill me with a forklift!
I know that Apache has vulnerabilities but it should come better than IIS. You can't realisticly give a verdict on IIS without looking at the libraries called.
As for the rest, I can imagine some commercial products coming in better, but not many.
See my journal, I write things there
So?
There are errors and there are errors. There are error that don't matter a jot, and there are errors that are show-stoppers.
I've worked on banking software containing code that was written in assembly for PD11s and developed over decades. The most horrible spaggetti code you could ever imagine. Why did the banks keep using it? Because for any particular input it always gave the correct output.
Years of bug fixing had made the code horrible and probably full of errors if you were looking at it from a purely theoretical/software engineering viewpoint. But from an input/output point of view, it was faultless.
Since when are unfounded results from a company that doesn't explain what the "32 defects" were, newsworthy. Don't act like these guys are worth my time, this is bullshit.
Ignore the "p2p is theft" trolls, they're just uninformed
By its very nature, Open source will tend to fix important bugs and leave unimportant ones unfixed, while standard QA processes associated with commercial software will tend to fix little UI issues during the release schedule before dealing with vulnerabilities.
So seems pretty clear to me that in Open source, the ratio of showstopper bugs to miscolored widget bugs will be much lower than for commercial software.
Agreed. Things like splint often report "warnings" on code that shouldn't be. For instance
int some_func(char *somebuf)
{
if (somebuf == NULL) return ERROR;
somebuf[0] = 'a';
return OK;
}
Will generate a warning with splint saying "pointer may be null" despite the fact it cannot be.
Those tools are generally too sensitive and give too many false positives to be useful in the long run.
Tom
Someday, I'll have a real sig.
Slashdot's summary of this article is way off base, and the article itself couldn't be less useful. Counting the number of "errors" in lines of code... and the ratio is supposed to mean something to us? As compared to unnamed other software? C'mon, I have better things to do with my time.
*plonk*
A hen is only an egg's way of making another egg. -- Samuel Butler
One of the best ways to get to know a large code base like Apache or something else is to find a repeatable bug and track it down. To fix a bug you do not need to understand the whole program, just the relevent parts. I've submitted bug fixes to several projects, so I must strenuously disagree, especially because, ahem, I have never submitted a bug fix to a proprietary project because its impossible.
First, as many posters have noted, Reasoning DID NOT TEST APACHE 2.1. They tested Apache 2.1-dev. That's dev, as in development branch. As in: I have new untested code, so don't use me on a production server until I'm released in the STABLE series.
For a valid comparison versus commercial software, the testers should have used Apache 2.0.46, the most current STABLE series release.
Second, I'd be interested to see a comparison of 2.0.46 versus 1.3.27. I have a pet theory that multithreaded C code has more bugs than single-threaded C code, and I'd like to see whether there is evidence to support it.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
You have the wrong idea here. There is a point in which you must realize what information you can release without comprimising the security of your system. While I can give you the plans to my vault, I will not give you the combination, nor the first or second numbers in it.
For the star wars geeks out there, if you were a Jedi, you don't go around telling everyone you're a Jedi, nor do you flash your light saber in public places. They do realize when to show their light saber, and when they can tell people they are a Jedi. Nor do they not tell anyone who they are, or never show their lightsaber.
You might want to check out Secrets and Lies which will give you a better understanding of security philosphy.
Every Super Villan uses Linux.
!!conf->providers => conf->providers => conf->providers != NULL
Their program has detected "defects" where there are none. Perhaps the greater coding style variation on open source projects exposes more defects in their automated program!
DROS - Open-Source Robot Software
One word: architecture.
And not just the architecture of the web server, but the architecture of the entire platform. But specifically looking at the architecture of Apache versus the architecture of IIS, you'll immediately see that the goals of the two pieces of software are not the same. Look at things like IIS's metabase - the structural details of the server's configuration are kept in an in-memory data structure, which is easily modified while the server is running. Apache, in contrast, reads its configuration at startup, and uses it to determine which modules of code are loaded, and how they are used to process requests - fixing the behavior of the web server at startup.
IIS follows typical MS enterprise software design - it has to interface with COM, and the NT security model, and active directory, and the registry, and a million other systems, all in the name of integration, and enterprise management. Apache doesn't have PHBs telling it that it needs another way for the metabase to be edited, or a new instrumentation API, or whatever else a particular large customer asked for - and can get on with just providing its facilities cleanly.
That's why IIS has so many more security holes, even if it does (as may or may not be the case) have the same raw coding error rate as Apache.
Am I the only one who looks at reasoning's results with suspicion (even when I agree with them). Any analysis using methods that are not open and repeatable is not science. This just feels like marketing to me. (it is sad because the study of code quality is such a worthwhile pursuit)
they dont say what they used for a comparison.
when they tell us what they used, then I will believe it.
this smells microsoft.
bring it on! we want to know what it was compared against, sure as hell was NOT IIS...