Slashdot Mirror
Linux Has Fewer Bugs Than Rivals
sushant_bhatia_progr writes "Wired has an article stating that according to a four-year analysis of the 5.7 million lines of Linux source code conducted by five Stanford University computer science researchers, the Linux kernel programming code is better and more secure than the programming code of most proprietary software. The report, set to be released on Tuesday, states that the 2.6 Linux production kernel, shipped with software from Red Hat, Novell and other major Linux software vendors, contains 985 bugs in 5.7 million lines of code, well below the industry average for commercial enterprise software. Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis. Commercial software typically has 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University's CyLab Sustainable Computing Consortium. This would be equivalent to 114,000 to 171,000 bugs in 5.7 million lines of code."
I think they mean "40 million lines of bugs" :)
...but while they were going through all those 5.7 million lines of code, would it really have killed them to debug them while they were at it??
Love the Third Amendment?
How can one be sure about closed source kernels like Windows XP. Even though I agree that it has to contain more bugs, without the actual source how can any one make any judgement in this matter?
"In questions of science the authority of a thousand is not worth the humble reasoning of a single individual."
The problem is that there is very often little vested interest in fixing bugs in closed software...if it can be covered up, then so be it. In open software, there's always a reason, even if it is just to keep people from pointing at your code and laughing.
Talk about misleading stats...
The Windows XP code base includes all of the extraneous crap that gets bundled with and on top of the kernel.
The "Linux" code base just includes the kernel.
First off, what does this statement mean?
"[Linux has] 985 bugs in 5.7 million lines of code, well below the industry average for commercial enterprise software. Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis."
So Linux has 985 bugs. Windows has bugs that appear frequently. Ok that doesn't really tell me anything. I tried to dig a bit deeper and came up with: "Coverity has not analysed the source code to Microsoft Windows because the company does not have access to the source code, Hallem said. Apple Computer's Mac OS X has a great deal of proprietary programming, but the core of the operating system is based on BSD, an open-source operating system similar to Linux."
So everything is based on estimates. Now, you know and I know that the Linux kernel has less bugs... but this is a tentative (at best, shoddy at worst) way of presenting that idea.
Small potatoes make the steak look bigger.
What about if you throw in KDE or GNOME, Mozilla, etc, everything that you'd have to add to really equal the features of Windows XP....
A rate of 20-30 bugs per 1000 lines would render most programs unusable.
:-)
:)
Sounds like Windows to me!
It's a joke, laugh.
In proprietary closed code, it's impossible for an outside party to determine the number of bugs.
The best you can hope to do is to count the number of *symptoms*, but that's not the same thing as bugs. It's possible, even likely, that a single bug in the code manifests itself as 100 apparently different external symptoms.
So this comparison is apples to oranges and completely meaningless.
Update: 18 hours after posting the study, the Linux kernel team had eliminated all the bugs documented in the study, forcing the researchers to correct the bug count down to 0 per 10,000 lines.
Assorted stuff I do sometimes: Lemuria.org
How do you analyze 5.7 million lines of code except to run it through a static analyzer? Static analyzers can't detect most errors, which tend to be data dependent. So a company selling an uberlint donates their tool, 5 academics run it and write a paper. *Blech*
Commercial software typically has 20 to 30 bugs for every 1,000 lines of code Bull. Software with 20-30 errors per KSLOC doesn't work. (I know, I just spent a year trying to save a project that had 10 errors per KSLOC. Even at that defect density (found and fixed) it was undeliverable. Notice that the link in TFA doesn't take you to anything that supports this assertion, just to the organization's home page. I could't find anything to support these numbers.
Weasely non-comparisons: 2.6 Linux production kernel, ... contains 985 bugs in 5.7 million lines of code as opposed to Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis.What, bug in Linux are found on an infrequent basis?
This is all hot air.
It's not.
There are bugs that affects applications to a level the user notices, and there are bugs that affect the maintainability of the code, the reusability of the code, and the ease of use of the code. Most bugs are hidden from the user's view.
Does a stack overflow in a piece of code that only occurs when a craftily created exploit is executed constitute a bug? Yes it does -- even when no exploit exists. Under normal operations it does not affect the use of the system -- at least not until an exploit is developed and widely used. The fact that Windows, especially the older versions of the OS, were vulnerable to so many simple exploits illustrates the bugginess of the code. And most of the bugs are not even in close enough to the surface to be so readily exploited.
Does an end user see API bugs? Does anyone outside of Microsoft experience architectural flaws in the Windows OS? No, but that does not mean they do not exist.
What about code that misbehaves when presented with certain data. That may not be a problem for the original application that the code was written for, but any time that code gets reused, the programmer must know about these "hidden features" (read: bugs). Again, the user never sees this sort of bug.
The number they came up with wasn't pulled out their ass. See this article on measuring bugs for a more detailed discussion on the topic.
When reading it, bear in mind that most commercial software is produced for in-house use, receives very little QC, and frequently does not even compile cleanly. It's usuall just "good enough" to get the job done.
the growth in cynicism and rebellion has not been without cause
I'm gonna call bullshit on this figure.
Keep in mind that you need to know the definition of a bug. It's not necessarially what you think it might be, but what the researchers defined. By their definition a condition that could never occur could be considered to be a bug. For example:
int foo ()
{
if (0)
return;
do_something();
return (0);
}
This overly-simple example could be considered to be a bug. If the condition is ever true the function will return an undefined value, but the condition will never be true so you couldn't possibly return an undefined value. It's not at all uncommon to find code with similar logic scattered throughout - improperly defined loops, conditionals, etc. could result in theoretical bugs that no path of execution can actually get to.
Then there are the kinds of bugs that only occur in extremly specific situations. About 13 years ago I had to track down a bug that caused a report package to crash. It took me a while to figure it out but eventually I did. The program would crash only on specific days. It'd only crash on Wednesdays. It'd only crash on certian Wednesdays - Wednesdays in September. Even more specifically, usually only the 3rd or 4th Wednesday in September.
The bug was that whoever wrote the code that printed a header on the reports was extremely anal about memory usage. He calculated exactly how many characters it would take for a buffer to hold the full date. The problem was he miscalculated by 1 character. With "Wednesday" being the longest day spelled out and "September" being the longest month, a 2 digit date (eg. Wednesday September 23) meant that the full date string would overflow the buffer by 1 character. This kind of bug wouldn't show up very often - only a few times a year - but it was a pretty nasty one when it did.
The article claims that "...the Linux kernel programming code is better and more secure than the programming code of most proprietary software..."
Of course it is.
Most proprietary software are user-level apps where a bug here or a bug there isn't critical. The economic loss that can be atributed to a bug in MS Word's bullet-point-algorithm isn't as great as a datacenter going down due a Oops in the linux-box running the SQL-database. To put it simply: there are different requirements for different tasks. A comparison between the quality of WinXP's GUI and GNOME for example would have been much more interesting.
I don't understand how this story could make slashdot's frontpage.
When I walked with dinosaurs (ok, IBM mainframes) as a sysprog, there was a utility program named IEFBR14.
The purpose of IEFBR14 was to do exactly nothing, and pass a zero return code to the caller after doing the 'nothing' (branching on the return address in register 14 - thus BR 14).
This was actually more useful than it sounds and was used frequently in MVS JCL (Job Control Language) to make JCL do its thing without having to run a real program in a JCL 'step'.
Thing is, this program that had to do precisely nothing, had no less than 3 patches issued from IBM. Mostly to do with not clearing R15 (the return code register) correctly.
Go figure!
How could they analyse the 2.6 kernel 4 years ago when 2.6 is less then a year old?
ReadingTFA: "The Linux source-code analysis project started in 2000"...
This is an ongoing study, to find the bugs checkout Bugzilla.kernel.org it's not like they hide them or anything
"Some things have to be believed to be seen." - Ralph Hodgson
For most proprietary software, the source is not available. So when contemplating this study the authors should have realized that it couldn't possibly be valid and decided to go do something else. The fact that they went ahead anyway, is a strong indication of their lack of dedication to scientific principles.
Sounds like Windows to me! :-)
:)
;-) <--- NOTE SMILEY. :) :)
It's a joke, laugh.
It's a sure sign that the moderators are getting stupid when you have to point out the jokes to them.
Is anyone else bothered by the idea that an article entitled "Linux Has Fewer Bugs Than Rivals" is posted that compares 5 million lines of code that make up just a kernel to 40 million lines of code that make up a kernel, an operating system, a desktop, its integrated components, and its included applications? 40 million lines has more bugs than 5 million? Who'd have thunk it?
Why does "Linux is just a kernel" suddenly not apply here? Is it because we're bashing Microsoft? This is kind of lame and makes the community look silly and vitriolic.
And yet, Linux has more security advisories than Windows each month.
I'd like to see a serious discussion of the methods and funding sources in this study. You know, the same kind of discussions that appear whenever a Microsoft study is posted. Mysteriously, that critical thinking is absent in articles like this.
40 million lines of code != 5 million lines of code. If Microsoft attempted such a skewed study, I can already picture the kinds of posts people would be writing, asking about methods, funding, and more. Those posts aren't appearing in this article. Why? Well, that's because this isn't about the quest for finding out which operating system actually has less bugs (sorry, I forgot, "Linux is just a kernel!"...funny how that oft-uttered phrase is missing in the article discussion), this is nothing more than a Microsoft-bash article intended to generate Microsoft-bashing replies.
OSTG, the corporation that owns Slashdot, makes money off of OSS products. It's rather convenient that they run a "tech news" site that just so happens to post news stories that are all critical of their competitors, isn't it? Imagine if Microsoft owned a tech news site that posted nothing but anti-Linux articles? You guys would be all over it, but your biases against Microsoft make you have a double-standard. So you ignore that Slashdot is a corporate-owned website with a major bias and an editorial staff that can barely read and write correctly, much less bother to read their own website.