Slashdot Mirror
Linux Has Fewer Bugs Than Rivals
sushant_bhatia_progr writes "Wired has an article stating that according to a four-year analysis of the 5.7 million lines of Linux source code conducted by five Stanford University computer science researchers, the Linux kernel programming code is better and more secure than the programming code of most proprietary software. The report, set to be released on Tuesday, states that the 2.6 Linux production kernel, shipped with software from Red Hat, Novell and other major Linux software vendors, contains 985 bugs in 5.7 million lines of code, well below the industry average for commercial enterprise software. Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis. Commercial software typically has 20 to 30 bugs for every 1,000 lines of code, according to Carnegie Mellon University's CyLab Sustainable Computing Consortium. This would be equivalent to 114,000 to 171,000 bugs in 5.7 million lines of code."
How can one be sure about closed source kernels like Windows XP. Even though I agree that it has to contain more bugs, without the actual source how can any one make any judgement in this matter?
"In questions of science the authority of a thousand is not worth the humble reasoning of a single individual."
From TFA
Seth Hallem, CEO of Coverity, a provider of source-code analysis, noted that the majority of the bugs documented in the study have already been fixed by members of the open-source development community.
Somehow I doubt that the XP kernel is 40 million lines of code. I know it's not good "news" but they should really compare apples to apples in their study.
This just in! "Hello world" has 0 bugs per three lines of code! Most stable and secure software ever devised!
- chrish
This is what you get for integrating your web browser into your operating system. Legality aside, there was a low cunning to that business move when M$ did it. Now, however, that decision is coming back to bite them on the tender bits: the browser is part of the OS, ergo bugs in the browser count as bugs in the OS.
//Information does not want to be free; it wants to breed.
How do you analyze 5.7 million lines of code except to run it through a static analyzer? Static analyzers can't detect most errors, which tend to be data dependent. So a company selling an uberlint donates their tool, 5 academics run it and write a paper. *Blech*
Commercial software typically has 20 to 30 bugs for every 1,000 lines of code Bull. Software with 20-30 errors per KSLOC doesn't work. (I know, I just spent a year trying to save a project that had 10 errors per KSLOC. Even at that defect density (found and fixed) it was undeliverable. Notice that the link in TFA doesn't take you to anything that supports this assertion, just to the organization's home page. I could't find anything to support these numbers.
Weasely non-comparisons: 2.6 Linux production kernel, ... contains 985 bugs in 5.7 million lines of code as opposed to Windows XP, by comparison, contains about 40 million lines of code, with new bugs found on a frequent basis.What, bug in Linux are found on an infrequent basis?
This is all hot air.
It's not.
There are bugs that affects applications to a level the user notices, and there are bugs that affect the maintainability of the code, the reusability of the code, and the ease of use of the code. Most bugs are hidden from the user's view.
Does a stack overflow in a piece of code that only occurs when a craftily created exploit is executed constitute a bug? Yes it does -- even when no exploit exists. Under normal operations it does not affect the use of the system -- at least not until an exploit is developed and widely used. The fact that Windows, especially the older versions of the OS, were vulnerable to so many simple exploits illustrates the bugginess of the code. And most of the bugs are not even in close enough to the surface to be so readily exploited.
Does an end user see API bugs? Does anyone outside of Microsoft experience architectural flaws in the Windows OS? No, but that does not mean they do not exist.
What about code that misbehaves when presented with certain data. That may not be a problem for the original application that the code was written for, but any time that code gets reused, the programmer must know about these "hidden features" (read: bugs). Again, the user never sees this sort of bug.
The number they came up with wasn't pulled out their ass. See this article on measuring bugs for a more detailed discussion on the topic.
When reading it, bear in mind that most commercial software is produced for in-house use, receives very little QC, and frequently does not even compile cleanly. It's usuall just "good enough" to get the job done.
the growth in cynicism and rebellion has not been without cause
Do you use only the Linux kernel? No. You run Slackware, or Fedora, or SuSE (my personal choice), or Gentoo, or something. A fresh copy of the latest version of a distribution is what should have been analyzed. All of the other stuff that's piled onto the kernel is what you use, such as the GUI, time-killing games, and all the other shit that a standard XP install disc contains. You don't click on line 11359 of the Linux kernel to open up Konqueror. You click on an icon, which starts a process that you use. The stuff piled onto the kernel is far more buggy than the kernel would be. I'd bet that a distribution of Linux is still far less buggy than Windows is, but it's certainly not 985 vs. 800,000. For example, when I minimize my windows on XP, there isn't hard drive activity for the next five minutes, as can happen with SuSE with not very much open. I'm sure something's misconfigured, but if a distribution, in 2004, is that poorly configured by default, then that's a bug.
HI, MY NAME IS ISAAC.
X.Org, Mesa and your choice of KDE/GNome and the tools necessary to have the same base application set as Windows has have over 5m lines of code, so it's safe to say Apache 2, Samba, etc. weren't included in this count, either, nor the GNU C compiler suite.
.NET or Java), it's possible to do so with no performance impact at all in all ISO compilers.
There's no central Linux repository for reporting bugs in ancillary packages, while at Microsoft's site, all reported bugs go into a single database that can be queried. Each individual Linux distro and each individual Linux package maintains its own bug lists, which would have to be some how amalgamated.
In order to do the stated comparison, you would need to state what distribution you were using, you would have to state which patches you were using, and you would have to document where the information on bug counts came from for both products.
On top of all this, bugs per 1000 lines of code, while an industry metric, isn't a valid measurement of code quality, either, by the way.
One good code metric is the number of control points per function. The more control points per function, the more likely that you have a bug. It's always possible to take a complicated function and break it up into smaller pieces, and with C/C++ (but not
Another good metric is the distribution of severity of the reported issues. Draw a bell curve of severity of issues, compare percentage to percentage. You're not concerned with 1000 bugs vs. 100 bugs, you're concerned about how many of those bugs compromise your system or cause a crash.
Another good metric is delta bugs over time. There will always be bugs, everyone knows that there will be bugs. The question is the bug count going up or down.
Another good metric is delta time between open and close. Again, there will always be bugs, but how fast they are getting closed is a measure of whether a product is good or bad.
Another good metric is distribution of the number of days that bugs have been open and reported (by severity).
The reason for this is that the number of lines to implement a given function using a given algorithm varies between programmers and programming styles. Some will use two lines or statements to do what can be done in one, to make code easier to read. The line number is something that a developer can manipulate easily.
However, these other measurements are tied directly into the quality of the code. Nobody cares how big the Linux kernel is compared to the Windows kernel, or vice versa. What they care about is how well the Linux kernel works compared to the Windows kernel.
Any count related to bugs, also, needs to take into account the fact that on Windows, you have billions of users any of whom could find and report a bug. On Linux, bugs are more likely to go undiscovered for a longer period of time, simply because there aren't as many people trying to hit them.
The Windows and Linux kernel tend to be very similar in bug counts. The kernels of both OSes tend not to have bugs, because kernels tend to have simple code that's hard to mess up in any meaningful way.
It's only when you start including all these ancillary subsystems, device drivers, etc. that you start to see significant percentages of the bug.
And it is exceptionally hard to get an accurate count of those on Linux. On vMac, I documented approximately one bug in ten that I fixed, and I think that would be typical of other open source projects (although I can't swear to that).
I think that on these other, valid measurements that aren't dependent on lines of code, that if you could collect the statistics on a package per package basis, and compare them, that Open Source would still come out ahead of Windows.
It's just a matter of using a meaningless metric on widely divergent code bases to prove any point is irresponsible, and by reporting it, the media is doing a disservice to the Open Source community by perpetuating the image of the community as a bunch of college students who don't understand real development practices.
If your code is acting bloated, and is running rather slow, it's likely and predicted that some loops you will unroll.
Is anyone else bothered by the idea that an article entitled "Linux Has Fewer Bugs Than Rivals" is posted that compares 5 million lines of code that make up just a kernel to 40 million lines of code that make up a kernel, an operating system, a desktop, its integrated components, and its included applications? 40 million lines has more bugs than 5 million? Who'd have thunk it?
Why does "Linux is just a kernel" suddenly not apply here? Is it because we're bashing Microsoft? This is kind of lame and makes the community look silly and vitriolic.