Open Code Has Fewer Bugs

ganns.com writes "Reasoning, which sells automated software inspection services, scrutinized part of the code of Linux and five other operating systems, comparing the number and rate of programming defects. Specifically, Reasoning examined the TCP/IP stack and found fewer errors in Linux. 'The open-source implementation of TCP/IP in the Linux kernel clearly exhibits a higher code quality than commercial implementations in general-purpose operating systems,' the company said in a report released last week. Reasoning also compared the code with that used in two special-purpose networking products and found it superior to one of them."

Comments

But aren't TCP/IP stacks mostly BSD?

[mccalli]

'The open-source implementation of TCP/IP in the Linux kernel clearly exhibits a higher code quality than commercial implementations in general-purpose operating systems,'

Really? But I thought most commercial OSes derived their TCP/IP stacks from BSD code in the first place. And since BSD is open-source, shouldn't these commercial OSes show roughly the same level of quality then? Or are they arguing that the Linux TCP/IP stack is superior to the BSD one?

Cheers,
Ian

Bah.

[KefkaFloyd]

I find the fact that they did not say what OSes they compared to be very... suspect. What about Mac OS X, FreeBSD, and other open source OSes that have Open Source TCP/IP implementations in their kernels? Since they did not say what OSes are being used...

"Reasoning declined to disclose which operating systems it compared with Linux, but said two of the three general-purpose operating systems were versions of Unix."

How lame. For all we know, they could have tested the Amiga OS, Mac OS 9, Windows 3.1, A/UX, and NeXTStep! Other than this, the article is pretty vague and does not seem to give me much meat on the subject, nor a link to the study (you have to go through some forms and give up personal info to get it at www.reasoning.com).

Re:Statistics

[Xtifr]

This is not the first such study, there was a paper published in the early nineties which tested various standard unix command-line tools from a variety of vendors. They subjected the tools to horrendous stress and abuse, and found (to their suprise) that the GNU tools were the most reliable, with approximately a 1% failure rate in their bank of tests. The second best was HP, with about 8% failure rate, and everyone else was between 12-20%.

I don't have a link, but the paper was pretty widely publicised at the time, and should be fairly easy to track down. It was the first major study to really show an emperical link between openness and reliability, but it was far from the last. This latest one is merely one more in a long list.

Is age the key factor?

[realnowhereman]

This is still an argument for the open source method, but I think that the code quality should be attributed to a different source. Perhaps it is not about an inherently good or inherently bad method. What if age is the key factor?

The Linux networking code has been in for a long time. Not in it's present form, obviously, but each change builds on the last; as it must in open source - it would be foolish to start afresh when you have something that works. So a cylcle develops and at each stage the code gets better. Compare this with proprietary; can they look at a competitors code? No. They must start afresh and so their code is effectively younger.

Further, if we measure software age not in units of time but in units of updates, open source has the advantage that there are many updates, there is always someone new to look at the code. No company can compete with the sheer quantity of viewings and therefore updates that occur in open source developments.

Or...

[intermodal]

People coding something because they want to (and because they need it for something for themself) leads to better code. I know when I do something for myself, I don't half-ass it.

Coding for the end result = quality

Coding for a living = paycheck

Any questions?

Re:Yes, but the code has diverged.

[Bodrius]

Aye. It could be that the TCP/IP stack that the article mentioned has "flourished" (become better software) because the people who develop it are VERY MUCH using it.

Linux geeks grok TCP/IP networking, and Linux users DEPEND on TCP/IP (not 'it would be nice to have web access and surf porn while I type this memo') for practically all of its market share. Like gcc, TCP/IP is part of the Linux deal.

It would be biased to regard this as conclusive evidence of the superiority of open-source unless other, less sexy areas of Linux development are compared to their commercial counterparts in the same way.

As evidence that certain commercial companies have not put priority on the TCP/IP stack of their OS, this could very well be good evidence.

But this doesn't necessarily mean the commercial companies are inferior; they may very well be right in having different priorities.

For example, for a Windows user it's more important that the Media Player works perfectly than having an efficient TCP/IP stack. Even on the server side it's not a big issue on their market. It's under so many layers of software, appearances and priorities that their clients would never notice if they made it better anyway.