Ask About Proprietary vs. Open Source Code Quality

Scott Trappe is CEO of Reasoning, a company that has gained a certain amount of noteriety (and a Slashdot mention) by running its Ilumna automated inspection service on several versions of TCP/IP -- and concluding that the Linux version has fewer bugs than most proprietary ones. Why is this? Let's ask Scott, and also ask him any other question you can think of about software quality and how to achieve it since, after all, that's his business. We'll send him 10 of the highest-moderated questions and post his answers when we get them back.

Comments

sample size and conclusions

[tim_maroney]

How can any conclusions about the relative virtues of two development methodologies with a universe in the millions of components be drawn from a single sample, and one as small and atypical as a TCP/IP stack?

Issues behind test cases for proprietary v.s. open

[Tekmage]

One of the bigger challenges facing open source projects as compared to their proprietary equivalents is how to manage confidentiality of test cases. With companies such as Red Hat and Ximian involved, it's certainly less of an issue for their core products and projects they over-see, but there will always be cases where there is friction when the best/only person who can fix a particular problem is on the outside, unable to work with the test cases in question.

What are your thoughts on this trade-off between test case management and confidentiality as it relates to proprietary v.s. open source code development?

So if open source is so good...

[anthony_dipierro]

Where can I get the source code to these automated inspection tools?

Stupidity and Lies (Broken Metric)

[oldCoder]

The companys bug scan software looked at TCP/IP stacks from different OSes. Presumably they implemented the same functionality. The statistics given are not for the stacks as a whole, but are given in "Defects per 1000 lines of code".

Think about that.

If Stack A is 3 times as large (bloated code) but has only 2 times the bugs as stack B, then stack A (worse in all respects) gets a better grade!!!

You can halve your defect count by doubling the number of lines of code in your module. What a rip! How could so many people read and write about this and not see the problem.