Slashdot Mirror
Ask About Proprietary vs. Open Source Code Quality
Scott Trappe is CEO of Reasoning, a company that has gained a certain amount of noteriety (and a Slashdot mention) by running its Ilumna automated inspection service on several versions of TCP/IP -- and concluding that the Linux version has fewer bugs than most proprietary ones. Why is this? Let's ask Scott, and also ask him any other question you can think of about software quality and how to achieve it since, after all, that's his business. We'll send him 10 of the highest-moderated questions and post his answers when we get them back.
What is your opinion on this topic? In your experience, does having the source closed make it any harder to find bugs and security flaws?
How can any conclusions about the relative virtues of two development methodologies with a universe in the millions of components be drawn from a single sample, and one as small and atypical as a TCP/IP stack?
Very popular slashdot journal for adul
"Reasoning declined to disclose which operating systems it compared with Linux, but said two of the three general-purpose operating systems were versions of Unix"
So did you go cherry picking to find OS's that had more bugs than linux, or was it random or what?
Too often the Open vs Closed argument turns into linux vs windows, and then criteria is arbitrarily picked. Since the two OS's are designed largely for very different purposes, the comparison is by definition never fair, no matter who conducts it.
Saying that one product is better doesnt necessarily mean that the way it was created is inherently superior.
Implementing properly documented standards is something the OS community is great at, since they're all on the same page. Creating from scratch is different.
Hence, TCP/IP is rock solid in linux, yet development on the desktop crawls along in 100 different directions at once, gaining little ground.
I don't need no instructions to know how to rock!!!!
Some proprietary products like Microsoft Office partially maintain there dominance by not disclosing the details of the file format, or modifying standard formats to reduce compatability. Do you think that competetive free products would be more widely accepted if the file formats were open/standardized, or would the dominance and familiarity of the current packages would maintain their market control?
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
Because there are obvious differences in the cultural enviroment of developing open source versus proprietary software, what is you opinion what factors affect the quality of code that is produced from these two enviroments and how?
Memories become legend, Legend fades to myth, and even myth is forgotten by the time that age comes again.-Robert Jordan
OK "Your TCP stack is cleaner than theirs" but what metrics are being used? How do we know bugs in their testing software doesn't skew the numbers?
Trolling is a art,
Would it be fair to ask if you are an advocate of any particular type of software, or you merely promote use of the "right tool for the right job"?
Smokey, this is not 'Nam, this is bowling. There are rules.
Is a certain percentage of bugs that result from the interaction of two or more otherwise bug free components?
If open source has such a direct correlation to better quality, why do you feel companies are still keeping their source proprietary? Do you think that we should try and convince them to open source their code in every case, and if so, what do you think needs to happen before they can be convinced to change their minds?
Overrated Moderation: This posts sucks... because.
Do you believe the 'evolutionary' pressures that led the Linux tcp/ip implementation being better are in action in other areas of opens source activity? I can see the tcp/ip implementation getting a lot of attention from coders as linux is primarily a server platform but are less obviously important areas performing similarly?
If so, which areas do you think are benefitting and which need more community action / peer pressure to excel?
Are there any areas you think this phenomenon will never apply? (eg areas in which proprietary code will always be better)
Sometimes I think Linux takes more that its fair share of the limelight. Generally when I see some aspect of Linux compared to other OSes, I'm interested in seeing how BSD fares as well. Not so much to decide which is better, but it's interesting to see how the two do against each other, given that they're both open source projects. It seems to me that they both have many different and similar goals, and take different approaches at doing things. I'd like to see how it all adds up.
So did you take a look at the BSD tcp/ip implementation, if so, how did it compare to the rest?
If you didn't, why not?
-kidlinux.
It sounds like your company focuses on analyzing the code bugs, and not necessarily the perceived bugs. What are your opinions on this? I know that locating and eliminating the bugs *is* a critical part of software QA, but do you feel that bug-free ensures true quality? A bug-free Open Source project may still be too difficult to use or confusing for the non-technically inclined.
My beliefs do not require that you agree with them.
Do you find that the quality of the programming depends upon the geographic location of the programmers? So, for instance, an open source program will be troubleshot and combed over by people from potentially a dozen different countries. Closed source software is checked by people where it is written. Since, as a general rule, education varies in quality and areas of emphasis around the world, does it help having people attacking a program from many different angles (i.e., open source, cheked world wide) rather than simply drawing from a set of people who may share many of the same abilities, backgrounds, etc.?
I assume some of this information may be "company secrets" but I'm very interested in learning what metrics are used to determine which source code is "buggier" than others. Is this something as simple as running lint + "gcc -Wall -ansi -pedantic" then piping the output to "wc -l" ?
Are there checks for use of unsafe functions like gets and the str* family of functions in C? Are there more complex data flow analysis algorithms at play here like those in the used in Stanford's Meta-level compilation techniques?
Inquiring minds want to know. A pronouncement like OS foo is has more/less bugs than OS bar is meaningless without a definition of what having more/less bugs means.
How much of a role do you think peer review plays in software quality?
In proprietary source systems, there is generally formal peer review, as per CMMI. But I have seen this done rarely (almost exclusively for CMMI level 3+ projects). There seems to be a disincentive to do formal peer review. There seem to be various reasons for this, cost, workplace environment, and group dynamics. Which do you think are most significant?
Whereas in open source projects, there is not the formal peer review, but rather seems like a mass informal peer review. This seems to foster an enviroment of besting each other, trying to find the most and most obscure bugs.
What do you say?
SCO to Hell
What errors are currently hard to detect automatically but which you would really like to be able to find?
What is the next category of errors that you're trying to detect with automatic code inspection?
To give you some ideas, what about: