Slashdot Mirror
Debugging in OSS Always Faster
dex@ruunat writes "Damien Challet and Yann Le Du of the University of Oxford studied a model of software bug dynamics, which resulted in a paper on cond-mat this morning. In this paper they study the difference in evolution of number of bugs in open and closed source projects. They conclude: 'When the program is written from scratch, the first phase of development is characterized by a fast decline of the number of bugs, followed by a slow phase where most bugs have been fixed, hence, are hard to find'. Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."
IAALS.
Why does it matter if the source is open? The article should be talking about software developed by a team of individuales (including OSS) vs. software by one individual. If the source is open, there may be additional people looking at the sourcecode, but if there are many contributors the source may be a big mess of 50 different coding styles. (can't agree on a single way to splice a string, etc)
--
Getting too much pr0n?
In the open source world, people are more willing to let a project die and be replaced than in close source.
I know when I work on open source, if something is really ugly and needs to be replaced, I do. In a close source world thats often considered too risky and old unmaintainable code hangs around forever.
No No.. I wasnt even implying that was possible... Redmond hasn't even come to know and love publically selected mirrors... We are stuck with becoming psuedo world citizens in the sense that we are limited by the fact that everyone else in the world is using windows update... I hated that about windows... now, when I want to update anything - I emerge sync and emerge -u world in Gentoo Linux, and the world rejoices.
-Digital Extremist
If you read 'The Cathedral and the Bazzar' by ESR he gives some very good reasons for Open Source development being better at bug finding:
1) With enough eyeballs, all bugs are shallow - someone will know the answer, or stimulate the person who does.
2) Users provide better bug reports including line numbers, decent config information, possibly even patches.
To which you can add number 3 and 4 of my own devising:
3) The kind of people who write and use OSS care about security and stability (often to an extreme) and so think that bug fixes are essential not a nice-to-have.
4) Closed source developes have to deal with management and merketing wienies - it's a wonder they even get buggy code out the door.
Beep beep.
I wonder how much of this is due to programming style. What I mean is, can there be a difference in how easy it is to debug apache (maintained and written by many people) versus an OSS project which was mostly written my one person? I'd imagine that a large number of programmers results in more 'generic' code, rather than the specialized shortcuts that are unique to most programmers. That would make it much easier to debug, IMHO.
Then explain these open source FAILURES
1) The kweather applet in kde
2) The gtk file dialog
3) the latency in the arts sound sever
4) the lack of support for pci modems
5) the color scheme of games.slashdot.org
I could be suprising for a number of reasons:
1. Closed-source projects are often have a more structured process for development, along the lines of CMM, ISO 9001, established methodologies, etc. Things like CMM and ISO 9001 are assumed to result in higher quality (I seriously dispute that, but its the widely held assumption).
2. Many closed-source projects have customers with support contracts. Many open-source developers are not bound by such contracts and are not forced to fix bugs. It is assumed that suppliers with contracts to fix bugs will fix bugs (see Microsoft to see if that is a good assumption).
3. The quality of closed-source programmers is assumed to be higher. That is because the stereotype of the open source person is someone in college, unemployed, or in some other way with too much time on their hands. The sterotype of the closed-source (ie, corporate) programmer is that they went through a rigorous review and selection process to qualify to work on the project (that is assumed by people who have never worked in corporate IT).
Sarcasm and hyperbole are the final refuges for weak minds
Two of the three open projects you cite also happen to be bigger than most projects, with more developers working in || at the same time than most projects. Does that help them? Read "the mythical man-month" for the answer. (Hint: no).
During development the closed source software will only be used by the developers, and in general the developers are not like their end users and may have little interest in actually using the software they have been payed to write. For open source however the software is likely to be available to users from a very early stage and the developers are likely to be active users of the software as well. It would be very surprising if the bugs were not squashed faster.
Once the software is released closed source has the problem that bugs will only be fixed if the producer sees profit in it. Major security bugs will be fixed "relatively" quickly, as they might impact future sales otherwise, but with closed source the producer may not fix known non-security critical bugs if they don't feel like it, and no one else can.
There is also a problem with bug reporting in the closed source world. Who actually reports closed source non-security critical bugs? There isn't a lot of incentive since they may not be fixed anyway and if they are the fix will likely just go into then next version (that could be a year or two away) and you will have to pay for. Also the fraction of the users that do not have a licenced copy are unlikely to report bugs.
Whatever the merits of this particular study's methodology the results are just plain common sense anyway.
This is yet another example of someone trying to sell the OSS agenda. Statistics can easily lie, and I suspect that the person involved in the study was not a neutral party. A more important question is do people on /. EVER find fault in OSS? Surely it can't be a nirvana. Be honest, and get real.
Seriously, grab gnat, go to http://www.adahome.com/ and read a tutorial or two, and start writing reliable, readable, and reusable code in the amount of time it takes to read through the tutorial.
This may come as a shock to you but closed source developers do have access to source code, as unbelievable as that may sound. That means that they can actually step through the code with a debugger just like their open source friends.
I've looked at a surprising number of program's source code. abcde, www-mechanize, ode physicis libraries, perl stuff, php stuff, python stuff, parts of the ogg utilities. Mostly because it doesn't have some feature that I want, or it does something that I'm interested in doing, and want to know how hard it is. It might not be much, but a cursory overview can tell you whether the author knew what they heck they were doing, and auditing even one code path (or even one function) can be helpful. Try it some time. If you're using debian: apt-get source abcde. I use it all the time as my "reference to how the hell bash programming works" :^)
--Robert
Sounds like the type of hell I put my developers through. I have been averaging roughly 2 bugs per day, while writing test automation, verifying fixes, generating reports, etc.
The customer probably would have only reported a dozen of those bugs if they weren't fixed, and they might only notice a couple dozen now that they are fixed and different from the previous version, but the developers fixed several hundred of the bugs that I found.
As for "Why in the world would I do THAT?", "because the code let me do it." Any good software tester should continually challenge the assumptions of the developers. The code needs to work correctly.
Actually, you're not quite synical enough. Stat's are used/abused on a daily basis. It's not about the figures, it's about the questions used to represent the figures.
E.g. DNA testing for criminal proceeds was once said to be as good as 1 in a million. So when the police get a suspect and the DNA matches, we have guilty verdict.
Well no. It merely says that in a country like the US, there are statistically speaking around 240 possible matches. Doesn't sound quite the same does it?
In this case, close family and ethnic origin will probably offer up the 239. (DNA testing has hopefully moved on to a more detailed testing, but there was a big cost factor with the improved accuracy.)
So you see we have the same figures, but completely different ways to influence someone.
In market research, this kind of usage is very common. When a client want their results to appear better than their competition's, you merely add addition critera to a question to filter the data until the figures show what you want them too. Once you've got the figures, you then rephrase the question to be somewhat less obviously manipulative. Then the final report gets written showing x is better than y.
I see this very regularly because I wrote the software that facilitates it.