Debugging in OSS Always Faster

dex@ruunat writes "Damien Challet and Yann Le Du of the University of Oxford studied a model of software bug dynamics, which resulted in a paper on cond-mat this morning. In this paper they study the difference in evolution of number of bugs in open and closed source projects. They conclude: 'When the program is written from scratch, the first phase of development is characterized by a fast decline of the number of bugs, followed by a slow phase where most bugs have been fixed, hence, are hard to find'. Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."

Comments

Possible explanation?

[Lane.exe]

Could this be because OS projects tend to be, on the average, smaller than closed-source?

Re:Possible explanation?

[Anonymous Coward]

not always.

What we need is a QUALIT RATING or SECURITY or DESIGN ratings for OSS.

There is too much crap that is accepted as is because it is FREE.

I want QUALITY software, I want to see what footprints the software uses, what info is left where etc (security ratings), I want to see USABILITY (design ratings), good layouts, not like 100 sepereate windows where an MDI type frame with docking would be better (photoshop vs GIMP etc) and so on.

OSS NEEDS this.

Re:Possible explanation?

[jafac]

I work in an environment where we do Peer Reviews, and I've worked, in the past, in an environment where "if it compiles, ship it" - and I'll say that even if the Peers occasionally miss problems in the Review - the coder who has to present it to the Peers has a TOTALLY different attitude.

I see code that's very carefully analyzed first, thoroughly commented, thoughtfully indented, module, class, and variable names, though generally longer, they make sense. People go out of their way to be elegant.

I think that Peer Review is probably MORE important to overall quality of the end product, than developer experience. That's just my opinion, but after living the chaos that was a non-peer reviewed environment for 10 years, the attitudes, etc. there's really a huge difference.

It's even better if the Review team reserves the right, by convention, to give the presenter a wedgie if they don't like their code.

Re:Possible explanation?

[Dalcius]

"Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used. Sure, this may hold for the Linux kernel, or Apache, or even Mozilla, but what about all of the open source projects on SourceForge?"

While this argument does hold merit, I do think you would be surprised at the amount of looking over that utilities get. Ever hear of SDMS? Probably not. Not many folks have, I would reckon. I did some work with this project almost two years ago trying to implement PostgreSQL support so our company could use the utility internally. I found a few bugs and sent back patches as well as my changes to incorporate Postgres support. I can't speak as to the bug changes, but some time later Postgres support was added to the project.

This is but one example, but I think it's fair to say that, although "millions and jillions of eyes are looking!!!11!" isn't correct, there are more eyes looking than you would imagine. There are only so many categories of software, and each have a good number of users.

"Sounds to me like you're working with an inadequate number of testers, or at least an inadequate unit testing plan for developers."

At the risk of sounding like an ass, have you or are you now working as a programmer? Testers are a valuable commodity, but are rarely used in the manner that they should be. The company that I work for is a good example, though we're attempting to move in that direction.
Even with that said, this point is generally moot as one can argue that "more heads are better than none."

ESR says...

[Realistic_Dragon]

If you read 'The Cathedral and the Bazzar' by ESR he gives some very good reasons for Open Source development being better at bug finding:

1) With enough eyeballs, all bugs are shallow - someone will know the answer, or stimulate the person who does.

2) Users provide better bug reports including line numbers, decent config information, possibly even patches.

To which you can add number 3 and 4 of my own devising:

3) The kind of people who write and use OSS care about security and stability (often to an extreme) and so think that bug fixes are essential not a nice-to-have.

4) Closed source developes have to deal with management and merketing wienies - it's a wonder they even get buggy code out the door.

Seems reasonable to me

[riptalon]

During development the closed source software will only be used by the developers, and in general the developers are not like their end users and may have little interest in actually using the software they have been payed to write. For open source however the software is likely to be available to users from a very early stage and the developers are likely to be active users of the software as well. It would be very surprising if the bugs were not squashed faster.

Once the software is released closed source has the problem that bugs will only be fixed if the producer sees profit in it. Major security bugs will be fixed "relatively" quickly, as they might impact future sales otherwise, but with closed source the producer may not fix known non-security critical bugs if they don't feel like it, and no one else can.

There is also a problem with bug reporting in the closed source world. Who actually reports closed source non-security critical bugs? There isn't a lot of incentive since they may not be fixed anyway and if they are the fix will likely just go into then next version (that could be a year or two away) and you will have to pay for. Also the fraction of the users that do not have a licenced copy are unlikely to report bugs.

Whatever the merits of this particular study's methodology the results are just plain common sense anyway.