Slashdot Mirror
Debugging in OSS Always Faster
dex@ruunat writes "Damien Challet and Yann Le Du of the University of Oxford studied a model of software bug dynamics, which resulted in a paper on cond-mat this morning. In this paper they study the difference in evolution of number of bugs in open and closed source projects. They conclude: 'When the program is written from scratch, the first phase of development is characterized by a fast decline of the number of bugs, followed by a slow phase where most bugs have been fixed, hence, are hard to find'. Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."
IAALS.
Studies reveal that debugging is easier when you do not strip symbols from binaries!
There are no karma whores, only moderation johns
Ah yes, because we all know that Windows is, in fact, the only closed software in existence.
Forget the whales - save the babies.
Why does it matter if the source is open? The article should be talking about software developed by a team of individuales (including OSS) vs. software by one individual. If the source is open, there may be additional people looking at the sourcecode, but if there are many contributors the source may be a big mess of 50 different coding styles. (can't agree on a single way to splice a string, etc)
--
Getting too much pr0n?
In the open source world, people are more willing to let a project die and be replaced than in close source.
I know when I work on open source, if something is really ugly and needs to be replaced, I do. In a close source world thats often considered too risky and old unmaintainable code hangs around forever.
As with all statistics, you can make them say whatever you want...
Maybe most OSS projects are easier to debug because of lack of features, or smaller scope, etc.
What percentage of OSS projects, on say sourceforge.net, have a version number 1.0? (and are "widely" used). The first one that comes to my mind is MythTV and/or FreeVo. I can't speak of freevo, but mythtv (while being impressive) is still very bug ridden and has been out for over a year.
Another factor is the user group, of course. With OSS I imagine the kernel gets more bugs submitted by users than mythtv, just because the users aren't so much code hackers... they just want to use it.
The one rule in the software engineering is that there are no rules.
no comment
The paper's conculsion seemed to be that debugging open source projects is faster because you don't have a version problem where customers report bugs in code that has already been modified for the next version.
I don't buy it. Many open source projects (ACE/TAO, Mozilla) for instance have large customer bases using non-current versions, and presumably finding bugs. Sure, if you only want to fix the bug in the released version, its faster, but it's not like closed source vendors don't have the source code to their previous release to debug with.
Sure debugging is faster if you always make everyone upgrade to the latest version before filing a bug report. Good luck getting mass acceptance with that.
I'd imagine that one of the most difficult parts of debugging for a large OSS project is dealing with the deluge of bug reports.
What do you want, Windows nightlies?
The very concept fills me with dread.
--
the strongest word is still the word "free"
"Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."
.... is to release it to the outside world)
...
...
I thought that this was the one pro OSS argument that was the least argued; release early and often (since the only way to simulate all the different conditions your software will encounter in the outside world
This is also based on the fact (?) that the hardest part about squashing bugs is finding them
Is this actually controversial? (seems pretty much common sense to me, but I do not do programming as a profession, have I been misled?)
PS
Not to take anything away, it's one thing to have something be so "obvious" as to be "common sense", and quite another to have some data to back it up
This was a paper written for a class on statistics. It was not a rigorous study. Their findings are based on a lot of assumptions. They have a very small sample set--they only test their model on Linux, fetchmail, and Mozilla. Many people, including myself, consider these the cream of the crop so far as OSS goes.
Before you praise it, I urge you to actually read the paper. Don't be intimidated by it--FUD is FUD, even if it's mixed with a heavy does of greek letters and charts.
I do a lot of coding working mostly with open source products, and sometimes closed source. When I get some bug come up in an open source product, I actually go digging into their code sometimes to figure out what went wrong. If it's closed source I can dig down through my code, but once I hit their code, it's a brick wall.
Frankly this is why I try to stick to open source software when I do development work. Hell of a lot easier to figure out how something works when you've got code and direct access to the developers via a mailing list.
This sig has been temporarily disconnected or is no longer in service
Because gdb is so slow with a realy big project. ...Symbol loading... )
(
Trust me on this, I'm in this situation every day.
Well, I think the explanation for this is pretty obvious.
If you've got open sores, you're going to want to get bugs off of them as quickly as possible. You're also going to notice sooner because it's still bleeding. If you've got closed sores, you might not notice flies buzzing around them near so quickly.
We don't have a state-run media we have a media-run state.
A spokesperson at Microsoft refuted the conclusions of two french researchers from Oxford University this afternoon, saying that the business model behind Open Source was flawed anyway, since fewer bugs meant less urgency in updating to newer versions of software where old annoying bugs had been eliminated (only to be replaced with fresh one in anticipation of the subsequent forced release). The spokesperson also mentioned the enormous success of Microsoft's recent Closed Source initiative, under Bill Gates's supervision, to make computing more stable and secure, and finished by indicating that the UK government, who is being turned by Microsoft into a strong Open Source opponent (see recent Slashdot story), belonged to them anyway, and that the "frogs" would be deported to France shortly.
No No.. I wasnt even implying that was possible... Redmond hasn't even come to know and love publically selected mirrors... We are stuck with becoming psuedo world citizens in the sense that we are limited by the fact that everyone else in the world is using windows update... I hated that about windows... now, when I want to update anything - I emerge sync and emerge -u world in Gentoo Linux, and the world rejoices.
-Digital Extremist
If you read 'The Cathedral and the Bazzar' by ESR he gives some very good reasons for Open Source development being better at bug finding:
1) With enough eyeballs, all bugs are shallow - someone will know the answer, or stimulate the person who does.
2) Users provide better bug reports including line numbers, decent config information, possibly even patches.
To which you can add number 3 and 4 of my own devising:
3) The kind of people who write and use OSS care about security and stability (often to an extreme) and so think that bug fixes are essential not a nice-to-have.
4) Closed source developes have to deal with management and merketing wienies - it's a wonder they even get buggy code out the door.
Beep beep.
all the users use the modified code at time t + 1 and report bugs exclusively on the updated code.This assumes that all the users update their software at every release.
This is completely bogus. Not every user is going to update immediately. In fact, the larger the company is the less likely this is due to their desire to qualify new software. This means as Open Source gets more popular with big companies, the more bogus this assumption is.
The paper also mentions nothing about QA efforts or beta sites on closed source, which are typically on the "immediately updated" products.
I'm not going to argue whether oss is better/worse than closed, but I heavily doubt this paper proves anything other than if you make lots of assumptions you can prove whatever you want.
I wonder how much of this is due to programming style. What I mean is, can there be a difference in how easy it is to debug apache (maintained and written by many people) versus an OSS project which was mostly written my one person? I'd imagine that a large number of programmers results in more 'generic' code, rather than the specialized shortcuts that are unique to most programmers. That would make it much easier to debug, IMHO.
The one rule in the software engineering is that there are no rules.
I thought the first rule in software engineering was "you don't talk about software engineering."
Do not read this sig.
I could be suprising for a number of reasons:
1. Closed-source projects are often have a more structured process for development, along the lines of CMM, ISO 9001, established methodologies, etc. Things like CMM and ISO 9001 are assumed to result in higher quality (I seriously dispute that, but its the widely held assumption).
2. Many closed-source projects have customers with support contracts. Many open-source developers are not bound by such contracts and are not forced to fix bugs. It is assumed that suppliers with contracts to fix bugs will fix bugs (see Microsoft to see if that is a good assumption).
3. The quality of closed-source programmers is assumed to be higher. That is because the stereotype of the open source person is someone in college, unemployed, or in some other way with too much time on their hands. The sterotype of the closed-source (ie, corporate) programmer is that they went through a rigorous review and selection process to qualify to work on the project (that is assumed by people who have never worked in corporate IT).
Sarcasm and hyperbole are the final refuges for weak minds
Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."
:)
Thats because in closed source projects you have to dis-assemble the binary first!
All generalizations are false.
25% Funny, 25% Insightful, 25% Informative, 25% Troll
During development the closed source software will only be used by the developers, and in general the developers are not like their end users and may have little interest in actually using the software they have been payed to write. For open source however the software is likely to be available to users from a very early stage and the developers are likely to be active users of the software as well. It would be very surprising if the bugs were not squashed faster.
Once the software is released closed source has the problem that bugs will only be fixed if the producer sees profit in it. Major security bugs will be fixed "relatively" quickly, as they might impact future sales otherwise, but with closed source the producer may not fix known non-security critical bugs if they don't feel like it, and no one else can.
There is also a problem with bug reporting in the closed source world. Who actually reports closed source non-security critical bugs? There isn't a lot of incentive since they may not be fixed anyway and if they are the fix will likely just go into then next version (that could be a year or two away) and you will have to pay for. Also the fraction of the users that do not have a licenced copy are unlikely to report bugs.
Whatever the merits of this particular study's methodology the results are just plain common sense anyway.
Successful OSS projects must be well documented in order to survive. Naming variables in an intuitive manner and providing insightful comments isn't about improving your annual review scores, it is about ensureing that others can and will read your code.
Companies like Microsoft need to introduce policies to create the same effect. Code reviews and extreme programming are good examples. They often degenerate into either a rubber stamp or a grudge match between different interpretations of Hungarian Notation.
Many eyes can. How many actually do? Unless you're talking about the really big projects, probably very few indeed -- one, I suspect, in many cases.
It's not fair to cite mainstream developments like Linux or Mozilla as the way all open source is any more than it's fair to cite Microsoft's history on things like security and reliability as the way all closed source is.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
This is yet another example of someone trying to sell the OSS agenda. Statistics can easily lie, and I suspect that the person involved in the study was not a neutral party. A more important question is do people on /. EVER find fault in OSS? Surely it can't be a nirvana. Be honest, and get real.
It should be impossible in a sensibly run closed source project as well. If you have a development team whose egos are such that they can't stand criticism and claim unique ownership, you are on the absolute lowest rung of the smart development process ladder. Teams full of such people will never produce good results, whether their source is closed or not.
As with many of the points raised in a thread like this, though, the problem is with industrial development processes and not actually with closed source. In a decent coding shop, no-one has sole knowledge or control of any area of code. In better shops, reviews are routine, and the good guys actively solicit feedback from their peers.
This is just another case (like the "many eyes" argument, and others) where the argument made in favour of open source looks great at first glance, but doesn't stand up to much scrutiny. To date, the most convincing argument I've seen for why some of the big open source projects have generally good code is still that those who do it frequently do so out of interest, rather than just to pay the rent (though certainly most of the work on the big OSS projects is done by pro's these days).
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
" Not always. A more likely explanation is the 'many eyes' that can review the code."
I went to a speech by Gene Spafford here a few years ago, when the subject of Linux code quality versus other systems (especially MS) came up. Someone mentioned Eric Raymond's "Thousand Eyeballs" theory, that more people looking at the code ensured better quality.
Spaff responded "that does no good if those thousand eyeballs are looking at things like networking your toaster instead of quality and security".
I don't think this point is emphasized enough. It's not enough that lots of people are looking at the code. You need lot's of people with training, expierience, and an eye for problems to look at the code. He pointed out that one of the biggest problems in development is that while people can learn C from a book, and even get good at it, they don't learn proper software engineering techniques, philosophies, and debugging skills that way.
In short, simply being open source and having lots of developers isn't a solution in itself.
Life is hard, and the world is cruel
Seriously, grab gnat, go to http://www.adahome.com/ and read a tutorial or two, and start writing reliable, readable, and reusable code in the amount of time it takes to read through the tutorial.
This may come as a shock to you but closed source developers do have access to source code, as unbelievable as that may sound. That means that they can actually step through the code with a debugger just like their open source friends.
Two of the three open projects you cite also happen to be bigger than most projects, with more developers working in || at the same time than most projects. Does that help them? Read "the mythical man-month" for the answer. (Hint: no).
You have seriously misread The Mythical Man Month, to the point of absurdity. The Mythical Man Month does emphatically not claim that more people can't do any more then fewer people.
What it says is that programmers are not interchangable, and as a result, adding programmers to an existing project will not get a 1-man-hour to 1-programmer-hour gain. While the old programmers are training the new programmers, and while the communication overhead increases, and while the new programmers are not yet settled in, productivity may fail to increase or may even decrease, over the next few months.
However, once the new guys are settled in and assuming a decent organization, the new organization can indeed move faster.
The Mythical Man Month's main point is to avoid treating programmers, or perhaps more accurately programmer-hours, interchangably. The conclusion based on that is the somewhat more famous part about not adding manpower to a late project because it will make it even more late.
Most strong open-source projects, and probably all the ones in this study, have a strong group of core people who are intimately familiar with what they need to know to continue improving the software. Other people may drift in but will find it difficult to contribute code back. (Seriously, just try to get a patch into Mozilla.) The Mythical Man Month does not apply to steady-state teams of people, only growing ones.
So yes, larger teams help those projects create their large software products. Go look at how much code is in Mozilla; no matter how wizard you are, I don't think a small team of people could even type that much code in anything less then a couple of months. How else do you think they could do it but with a large team?
I think those are probably fossil practices left over from pre-Internet days.
It is important to shield the developers from telephone calls and visits from the customer, because otherwise the developers will get nothing done, not necessarily because they have no time (time_in_day - time_dealing_with_customers may still be large), but because they are constantly being interrupted.
As long as the developer has discipline and the customer realizes that the answers won't be instant, though, there can be great value in having email access to the developers, for both parties.
Hopefully more companies will modernize their policies soon.
I then boot to Linux and port my code. I've been writing portable code for half a decade, so I know what I'm doing, more or less. But, I can get more work done with Visual Studio, faster.
In case it makes a difference to your perception, I write end user apps, sometimes with heavy graphics requirements and GUI frontends.
Due to the nature of my work, I can't rely on masses to test everything before I ship.