Debugging in OSS Always Faster

dex@ruunat writes "Damien Challet and Yann Le Du of the University of Oxford studied a model of software bug dynamics, which resulted in a paper on cond-mat this morning. In this paper they study the difference in evolution of number of bugs in open and closed source projects. They conclude: 'When the program is written from scratch, the first phase of development is characterized by a fast decline of the number of bugs, followed by a slow phase where most bugs have been fixed, hence, are hard to find'. Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."

Comments

Possible explanation?

[Lane.exe]

Could this be because OS projects tend to be, on the average, smaller than closed-source?

Re:Possible explanation?

[SpaceLifeForm]

Not always. A more likely explanation is the 'many eyes' that can review the code.

Re:Possible explanation?

[Anonymous Coward]

not always.

What we need is a QUALIT RATING or SECURITY or DESIGN ratings for OSS.

There is too much crap that is accepted as is because it is FREE.

I want QUALITY software, I want to see what footprints the software uses, what info is left where etc (security ratings), I want to see USABILITY (design ratings), good layouts, not like 100 sepereate windows where an MDI type frame with docking would be better (photoshop vs GIMP etc) and so on.

OSS NEEDS this.

Re:Possible explanation?

[macrom]

I would think the opposite -- that there are more (talented?) eyes looking at OS projects than CS projects. Many times closed projects have several developers, but only one ever sees a particular module at any given time. At my current company, we have 30 or so developers, but the modules I write are owned by me and usually only seen by me. Peer review sessions can alleviate this, but those are generally short and cover major functionality. With OSS, you have an untold number of eyes viewing a project that can help catch problems in a more timely manner.

The theory that your best work will be done when the most eyes are watching can also apply. I think we (developers) are all guilty of shoving some nasty code in a project at some time or another under the notion that no one else will ever see it. When the whole world can look at your work, sometimes those attitudes change.

Re:Possible explanation?

[tomstdenis]

Linux. Nuff said.

Besides, you'd be surprised how many commercial libs out there are based on OSS. Like all the zlib variants. Or all the MP3 decoders, etc...

The difference is more of a "reputation" is at stake in OSS. You can't just half-ass write something that works most of the time when your name is all over it.

Of the few software firms I've worked in most of the software developers are in the mindset "if the program runs and doesn't crash, its all good" which leads to horribly incomplete poorly factored source code that is a bitch to work on.

Tom

Re:Possible explanation?

[athakur999]

Something else I can think of is that your testers have access to the code and can give you more precise bug reports than if they were doing traditional black box testing. Better bug reports and information means less time investigating.

Re:Possible explanation?

[jared_hanson]

There are a number of different possiblities here, given the different dynamics of each group.

1. In general, there are more users of closed source software, so bugs may be discovered at a faster rate. With limited development resources, the greater number of bugs take longer to fix.

2. Users of open source software tend to be more programmer-minded. They find bugs and fix them themselves, since they have access to the source code. Everyone fixing their own bugs leads to faster debugging times.

3. In larger companies, development and test typically are two distinct groups. There is an inherent lag in this that leads to slower response time in removing bugs. Open source software is developed and tested by the same core, ususally small group. Since the same people develop and test, they are more likely to find the bugs in their own code and fix them quicker.

Just some observations, and I am sure there are other reasons. I'm sure the results of the study are a combination of many factors.

Re:Possible explanation?

[Anonymous Coward]

It could be, but based on my own development I would think it's a matter of documentation versus code availability.

There are some commercial libs out there (e.g. DirectShow, QuickTime) which have poor, incorrect, or no documentation for a variety of functions. If I had the source, I'd understand what these functions wanted and were capable of doing....instead I spend extra time with the cat 'n mouse testing of a single function (e.g. QuickTime's use of H.263 when setting up the data rate and getting it to work how you want). OSS usually (IMHO) lacks documentation but actually seeing the source will be helpful (not every time, but some times).

[rant]
I wish the QuickTime team would get a little less arrogant. They've put together some great stuff over the years, but the hide and seek crap they pull with their documentation and even live support is really pathetic. Come on: you're super-achievers but you can't document your work? You aren't *that* special..
[/rant]

Re:Possible explanation?

[Osty]

With OSS, you have an untold number of eyes viewing a project that can help catch problems in a more timely manner.

Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used. Sure, this may hold for the Linux kernel, or Apache, or even Mozilla, but what about all of the open source projects on SourceForge?

At my current company, we have 30 or so developers, but the modules I write are owned by me and usually only seen by me. Peer review sessions can alleviate this, but those are generally short and cover major functionality.

Sounds to me like you're working with an inadequate number of testers, or at least an inadequate unit testing plan for developers. Testers are invaluable because that's all they do -- test code. This frees up the developer to be able to actually write code while not having to sacrifice quality for lack of testing. Sure, it's more expensive to hire both testers and developers, but I'd bet that of your 30 or so developers, you could fire 10 of them and hire 15 testers for the same cost, and still have enough man power for development while increasing your code quality from rigorous testing. (Note: This isn't saying that developers can or should write bad code so long as they have testers. Developers should still aspire to writing quality code, so that the testers can focus on the really heinous parts, and not on trivial bugs or logic errors. But at least with testers and good unit tests, you're less likely to let slip a silly bug.)

The theory that your best work will be done when the most eyes are watching can also apply. I think we (developers) are all guilty of shoving some nasty code in a project at some time or another under the notion that no one else will ever see it. When the whole world can look at your work, sometimes those attitudes change.

That may be true in theory, but it doesn't pan out in practice. In practice, most open source projects will only be seen (code-wise) by the author(s). If you're lucky, you might have two or three active users that will submit bugs or patches, but often that's not the case.

Re:Possible explanation?

[dasmegabyte]

We have egos at work too. I am not supposed to TOUCH anything my coworkers do, it is THEIR code and THEIRS to command, hands off! Even when I sneak a peek and find obvious bugs, I am not allowed to fix them because it violates territoriality.

This sort of code hording is impossible in OSS, and it's a shame. It's well known in literary circles that the more eyes that grace a page, editorially, the more likely you'll be to have a coherent and gramatically correct work. Code is no different IMO.

Of course, code hording does decrease the inevitable argument over implementation that plague many software houses...and for which OSS is uniquely famous.

Re:Possible explanation?

[Graspee_Leemoor]

That explains why I debug so fast! I have many eyes. At last count I had 27! (Some of them are in the back of my head though, which means they can only debug things behind me).

graspee

Re:Possible explanation?

[ChrisMaple]

I look at OS code only when I have a problem with the binary executable, and need to fix it if I can.

The code I have seen has been of only moderate quality (from the standpoint of design for fast execution) and the commenting has been grossly inadequate. On the plus side, structure and modularity are usually quite good. Overall, I don't see a lot of evidence that developers have been behaving better because someone else might see the source.

Re:Possible explanation?

[1010011010]

Commercial software needs those things too. There's the security ratings for Word and Outlook? Where's the usability ratings for IE?

Etc.

Re:Possible explanation?

[drzhivago]

That is a general insight into bug testing and perfectly valid, except your statement is a bit flawed.

If the testers have some general coding and OS knowledge, they can help determine the fix before sending off an issue. It doesn't really matter whether the code is publicly available or not. Testers for non-open source projects can have access to the code as well. The code isn't usually kept in a vault with armed guards protecting it, you know. It is more a matter of having knowledgable and studious testers.

Re:Possible explanation?

[muyuubyou]

I think it's because people starting fresh can see bugs where the original programmers think it's perfectly settled.
Those are, precisely, the bugs escaping the first phase the article writer talks about.

Programmers are humans too and get bored. First passes of debugging are done more or less deeply, but unconsciously the following passes can't catch so much attention from the programmers, no matter how hard they try.

In closed source, bringing new programmers not involved from the beginning is outrageously expensive: they have to spend lots of hours just getting used to the code. In Open Source, most if not all of those extra debugging passes are done for free.

Re:Possible explanation?

[jafac]

I work in an environment where we do Peer Reviews, and I've worked, in the past, in an environment where "if it compiles, ship it" - and I'll say that even if the Peers occasionally miss problems in the Review - the coder who has to present it to the Peers has a TOTALLY different attitude.

I see code that's very carefully analyzed first, thoroughly commented, thoughtfully indented, module, class, and variable names, though generally longer, they make sense. People go out of their way to be elegant.

I think that Peer Review is probably MORE important to overall quality of the end product, than developer experience. That's just my opinion, but after living the chaos that was a non-peer reviewed environment for 10 years, the attitudes, etc. there's really a huge difference.

It's even better if the Review team reserves the right, by convention, to give the presenter a wedgie if they don't like their code.

Re: Possible explanation?

[SuperDuperMan]

Do you really believe that all these OSS developers are not for pay developers also? I don't think it's a matter of the skill level of the open vs the closed I think it's a matter of the open being developed in the free time of individuals and there is no monetary pressure to get something done sooner rather than later.

Developers who do programming for a living have to get paid so the money has to come from the product being sold and that leads to it needing to be done more quickly.

Those that are doing this work for fun don't have these pressures but they probably do when they are at their day jobs. I don't think a majority of the OSS developers do something other than programming for their wages.

Re:Possible explanation?

[macrom]

Sometimes it's not egos that cause a lack of eyes but time. I would love to spend more time as a team collaborating on code, but I barely have enough time to get my own modules written and tested. Then there's also the problem of talent -- most of the people in my development department are VB programmers. I wouldn't want them looking and evaluating my C++ if my job depended on it. Well, maybe then, but I would at least throw a fit! Add to it that we don't really have a talent pool for reviews and you're stuck with a problem that plagues all too many companies these days : lots of coders that create crappy code.

Re:Possible explanation?

[mrroach]

MDI would be better than gimp's approach? If you want to simulate mdi, just keep all your gimp windows in their own workspace... Or if you are really psychotic, run gimp inside an xnest session.

I'm curious though, what makes trapping windows inside one bigger "mother" window inherently better? (StarOffice Desktop anyone?)

In my opinion, programs should leave window management to the window manager, that's what it's there for.

-Mark

Re:Possible explanation?

[Dalcius]

"Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used. Sure, this may hold for the Linux kernel, or Apache, or even Mozilla, but what about all of the open source projects on SourceForge?"

While this argument does hold merit, I do think you would be surprised at the amount of looking over that utilities get. Ever hear of SDMS? Probably not. Not many folks have, I would reckon. I did some work with this project almost two years ago trying to implement PostgreSQL support so our company could use the utility internally. I found a few bugs and sent back patches as well as my changes to incorporate Postgres support. I can't speak as to the bug changes, but some time later Postgres support was added to the project.

This is but one example, but I think it's fair to say that, although "millions and jillions of eyes are looking!!!11!" isn't correct, there are more eyes looking than you would imagine. There are only so many categories of software, and each have a good number of users.

"Sounds to me like you're working with an inadequate number of testers, or at least an inadequate unit testing plan for developers."

At the risk of sounding like an ass, have you or are you now working as a programmer? Testers are a valuable commodity, but are rarely used in the manner that they should be. The company that I work for is a good example, though we're attempting to move in that direction.
Even with that said, this point is generally moot as one can argue that "more heads are better than none."

Re:Possible explanation?

[Herkum01]

Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used.

In other words, he is no worse off writing open-source software than closed-source. Most closed-source software is only checked over by one person, the one who wrote the software. At least with open-source there is the potential that other poeple will get involved with the project, with closed-source there is no potential.

Sounds to me like you're working with an inadequate number of testers, or at least an inadequate unit testing plan for developers

Most corporations, don't have quality testers and will not spend the money on them. Usually they roll it out to a limited number of users, who will HAVE TO USE THE PRODUCT and that is the test bed. That does not make them eager or exceptional or effective. At least with open-source, if someone is interested in a project they are usually eager to contribute, either by coding or testing, more so than someone who believes that it is an extra hassle to their regular job.

If you're lucky, you might have two or three active users that will submit bugs or patches, but often that's not the case.

Most closed-source products will not even acknowledge that they are fixing bugs, they have spent their money for development, got their money from the customer, so they are done as far as they are concerned. The programmers have moved onto the next project and the customers are just screwed. At least with an open-source project progress can continue alibi slowly if at all. However, someone to pick up the project and run with it.

While most open-source projects will not go anywhere, they always have the potential, closed-source projects have no potential at all. Once the company is done investing money into it, it stops for basically all time regardless of how good or bad it is.

Re:Possible explanation?

[macrom]

Sure, it's more expensive to hire both testers and developers, but I'd bet that of your 30 or so developers, you could fire 10 of them and hire 15 testers for the same cost, and still have enough man power for development while increasing your code quality from rigorous testing.

Osty, you're exactly right. Sometimes we just have to make do. The few of us decent programmers hope for a day when the management gets the stones to get rid of the dead weight. My company is like many others : keeping your job is at least 50% politics, 50% performance. And sometimes the management shows complete ineptitude when evaluating either. I just keep chugging along hoping that I can help the change someday. Or the job market turns around for the better, whichever comes first!

Re:Possible explanation?

[Pflipp]

Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used. Sure, this may hold for the Linux kernel, or Apache, or even Mozilla, but what about all of the open source projects on SourceForge?

As soon as it gets used, it gets seen, which is not true for even popular closed source material.

I once programmed against libgnet 0.2. This library was (at least back then) so obscure that only two websites from google had the (fairly lacking) HTMLized API docs -- one of them being obsoleted by the other, and this was all the net presence there was. (Looking back I wonder how I got track of gnet in the first place -- probably throught Debian.)

I noted my program stopped working at some kind of magic number. I found the number was not in my code, so I finally got to downloading the source for libgnet, and I found a 1-off problem with the used algorithm. I could describle exactly to the author what he had to change.

Not that the author (probably, as many others, unable to maintain his little project even in its 0.2 stage) ever bothered to reply on my bug report, although this could also be because of the tone I wrote it in (I was fairly impressed by the layout of his source, which was amazingly precicely spaced, so I wrote kind of in a "I can't imagine little me finding a bug in code made by You, oh Master of Code"-style. That was before I learnt about what GNU indent by default does to code :-)

Re:Possible explanation?

[Deusy]

Add to that the fact that OSS software gets to it's users a lot earlier than it's closed source counterparts.

After all, it's the users who find the obscure bugs in places that the developers would never think to check.

Then, as the project develops from an early age, the community feeds back in multiple ways - feature requests, bug reports, public mail list discussions that involve non-coding developers. This contributes greatly towards a projects direction.

I'd go as far to say that the user involvement in popular OSS projects probably finds more bugs than extra developers looking at code.

Re:Possible explanation?

[Osty]

At the risk of sounding like an ass, have you or are you now working as a programmer?

Yes, I am currently working as a programmer. I'm young, and this is my first programming job (held it for three years, though), so I may be biased.

Testers are a valuable commodity, but are rarely used in the manner that they should be.

I don't know what other companies do, but I know what mine does, and our testers test. We do have a hybrid developer/tester position, who is generally responsible for writing the code the testers need for testing, but our testers do sometimes write harness code as well. Most of the time, however, testers write test cases, not code (generally in XML), and run those test cases, and then bug developers. The good testers provide excellent repro steps and a good explanation of the behavior they saw and the behavior they expected, so you as the developer can take the bug and run with it without having to do major investigations just to learn what the bug is about.

Re:Possible explanation?

[damiam]

That's true, but the Photoshop UI does have some advantages over Gimp - the menu and toolboxes are always visible, and not liable to get hidden behind picture windows. Sure, you can say "Just tell your WM to make the main GIMP window stay on top", but not all WMs (Windows, for example) can easily do that. Even then, it's annoying to maximise an image and have it overlap the toolbox.

I like the way Photoshop for Mac is done. It's not MDI, because all windows are managed by the WM, but it gives it enough hints to ensure that things just work. For example, if I click on the GIMP's icon in my dock/taskbar/panel, I want to see the image I'm working on, not just the GIMP toolbox. Photoshop gets this right, and GIMP doesn't.

Re:Possible explanation?

[listen]

The gimps ui is totally app centered. The "in charge" window is the toolbox.

The gimp could be fixed pretty easily to annoy a lot of people less:
1) Menus across the top of each image window.
The right click to access the main menu is not familiar at all to most people. To fitts law lovers :
when gtk supports universal menus, that will work,
currently, it doesn't.
2) The toolbox should not be the master of the universe -
When the last image is closed, the gimp should close.

But I think the whole thing would be even better off with a "normal" interface, ie like Koffice or Gnumeric, with dockable toolboxes, etc.

Re:Possible explanation?

[ColaMan]

You can't just half-ass write something that works most of the time when your name is all over it

Well, looking at most OSS projects, you can do that as long as :

1) No-one else has done it yet.
2) You mention in the source that this is a "half-ass hack that I threw together to make something work"

Re:Possible explanation?

[afidel]

I think he was refering to beta sites or tester, as in customers who are running pre-production code. In that case it depends on the relationship but most closed source shops do not share their beta code with their testers.

Re:Possible explanation?

[Nerull]

"I can't imagine little me finding a bug in code made by You, oh Master of Code"-style

That style reeks of sarcasm, even if it wasn't intended that way. I would have, and I'm sure that he, if he saw it, did, taken it as an insult. :)

Re:Possible explanation?

[fredrik70]

Well, you got the headers at least even in closed source and hopefully someone put a decent comment explaining the function. (well, people really should do that IMHO). Otherwise I do agree that source is godsent when you have no decent docs. Actually, thinking of it, I do believe if you have a large project, module owners should really only have to worry about their own code and other groups modules should only be specified in docs, otherwise other module owners might start relying on implementation specific stuff in the other module. one must emphasise though that the module owners better keep their docs uptodate or get whipped in public.

Re:Possible explanation?

[ClosedSource]

"Most closed-source software is only checked over by one person, the one who wrote the software."

And how did you come to that conclusion?

Re:Possible explanation?

[yaphadam097]

Perhaps instead it is the process. There doesn't tend to be as much Big Design Up Front (BDUF) in open source projects as in many commercial projects. The development process for OSS is either driven primarily by a single project lead or a small group of committers rather than a requirements gathering/business analysis team like at many software firms. Thus the process is more fluid and code oriented. If design flaws occur it is usually due to lack of vision of the project as a whole and not due to programmers misunderstanding business people's requirement documents (Which often seem written explicitly to elicit misunderstanding, IMO)

Re:Possible explanation?

[sheldon]

Point 3 could be rewritten this way as well:

3. Open source software is usually developed and tested by the same core group, usually a small group. There is an inherent risk that bugs will not be identified as such because of the familiarity the people have with how the code works, as well as the peer pressure resulting from personal relationships. On the other hand in larger companies testing and development are usually two distinct groups, as such there is an independence granted to the tester to identify bugs openly and honestly with the resulting goal of the company shipping a better product.

The question you have to ask yourself is which is more true.

I have done testing in the past, and I've seen projects done both ways. How often have you seen this exchange between user and developer...

User: "Well when I click on button B after checking off box A the app blows up."

Developer: "You idiot! You're not supposed to click on Button B after checking off box A, you're only supposed to click Button C."

Developer then proceeds to document this call as an enduser training issue.

A Tester would report it as a defect in that Button B should be disabled when box A has been checked"

Unfortunately the situation is far too common than many would like to admit.

Re:Possible explanation?

[clem]

That explains why I debug so fast! I have many eyes. At last count I had 27! (Some of them are in the back of my head though, which means they can only debug things behind me).

Well, as they say, hindsight is 20/20.

Re: Possible explanation?

[jcast]

Sure a lot of the OSS developers write proprietary code, but most proprietary developers don't write OSS code. OSS developers can write proprietary code and still be drawn from the top notches of proprietary developers---i.e., all OSS is written by the people who write the very best proprietary code.

Re:Possible explanation?

[PetWolverine]

...and I've worked, in the past, in an environment where "if it compiles, ship it"

Which versions of Windows did you work on?

Re:Possible explanation?

[carpe_noctem]

We have egos at work too. I am not supposed to TOUCH anything my coworkers do, it is THEIR code and THEIRS to command, hands off! Even when I sneak a peek and find obvious bugs, I am not allowed to fix them because it violates territoriality.

Wow, did you work at SCO?

Re:Possible explanation?

[jafac]

tee-hee.

hint: same company (different team) shipped a bit of software that was bundled with Win95.
I know, doesn't narrow it down much.

I'm mainly just trying to criticize the 90% of the commercial software industry which operates under CMM level 0, and the idiotic customers who demand nothing better than that.

On the other hand, having to organize, coordinate, and participate in Peer Reviews slows the pace of development to roughly 10% of "normal" development.

Re:Possible explanation?

[InterruptDescriptorT]

What we need is a QUALIT RATING...

Looks like what we need is a QUALIT RATING on the spelling in peoples' posts.

Re:Possible explanation?

[Osty]

As soon as it gets used, it gets seen, which is not true for even popular closed source material.

That's true if you're actually using the source code, but in most cases that's not happening. Hell, even with your example of a library you're still not directly looking at the source code. You went to the source when you had issues, yes, but with a well-documented library you wouldn't need to do that. And that's not even saying anything about applications, which you wouldn't even be interacting with in code.

Open source advocates say things like, "I can audit the code myself to see what it's doing," but how many people do you think actually do that? Beyond the clinically paranoid or overly zealous, I'd suggest very few people actually do that. I know I don't, and I don't know anybody who does.

Re:Possible explanation?

[po8]

Heh. I was reading the source code for XGammon for some random reason a few months ago, and the README said

An Imakefile is provided now.
Thanks to thomas@ghpc8.ihf.rwth-aachen.de and
Bart Skinner (bart@skinner.cs.uoregon.edu)

and later

Portation: SunOS 4.1 4.2 bart@skinner.cs.uoregon.edu, was the first and many others.

I sat there for a moment trying to figure out who this Bart Skinner person was: I had done CS at U. Oregon in that time frame, and couldn't remember the name. Finally it came to me: it was me.

I had given the authors some help with the program back in the day and since totally forgotten about it. They had credited me (thanks!) but had confused the hostname of the machine I was on with my last name: as a result, I didn't recognize me for a moment.

It was a pretty obscure piece of code in its heyday. In 2003 it's even more obscure. A few months ago I was looking at the source...

Re:Possible explanation?

[Anonymous Coward]

The code isn't usually kept in a vault with armed guards protecting it, you know

Boy, are you in for a rude surprise when you graduate.

But seriously, most testers for larger systems are busy enough maintaining test scripts, documentation, result sets etc. to actually try to stay up to date on the build tree & process. Admittedly this is less true for unit test than for system and integration but then unit testers are often just developers in funny hats.

This, of course, begs the question of why this isn't a serious problem for OSS but I'll let somebody else tackle that.

Re:Possible explanation?

[Dalcius]

To be fair, I'm young as well (I'm 20). Minus a year for college, I've been working as a programmer at my company since the summer of 2000 (with a lot of recreational programming before that).

For the last year, I've been working as a "Software Analyst". I get bug reports that our setup folks can't solve and I solve them. This often requires a lot of code hunting. We've got everything from extremely junky Fortran 77 (no whitespace, no variable names over 6 characters -- and Fortran at that) for our legacy app, and some CGI programs written in spaghetti-code C (with a good mix of HTML templating and javascript thrown in)! Even better, this all runs on HP-UX. Needless to say, I get plenty of practice fielding bad code and weird issues. =)

Our development team is comprised of 12 developers total, and only in the last 4 years has it grown past a four man team. The company is now around 50 heads total, and a QA department is in sight (the Software Analysts will eventually be QA).

Anyway, our newer products are all based on open source tools. PostgreSQL, Apache, Linux, Perl, Mason, ORBit, etc. The source code, mailing lists, IRC, etc. and the open attitude have made things a breeze, not to mention these tools have saved us a very large sum of money in licensing costs.

This is quite possibly the exception to the rule, I have no problem admitting that. Somehow, though, from what I hear of other companies, this isn't all too uncommon.

Cheers

Re:Possible explanation?

[miu]

We have egos at work too. I am not supposed to TOUCH anything my coworkers do, it is THEIR code and THEIRS to command, hands off! Even when I sneak a peek and find obvious bugs, I am not allowed to fix them because it violates territoriality.

Not sure how long you've worked at your current job, but this attitude might have developed because of a bad experience with an 'agile' methodology. Some shops have used this as an excuse for chaos, and instead of peer review and pair programming you get people changing code they don't understand or arguing about 'for (;;)' vs. 'while (1)' or tabs vs. spaces or a config parser or any other silly thing. It could also be caused by of favoritism or management with an agenda.

It's not always just ego that drives programmers to act like this, sometimes people are driven to it by group politics and mistrust.

You could try making your suggestions directly to the code owner rather than in front of your peers. It might make them less likely to react defensively.

Re:Possible explanation?

[Gonzoman]

I think the difference between OSS and proprietary software is this:

I find propietarty software restricted to what the developer originally had in mind.

With OSS someone has almost always included the feature that I would realy realy like that package to do.

Sorry abour the bad typing: I'm on my tenth bottle of beer.

Re:Possible explanation?

[mrjb]

Two more possible causes. If I work on a project for my employer, it is done with the tools of their choice (ASP for example), which sometimes requires to twist code in misterious ways to make it work.

Second, starting over from scratch is often not an option in closed source, and sometimes building along on ill-designed legacy software needs dirty hacks to make things work.

When I work on a project for myself, I use the tools that best help me to do the job, and I have the freedom to start over from scratch if new insights require so. Sure enough, using sub-optimal tools and 'dirty hacks' leave more room for bugs. Especially dirty hacks give bugs of the hard-to-find kind the opportunity to sneak in.

Re:Possible explanation?

[Osty]

I find propietarty software restricted to what the developer originally had in mind.

That's not entirely true. Specifically, if your closed source app includes a decent plugin architecture and enough documentation to get going, you're no longer restricted to what the original developers planned. For example, take Winamp. In the broader sense you're still limited to playing media, but the original developers surely didn't write it with playing music from NES ROMs in mind. Yet because they build a decent plugin architecture for various components (input, output, visualization, general), winamp can do just that. Another good example would be Internet Explorer. It was designed to be a best-of-breed web browser, but you can do so much more with it. You can embed it in your own applications, you can extend it, you can write activeX objects that allow it to display other formats than HTML (like pdf or Word documents).

Having source code is not a requirement for extending an application to suit your needs if the developers had enough forethought to allow for that with proper architecture. Granted, if the architecture isn't there then you can't do much, but that's more the exception than the rule these days.

Re:Possible explanation?

[HeresJohnny]

If no one else has done it yet, this is an early step in the process to making something in the OSS world.

What can follow is that others see what was done, if it works, it's used, if it doesn't someone rewrites it so it does work. This brings it closer to being the piece of software it needs to be to serve the intended purpose.

Debugging is part of this reworking. I don't know if this debugging is any quicker than closed source projects, but it's being done by folks who need the functionality in something they are doing.

Re:Possible explanation?

[BlackHawk-666]

The Linux kernel is over 4,000,000 lines of code now. That's definitely bigger than plenty of other closed source programs.

Re:Possible explanation?

[MrResistor]

Your "untold number of eyes" is nearly indistinguishable from "0" unless your open source project is widely used. ...
That may be true in theory, but it doesn't pan out in practice. In practice, most open source projects will only be seen (code-wise) by the author(s). If you're lucky, you might have two or three active users that will submit bugs or patches, but often that's not the case.

I think you've missed the crucial point there.

While it may be true that in reality nobody is looking at the code, there's always the potential for millions of people to be looking at it, and there's certainly no way to tell that when you're writing it. Even if nobody cares about your little project today, that could totally change tomorrow. I very much doubt that anyone writes OS code under the assumption that no one is ever going to look at it. It doesn't matter if anyone ever actually looks at it, you still write it as if they will.

Re:Possible explanation?

[beerman2k]

Any software development team doing only black box testing is foolish. Proper testing needs to be both black box and white box.

Re:Possible explanation?

[Pflipp]

You went to the source when you had issues, yes, but with a well-documented library you wouldn't need to do that.

I would as soon as I had issues. Now what do you want to say; that not-widely used software gets no reviews, or that widely used, well-documented software doesn't?

"I can audit the code myself to see what it's doing," but how many people do you think actually do that? [..] I know I don't, and I don't know anybody who does.

But now you do.

If you keep claiming nobody ever looks at OSS source code, then I'm starting to wonder who is making this stuff. But the answer is simple: a small group of programmers and a larger group of interested people, bug reporters, etc. that spam the programmers with bug reports as soon as they're not satisfied about something. Maybe looking at the source would still be the last thing they'd do, but the overall involvement is much bigger than in the case of "like it or just don't buy it" software.

Re:Possible explanation?

[Pflipp]

Yup, I'm still wondering what (if any) the guy on the other end of the line was thinking when reading my email :-)

Re:Possible explanation?

[Osty]

If you keep claiming nobody ever looks at OSS source code, then I'm starting to wonder who is making this stuff. But the answer is simple: a small group of programmers and a larger group of interested people, bug reporters, etc. that spam the programmers with bug reports as soon as they're not satisfied about something. Maybe looking at the source would still be the last thing they'd do, but the overall involvement is much bigger than in the case of "like it or just don't buy it" software.

I'm not saying that nobody looks at the source code. That'd be blatantly untrue. What I'm saying is that the commonly held idea that open source is good because you can audit the source code before using it is overstated. You can, but most people don't. Other than the hardcore, overly paranoid, or zealous open sourcer, few people care. They just want to use the software, not spend days poring over the code before they feel safe to use it.

As far as bug reports go, you don't need source code in 99% of the cases unless you're the kind of person that likes to submit patches with your bug reports. In fact, Microsoft's Dr. Watson is the best idea for reporting problems in closed source software. When you have a crash, Dr. Watson records a debuggable dump and gives you the option of sending it directly to Microsoft. Sure, they don't evaluate every dump they get, but getting a number of dumps on the same issue means there's a bug, and the dumps have enough information (not PII) that they can debug more easily without having to deal with vague and often incomprehinsible repro steps given by a customer.

There are certainly successful open source apps with lots of contributors and bug reporters and people who look at the code. However, given the total number of open source applications, the successful ones appear to be the minority. Yes, the same can be said about closed source software, but at least the guys writing that aren't working under the assumption that hundreds or thousands of people will help them debug the code. Ask the Mozilla guys about the harsh realities of open source. For the first couple years of the project (and even today, to some extent), almost all of the work done on Mozilla was by Netscape employees. They put the Mozilla source code out there, and aside from a few hardcore hackers who had been itching to get their hands on it, very few people stepped up to help. Yes, they did pull it off and released a rather nice product (XUL aside), but Mozilla is still largely Netscape code.

Re:Possible explanation?

[larry bagina]

I prefer white boxes, myself. Of course, I wouldn't kick Mariah Carey or Halle Berry out of my bed.

Re:Possible explanation?

[ColaMan]

I wasn't trying to say that this process is no good - I was just pointing out that there's a lot of code out there that's just like that. It's good to have that code no matter how buggy (or strangely commented!).

I've been *very* glad at times to have someone's quick hack available (even if it doesn't work for half the time) when I need to do my own quick hack.

In other news ...

[The Clockwork Troll]

Studies reveal that debugging is easier when you do not strip symbols from binaries!

Re:In other news ...

[deadsaijinx*]

in other news, studies reveal that common sense wins again.

Re:In other news ...

[deranged unix nut]

...studies reveal that bugs are found more quickly when people actually test software.

Unfortunately, I've talked to a couple closed source software firms who (unlike my current employer) don't test software. According to them, their customers don't want to pay for software to be tested.

Yes, I test software for a living, and it is closed-source, and I'm fairly certain that I do a much more complete job of testing, and I try more devious tests than are performed against most OSS projects.

If someone wants to pay my salary, I'm sure I could file a thousand bugs a year on linux packages.

Re:Who's surprised?

[M.C. Hampster]

What's so surprising that OSS is easier to debug? You don't see Windows anywhere near bug free do you?

Ah yes, because we all know that Windows is, in fact, the only closed software in existence.

Re:Who's surprised?

[Anonymous Coward]

what software package do you know of that is bug free? (besides solitare;) )

Not surprising

[worst_name_ever]

Debugging is always faster when you have a bunch of viral Linux commune hippies on the job!

Faster to debug?

[Anonymous Coward]

Why does it matter if the source is open? The article should be talking about software developed by a team of individuales (including OSS) vs. software by one individual. If the source is open, there may be additional people looking at the sourcecode, but if there are many contributors the source may be a big mess of 50 different coding styles. (can't agree on a single way to splice a string, etc)

--
Getting too much pr0n?

Re:Faster to debug?

[whizzard]

Why does it matter if the source is open?

When a project is open source, every user becomes a potential bug fixer.

Re:Faster to debug?

[tomstdenis]

While generally that is true you'd be surprised how many people trust you at face value. [cheap plug] one of my OSS projects is a crypto library. You'd be surprised how few people actually verify test vectors for themselves before deploying a project using my lib.

While I [as I suppose many OSS developers] try to write good code, all too many end users [often other developers] will simply plomp a library in and use it.

How many times can you honestly say you verified the source for vorbis/ogg or zlib or etc... for corectness?

Tom

Re:Faster to debug?

[Otter]

Why does it matter if the source is open? The article should be talking about software developed by a team of individuales (including OSS) vs. software by one individual.

There are broadly speaking two types of open source projects, often called baazar and cathedral, as put by Raymond [8]. Baazar projects such as Linux release new versions as often as possible, while cathedral projects release new versions at a much lower pace; in that respect cathedral projects are similar closed source projects, hence, cathedral open-source projects and closed-source projects have the same dynamics in our model, thus we shall refer to them as closed source, and to baazar projects as open source.

They actually did think of that -- and decided to just use the wrong terminology.

Re:Faster to debug?

[Geekboy(Wizard)]

That's why style(9) was created. Do it one way, the right way, MY way. ;-)

(Also, ask all of your contributors to send patches in your style. If you are the main coder for a project, that is an easy thing to handle. If you are part of a team, have the team code in a particular style. Doesn't matter which one, as long as you guys are consistant.)

Re:Faster to debug?

[a_n_d_e_r_s]

Most developers does not want to waste time making sure software works expecially not when a lot of others are using it and it works for them -
they are assuming, like most other users, that
the code is correct and if not; someone will
file a bug resport about that fact.

They are confident of the fact that this report
establish; The open code contains fewer bugs.

Look at lifecycle

[Anonymous Coward]

In the open source world, people are more willing to let a project die and be replaced than in close source.

I know when I work on open source, if something is really ugly and needs to be replaced, I do. In a close source world thats often considered too risky and old unmaintainable code hangs around forever.

Re:Look at lifecycle

[youaredan]

That's a prime example of survival of the fittest, and the true formula for the best software. Competition is the key, and not just inter-OS competition like Outlook->Evolution, but things like different ftpd variants, different crons, syslogs, etc - Gentoo Linux has most openly encouraged this variety from what I can see..

Re:Who's surprised?

[youaredan]

I wasn't implying that - just that the LARGEST example of closed source in existence... I mean, the bug fixes in things like phpMyAdmin or Apache etc get fixed much faster than things like IIS, Office, etc... probably because the manufacturer feels the need to "batch" updates because they don't allow the user the ability to do nightly updates, or any "non-milestone" updates. When stuff breaks, its fixed and gets sent out to the world fairly quickly...

Not very impressed

[zapp]

As with all statistics, you can make them say whatever you want...

Maybe most OSS projects are easier to debug because of lack of features, or smaller scope, etc.

What percentage of OSS projects, on say sourceforge.net, have a version number 1.0? (and are "widely" used). The first one that comes to my mind is MythTV and/or FreeVo. I can't speak of freevo, but mythtv (while being impressive) is still very bug ridden and has been out for over a year.

Another factor is the user group, of course. With OSS I imagine the kernel gets more bugs submitted by users than mythtv, just because the users aren't so much code hackers... they just want to use it.

The one rule in the software engineering is that there are no rules.

Re:Not very impressed

[deadsaijinx*]

The one rule in the software engineering is that there are no rules.

YOU FOOL! Your paradox has just ripped a hole in the space time continum! Now the planets will collapse upon themselves, destroying the entire universe. We will all enter a parrallel dimension the size of a pin tac. it will be very cold. jerk

Re:Not very impressed

[Otter]

As with all statistics, you can make them say whatever you want...

While I applaud all the people trying to propose alternative explanations for their findings -- if you read the paper, there are no findings.

They have a statistical model that shows that "release early, release often" will cause bugs to be fixed faster, all things being equal, and that a handful of projects (Linux, Mozilla) have scaling behavior that fits their model. That's all.

Re:Not very impressed

[malfunct]

What confuses me is whether the actual debugging process is slower, or whether it is the bug fixing process?

The speed of debugging the actual project is a function of code complexity and code quality which is not necessarily related to whether it is open or closed source. It may well be that more open source projects are less complex, or higher quality than closed source but this does not mean that if your project is open source it would be easier to debug.

On the other hand bug fix speed may well be impacted by the release model. If you release all the time (daily in some open source models) you would nearly always see bug fixes faster than a process that releases less often. Again this is not a function of open source or closed source. I will admit that open source projects allow "releases" more often because usually you can grab code directly from the CVS.

Re:Not very impressed

[rmohr02]

What percentage of OSS projects, on say sourceforge.net, have a version number 1.0? (and are "widely" used).

I'd say that projects saying their development status is at 5 - Production/Stable would be a better comparison.

Looking through the first couple pages I see phpMyAdmin, Gaim, BitTorrent, and NTFS support for Linux. I'm sure there's more widely used apps there, but I either don't know them, or didn't want to look through all the pages.

Re:Not very impressed

[dasmegabyte]

Truedat. Plus, in OSS, there's often a very coherent problem, which requires a certain solution. If you need a journaling filesystem, you take a filesystem and add journalling. If you think that the solution other people are working in needs advanced metadata more than it needs journalling, you take a filesystem and add advanced metadata. You don't try to add both, at the same time.

And that's what every company I've ever worked for has tried to do. Because it's integrated, package d solutions that make money for software vendors, not apps that do one thing great. I'm writing a word processor right now because I want a word processor that has three things: a big window, a built in thesaurus, and built in MLA markup. I will never be able to sell it, because it does 1/100 of what MS Word does, and MS Word is only $100. Even if it solves peoples' problems, it's just not fancy enough to be worth money.

Re:Not very impressed

[shaitand]

As for the speed of debugging the actual project, closed source is equivelent to your desktop pc and open source is equivelent to a monster 2000 pc cluster at a univerisity. Each pc is no faster than your desktop, but they function in parallel...

As for the second, release often.... technically the most current source is always released due to CVS, it would be impossible for closed source to compete with this. And a closed source project cannot release as fast as an open one, I can release a new version on a whim, they have to package it, convince people to buy it, and sell it. It costs me nothing, it costs them at least $500 for all those cd's...

Re:Not very impressed

[smeenz]

I have to agree. I've been seriously trying to use openoffice over the last few weeks, and several times it has really #$%$#ed me off because of buggy features, to the point where I nearly gave up and went to use excel or word instead.

I found that when making a chart in the spreadsheet package that the undo list was very unreliable. If I made a few changes, then undid more than 1 or 2 of them, it would not go back to anything like what I had originally. Even if I reapplied (redo) the changes, ie, undo/undo/undo/redo/redo/redo, I didn't end up with what I started with.

Secondly, Mozilla is perhaps one of the biggest open source projects. A quick trip to bugzilla.mozilla.org will show bugs that have been waiting to be fixed for serveral *YEARS* now. Reading the comments on some of them, it's clear that there are some very strong egos involved. In several cases, where a large group of people want "x" changed, it simply doesn't happen because the developer doesn't believe in that same change. That's a very closed minded attitude for an open source project.

You can rant and rave about microsoft's shortcomings, but by and large, if you try and do something with their products, it will either work, or the app will crash. What doesn't often happen is that the function "sorta" works.

Re:Not very impressed

[nathanh]

As with all statistics, you can make them say whatever you want...

This is false. An urban myth. Something that people like to say to make themselves sound more knowledgeable than they really are. The reality is that statistics is a mathematical field and it is as rigorous as any other mathematical field.

The real problem is that people think they understand statistics when they do not. They will claim "statistics say blah" when the statistics say nothing of the sort. The non-mathematical audience blames the statistics instead of their own ignorance.

Re:Not very impressed

[deranged unix nut]

That depends on the size of your test team.

Developers can only do so much at one time, if they can only fix an average of 5 bugs per day, then 400 testers each finding 5 bugs per day is overkill since some bugs will hide others and some fixes will introduce new bugs.

On the flip side, if a tester focuses on testing a program long enough, the tester can write automated test code. Personally, my test automation for the closed source project that I am employed to test performs approximately three weeks worth of manual testing work in a 30 minute test run.

That's funny, I've purchased several closed source software packages over the web, and other than bandwidth, it didn't cost the developer anything to release that new version.

Re:Not very impressed

[Mostly a lurker]

You can rant and rave about microsoft's shortcomings, but by and large, if you try and do something with their products, it will either work, or the app will crash. What doesn't often happen is that the function "sorta" works

Not often, only on the good days -- or is the reference to the products of some other Microsoft company I do not know about.

Re:Not very impressed

[Mostly a lurker]

As with all statistics, you can make them say whatever you want...

This is false. An urban myth. Something that people like to say to make themselves sound more knowledgeable than they really are. The reality is that statistics is a mathematical field and it is as rigorous as any other mathematical field.

Thank you for correcting my misunderstanding on this. I have always naively assumed that the raw data selected for analysis was somewhat important. In the future, I shall never doubt the result of those benchmarks I see reported as long as the numbers are correctly calculated using appropriate statistical formulae.

Re:Not very impressed

[mickwd]

So what you're now questioning is the raw data to which the statistics are applied, NOT the statistic techniques themselves.

The problem with statistics is that people try to use them in such a way to try to "prove" the correctness of what they are trying to say, or the "goodness" of what they are trying to sell.

Re:Not very impressed

[greenrd]

I found that when making a chart in the spreadsheet package that the undo list was very unreliable. If I made a few changes, then undid more than 1 or 2 of them, it would not go back to anything like what I had originally. Even if I reapplied (redo) the changes, ie, undo/undo/undo/redo/redo/redo, I didn't end up with what I started with.

Did you file a bug report?

If no-one has a record of the bug and how to reproduce it, they're unlikely to fix it.

A quick trip to bugzilla.mozilla.org will show bugs that have been waiting to be fixed for serveral *YEARS* now. Reading the comments on some of them, it's clear that there are some very strong egos involved. In several cases, where a large group of people want "x" changed, it simply doesn't happen because the developer doesn't believe in that same change.

Typically, in my experience, it's not that the committers are opposed to any change, they are just opposed to the proposed half-assed changes.

There are thousands and thousands of bugs in the database and simply not enough active developers to fix them all. Besides which, effort is being shifted to the new Mozilla Firebird project.

Re:Not very impressed

[smeenz]

No need to file a bug report as several have already been submitted

http://www.openoffice.org/project/www/issues/sho w_ bug.cgi?id=8009
http://www.openoffice.org/project /www/issues/show_ bug.cgi?id=15641
http://www.openoffice.org/projec t/www/issues/show_ bug.cgi?id=5802

2 of those haven't been touched for some time.

I only mention this because the whole topic here is about how things get fixed more quickly in open source projects, and I know of several glaring examples of where this is not the case at all.

As for mozilla, one of my pet peeves is this one -

http://bugzilla.mozilla.org/show_bug.cgi?id=8935 0

A 'bug' originally submitted in October 2001 asking for the Home button to be restored to its rightful place on the tool bar, like every other browser on the planet (okay that's a generalisation). My point is that despite everyone agreeing it should be there, and with some people even writing and submitting the code to do it themselves, the assigned programmer refuses to accept the patch or even agree that some people might want it there.

Comment #78 (not mine) sums it up nicely:

"I am appalled.
38 votes, 77 comments.
Wontfix because I don't want to."

I apologise if this sounds like flamebait, but it does irritate me somewhat.

Re:Not very impressed

[Anonymous Coward]

Actually, you're not quite synical enough. Stat's are used/abused on a daily basis. It's not about the figures, it's about the questions used to represent the figures.

E.g. DNA testing for criminal proceeds was once said to be as good as 1 in a million. So when the police get a suspect and the DNA matches, we have guilty verdict.

Well no. It merely says that in a country like the US, there are statistically speaking around 240 possible matches. Doesn't sound quite the same does it?

In this case, close family and ethnic origin will probably offer up the 239. (DNA testing has hopefully moved on to a more detailed testing, but there was a big cost factor with the improved accuracy.)

So you see we have the same figures, but completely different ways to influence someone.

In market research, this kind of usage is very common. When a client want their results to appear better than their competition's, you merely add addition critera to a question to filter the data until the figures show what you want them too. Once you've got the figures, you then rephrase the question to be somewhat less obviously manipulative. Then the final report gets written showing x is better than y.

I see this very regularly because I wrote the software that facilitates it.

Re:Not very impressed

[nathanh]

Thank you for correcting my misunderstanding on this. I have always naively assumed that the raw data selected for analysis was somewhat important.

Selecting raw data is part of statistics. Your sarcasm is unwarranted.

Re:Not very impressed

[Mostly a lurker]

Selecting raw data is part of statistics. Your sarcasm is unwarranted

Selecting raw data certainly is part of statistics. It is not, however, one of the mathematically precise areas of statistics. It is a fact that people very often select data to prove whatever point they are trying to make. Professional statisticians are not immune to this tendency.

Re:Not very impressed

[nathanh]

Selecting raw data certainly is part of statistics. It is not, however, one of the mathematically precise areas of statistics.

Tough. Measuring raw data for a physics experiment isn't precise, either. You're supposed to know and account for that. Errors during measurement DO NOT invalidate the rigorous mathematical nature of statistics.

It is a fact that people very often select data to prove whatever point they are trying to make. Professional statisticians are not immune to this tendency.

And that is a failing of the person, not of the statistics. People can abuse any science for their own agenda (look at eugenics) but that's a mistake a person makes, not the science. Think of Pons and Fleischmann if you want an example of another field selecting data to "prove whatever point they are trying to make". Their actions didn't cause an outcry "physics can prove whatever you want!" because most people realised the fault lay with Pons and Fleischmann, not with physics.

Re:Not very impressed

[scotch]

Whoever modded this offtopic is a sad, humorless, and droll excuse for a human. Parent post is fucking funny.

Re:Not very impressed

[scotch]

Well really, that's not so much a bug as a design choice.

Re:Not very impressed

[AngelofDeath-02]

Perhaps if statistics were not so widely used to convey a desire idea or thought, we might blame the statistician, not the statistics. This however is not the case.

Faster debugging in Open Source

[Cato the Elder]

The paper's conculsion seemed to be that debugging open source projects is faster because you don't have a version problem where customers report bugs in code that has already been modified for the next version.

I don't buy it. Many open source projects (ACE/TAO, Mozilla) for instance have large customer bases using non-current versions, and presumably finding bugs. Sure, if you only want to fix the bug in the released version, its faster, but it's not like closed source vendors don't have the source code to their previous release to debug with.

Sure debugging is faster if you always make everyone upgrade to the latest version before filing a bug report. Good luck getting mass acceptance with that.

Re:Faster debugging in Open Source

[Zork the Almighty]

Good points, however when developers fix a bug in the latest version, can't they *sometimes* go back and fix earlier versions as well ?

I think that the most debugging benefit that open source provides is that knowledgeable users sometimes do all the work for you...

Re:Faster debugging in Open Source

[iabervon]

Frequently, if a bug never got reported in an old version, it didn't get fixed, and it's still present in the new version (unless the project has taken to rewriting the whole thing frequently). If a bug has been reported in the old version and fixed in the latest version, you can frequently determine this, because you have access to the bug reports and mailing list archives. So you know if upgrading will fix your problem.

It's also possible that you get can and apply a patch that solves your problem. I've done this when I was having a problem with the latest (at the time) release of JBoss that had a fix proposed. I got the source to the version I was using before, patched it with the patch from the mailing list, built it, and it worked fine. Try doing that with a closed-source product.

Re:Faster debugging in Open Source

[deranged unix nut]

Good points, however when developers fix a bug in the latest version, can't they *sometimes* go back and fix earlier versions as well ?

Sure, it happened dozens of times in windows after the start of the security push. First they fixed the bugs in Windows 2003 and XP (before the products were shipped), then they went back and put those security fixes into Windows 2000 service packs.

bug reports

[DarkSkiesAhead]

I'd imagine that one of the most difficult parts of debugging for a large OSS project is dealing with the deluge of bug reports.

Re:bug reports

[Lairdsville]

The site linked just says:
Sorry, links to Bugzilla from Slashdot are disabled.
Grrrr.

Re:bug reports

[Gherald]

"Sorry, links to Bugzilla from Slashdot are disabled."

Such innovation!

Re:Who's surprised?

[SweetAndSourJesus]

What do you want, Windows nightlies?

The very concept fills me with dread.

I thought this was agreed on ....

[Anonymous Coward]

"Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."

I thought that this was the one pro OSS argument that was the least argued; release early and often (since the only way to simulate all the different conditions your software will encounter in the outside world .... is to release it to the outside world)

This is also based on the fact (?) that the hardest part about squashing bugs is finding them ...

Is this actually controversial? (seems pretty much common sense to me, but I do not do programming as a profession, have I been misled?)

PS

Not to take anything away, it's one thing to have something be so "obvious" as to be "common sense", and quite another to have some data to back it up ...

Re:I thought this was agreed on ....

[ClosedSource]

"In closed source software, the only way to find software is to pound on software hard and look for problems. You are going backwards from effect to cause, and you can never actually look at the software."

I guess the "you" you are refering to is the user, not the programmer. Closed source teams can examine the code, of course, so testing is not the only option.

Models?

[DogIsMyCoprocessor]

From the abstract -

We introduce a model of software bug dynamics where users, programmers and maintainers interact through a given program. When the program is written from scratch, the first phase of development is characterized by a fast decline of the number of bugs, followed by a slow phase where most bugs have been fixed, hence, are hard to find. For a given set of parameters, debugging in open source projects is always faster than in closed source projects. Finally, we determine qualitative lowers bounds to quality of Linux programmers.

I have introduced a model of sociological models. For almost all parameters, the sociological model turns out to be complete and utter bullshit.

Mediocre Propoganda at Best, A Joke at Worse

[Gabe Garza]

I know this is Slashdot and the party line is "OSS Rules!," but this seems pushing it even for this audience.

This was a paper written for a class on statistics. It was not a rigorous study. Their findings are based on a lot of assumptions. They have a very small sample set--they only test their model on Linux, fetchmail, and Mozilla. Many people, including myself, consider these the cream of the crop so far as OSS goes.

Before you praise it, I urge you to actually read the paper. Don't be intimidated by it--FUD is FUD, even if it's mixed with a heavy does of greek letters and charts.

Re:Mediocre Propoganda at Best, A Joke at Worse

[Chris_Stankowitz]

You have angered the /. gods now.

Mod parent and grandparent and great-grandparent down.

Also, mod parents children down.

Also, mod great-great-grandparents great-great-granddaughters down.

Also, say up unto them verily, that the mod of the parent will be cast down the generations to be a mod on the children, and on the children's children, and on the children's children's chilluns.

And also, mod down the nephews of the parents of the sibilings of the grandparent for though they be trolls or flaimbait, they are righteous in the eyes of the moderators.

And thou shalt visit the mods onto the descendents on through the generations, for I, your Mod, have smote upon thee a mod pestilence that shalt not be lifted until the second coming of the JonKats.

Thanks be to Mod, Amen

Re:Mediocre Propoganda at Best, A Joke at Worse

[Graspee_Leemoor]

"Mod parent and grandparent and great-grandparent down.

Also, mod parents children down.

Also, mod great-great-grandparents great-great-granddaughters down"

You remind me of the Emperor in Episode 1:

"Mod them down. All of them."

graspee

Re:Mediocre Propoganda at Best, A Joke at Worse

[ArmorFiend]

Two of the three open projects you cite also happen to be bigger than most projects, with more developers working in || at the same time than most projects. Does that help them? Read "the mythical man-month" for the answer. (Hint: no).

Re:Mediocre Propoganda at Best, A Joke at Worse

[Gabe Garza]

Two of the three open projects you cite also happen to be bigger than most projects, with more developers working in || at the same time than most projects. Does that help them? Read "the mythical man-month" for the answer. (Hint: no).

I agree that Mozilla and the Linux kernel are very good pieces of software--I use them each for many hours each day. I don't have a problem with them: I have a problem with using only three projects has a sample size. They're attempting to make a precise model to describe the difference in debugging time between two different development methodologies. If they want to win me over, they'll need to show that it is mostly correct for a wide variety of both open and closed source projects.

No one will argue that there are some open source projects that are world-class, and has good or better then any closed source projects. Any statement stronger then that is going to need a stronger foundation then this paper can offer.

Re:Mediocre Propoganda at Best, A Joke at Worse

[scotch]

See the post where the parent was stolen from HERE.

Of course debugging is easier...

[sterno]

I do a lot of coding working mostly with open source products, and sometimes closed source. When I get some bug come up in an open source product, I actually go digging into their code sometimes to figure out what went wrong. If it's closed source I can dig down through my code, but once I hit their code, it's a brick wall.

Frankly this is why I try to stick to open source software when I do development work. Hell of a lot easier to figure out how something works when you've got code and direct access to the developers via a mailing list.

Re:Of course debugging is easier...

[Dalcius]

I have to concur with this. My day job is working as a "Software Analyst" -- in short, I get problem reports that our setup folks can't figure out and I solve them. This usually requires a lot of digging through code.

Just this week we've had a major problem with one of our customers: IE is continuously crashing (an internal IE problem). It doesn't help that the customer isn't very good at giving us detailed information about who it happens to when, but what makes it even worse is the cryptic nature of the error information from IE and the fact that we have no access to source code.

On the other hand, much of our newer products are built on top of open source tools (which saves us an extreme amount of time). The few problems that we've had have been relatively easy to fix due to the more open nature of source and documentation with the OSS tools and services that we use.

Absolutely false!!!

[si_brain]

Because gdb is so slow with a realy big project.
( ...Symbol loading... )
Trust me on this, I'm in this situation every day.

Easy explanation

[meta-monkey]

Well, I think the explanation for this is pretty obvious.

If you've got open sores, you're going to want to get bugs off of them as quickly as possible. You're also going to notice sooner because it's still bleeding. If you've got closed sores, you might not notice flies buzzing around them near so quickly.

Re:Easy explanation

[cballowe]

Not quite -- some bugs are good for open sores as they clean out the dead tissue and allow them to heal faster -- haven't you ever seen Gladiator :P

Microsoft Refutes Oxford Researchers' Conclusions

[xelph]

A spokesperson at Microsoft refuted the conclusions of two french researchers from Oxford University this afternoon, saying that the business model behind Open Source was flawed anyway, since fewer bugs meant less urgency in updating to newer versions of software where old annoying bugs had been eliminated (only to be replaced with fresh one in anticipation of the subsequent forced release). The spokesperson also mentioned the enormous success of Microsoft's recent Closed Source initiative, under Bill Gates's supervision, to make computing more stable and secure, and finished by indicating that the UK government, who is being turned by Microsoft into a strong Open Source opponent (see recent Slashdot story), belonged to them anyway, and that the "frogs" would be deported to France shortly.

Re:Who's surprised?

[youaredan]

No No.. I wasnt even implying that was possible... Redmond hasn't even come to know and love publically selected mirrors... We are stuck with becoming psuedo world citizens in the sense that we are limited by the fact that everyone else in the world is using windows update... I hated that about windows... now, when I want to update anything - I emerge sync and emerge -u world in Gentoo Linux, and the world rejoices.

Re:Who's surprised?

[Gherald]

Are you running sol.exe as your shell? If not, well then that could be the problem...

Community Involvement

[fingers1122]

Well, with an open source project, users have direct access to the code and debugging is going to be faster because, essentially, your community of users is going to become part of the debugging process. With a closed source project, a team of developers is responsible for debugging the program, but with an open source project, the community is able to get involved.

ESR says...

[Realistic_Dragon]

If you read 'The Cathedral and the Bazzar' by ESR he gives some very good reasons for Open Source development being better at bug finding:

1) With enough eyeballs, all bugs are shallow - someone will know the answer, or stimulate the person who does.

2) Users provide better bug reports including line numbers, decent config information, possibly even patches.

To which you can add number 3 and 4 of my own devising:

3) The kind of people who write and use OSS care about security and stability (often to an extreme) and so think that bug fixes are essential not a nice-to-have.

4) Closed source developes have to deal with management and merketing wienies - it's a wonder they even get buggy code out the door.

Re:ESR says...

[cadfael]

Whereas I have to admit the reality of 4) at times, I do write closed source software, and oddly enough, I do care about security and stability. May I suggest that 3) should be

3) The people writing OSS don't have to deal with #4, which makes security and stability a more achievable goal.

(I can't argue with the reality of CS being often less secure and less stable - that is just 4) makes it so)

Re: ESR says...

[Black Parrot]

> 1) With enough eyeballs, all bugs are shallow - someone will know the answer, or stimulate the person who does.

> 2) Users provide better bug reports including line numbers, decent config information, possibly even patches.

As a part of these phenomena there is the direct interaction between the user and the hacker. A nobody like me can post a problem to the LKML and get a reply from than Alan Cox suggesting things to try that will give him more information about the bug. Or can post something to some project's bugzilla site and as often as not get a response from the lead developer on the very same day.

Compare this to the "help" desk bureaucracy for the typical CSS application, where they whole setup is arranged to buffer the developers from the people spotting the problems.

Re:ESR says...

[AsparagusChallenge]

#3, "who write and use OSS", implies something very important: the people who writes it has to use it. That's why they care.

Re:Debugging Software

[Anonymous Coward]

I'll bite.

http://www.gnu.org/software/ddd
http://www.gnu. org/software/gdb

This should cover what you need (as long as its on Lunix/BSD/Unix other than SCO)

Actually Cathedral vs Bazaar, not Open vs Closed?

[GersonK]

It seems that they're comparing models of a software developed with frequent releases (Bazaar, which they declare to be "OSS") against infrequent releases (Cathedral, which they declare to be "closed"). The always better that they come up appears to be based on the release cycle, not the visibilty of the code. (I haven't quite been able to parse how well their theory maps tot eh data they mined from linux and mozilla, though)

Bogus assumptions

[ckessel]

The paper makes some critical assumptions, one of which is:

all the users use the modified code at time t + 1 and report bugs exclusively on the updated code.This assumes that all the users update their software at every release.

This is completely bogus. Not every user is going to update immediately. In fact, the larger the company is the less likely this is due to their desire to qualify new software. This means as Open Source gets more popular with big companies, the more bogus this assumption is.

The paper also mentions nothing about QA efforts or beta sites on closed source, which are typically on the "immediately updated" products.

I'm not going to argue whether oss is better/worse than closed, but I heavily doubt this paper proves anything other than if you make lots of assumptions you can prove whatever you want.

Re:Bogus assumptions

[shaitand]

This is true, but any size business will update more often with open source software. OSS is generally the same version carried on for eternity, rewriting portions anywhere it makes sense. The kernel for instance, takes very small incremmental steps. There is also no licensing cost for the upgrade. So while large businesses will upgrade less often than small, large using open source will tend to upgrade more often than large using closed source.

Re:Bogus assumptions

[kbielefe]

I agree with you that every user doesn't upgrade immediately at every release. However, I don't know about you, but if there is a bug in some open source software, and that bug is annoying and obvious enough that I would want to submit a bug report, the first thing I do is upgrade to the latest version and see if it has already been fixed. If it hasn't, I submit a bug report with my workaround patch, and then upgrade when the bug is fixed.

First of all, upgrading is easier than submitting a bug report. Second of all, I don't want to feel like an idiot for reporting something that has already been fixed. And if you report a bug, you have to upgrade anyway or at least patch if you want it fixed. It doesn't cost anything, so why not take the easier route?

I'd say that as Open Source gets more popular with big companies, the less bogus that assumption is. It is in closed source that the assumption is bogus, because the cost of evaluating and qualifying a new version of closed source software is so much higher. Also, more open source upgrades are bug fixes only, where closed source upgrades tend to focus more on features. They throw in bug fixes for good measure, but you can't sell new versions of software based on bug fixes that most people didn't even care about.

Re:Bogus assumptions

[deranged unix nut]

Which explains why up until a recent BIND worm hit, I was running a 5 year old version...

In my experience, people don't upgrade unless absolutely necessary, and even then only if it won't disrupt other activities. Open source doesn't force them to upgrade any more than closed source does...

Re:Bogus assumptions

[Animats]

The analysis in the paper basically assumes that users of open source all run the latest development release, and are thus basically beta testers. That's unrealistic.

In fact, software distributed that way is terrible. For anything that has to work, you want a stable release with occasional bug-fix-only updates, with a clean separation from new versions with new features.

Silly Story

[Junks Jerzey]

Give a random developer 200,000 lines of code he's never seen. Ask him to find a bug. Odds are he'll have lots of trouble doing so. When he's finished, will it make any difference if you tell him that the code was open or closed source? A big project is a big project, period.

Re: Silly Story

[Black Parrot]

> Give a random developer 200,000 lines of code he's never seen. Ask him to find a bug. Odds are he'll have lots of trouble doing so. When he's finished, will it make any difference if you tell him that the code was open or closed source? A big project is a big project, period.

Most bugs are found by using the software rather than by reading its code.

The debugging difference between OSS and CSS is (mostly) only interesting in terms of what happens after the bug is found.

Re: Silly Story

[Type-R]

Most bugs are found by using the software rather than by reading its code.

Which is one of the reasons most code is crap. How many other engineering fields put things out there and don't think about problems until after it's borken. Hey jed, I think the bridge fell over again, let's try heavier bolts

See the OpenBSD project for an attempt to do this the other way around (i.e. audit the code BEFORE you know it's broke.)

Re: Silly Story

[Nuttles]

Agreed. I don't things are going to change anytime soon though. From my experiance (2 years developing), my supervisors push for features much more than stability or security.

Weak

[aztektum]

How come stories like these always seem like they get posted because of zealotry?

Re:Weak

[DrVxD]

It seems that way because it is that way

Re:Who's surprised?

[youaredan]

Real men play freecell

In other news...

[Bigmell]

People who use OSS extensively also tend to have bigger penises.

Re:In other news...

[Ricin]

People who use OSS extensively also tend to have bigger penises.

Please don't tell me SCO is after my penis now.

Programming Style

[bobthemuse]

I wonder how much of this is due to programming style. What I mean is, can there be a difference in how easy it is to debug apache (maintained and written by many people) versus an OSS project which was mostly written my one person? I'd imagine that a large number of programmers results in more 'generic' code, rather than the specialized shortcuts that are unique to most programmers. That would make it much easier to debug, IMHO.

Re:Who's surprised?

[len_harms]

what about minesweeper?

Workaround

[Gherald]

..paste the link into a different browser window. We don't want to /. them, however, so here's a snippet of what the link has... 31260 rows of this stuff: ID Sev Pri Plt Owner State Result Summary 350 min P5 All wtc@netscape.com ASSI PR_ExplodeTime() works only if given a PRTime argument be... 465 enh P3 All dougt@meer.net REOP FTP PORT (active) command support 540 enh P3 All myk@mozilla.org NEW Need feature to edit long description/comments 554 enh P5 All nobody@bugzilla.org NEW Ought to have a "show all recent activity" query. 639 nor P2 All saari@netscape.com NEW we need minimization notifications 753 nor -- All paper@animecity.nu ASSI Combined nsImage* & gfxImageFrame 915 nor -- All table@layout.bugs NEW column style resolution for text-align,vertical-align not... 937 nor P2 Mac sdagley@netscape.com ASSI Support MacBin encoding for FILE uploads in FORMs 972 nor P2 Mac sfraser@netscape.com NEW [FONT MAC] CSS font-weight: all font weights show as eith... 1289 nor P4 All table@layout.bugs NEW Absolutely positioning of table elements [ABS POS] 1334 enh P3 All leif@ogre.com NEW Support for LDAP v3 Controls 1336 nor P4 All leif@ogre.com ASSI Problem managing schemas using PerLDAP 1339 enh P3 All mwyner@netscape.com ASSI New SiteConfig.pm module 1382 nor P2 All leif@ogre.com ASSI Problems with UTF-8 encoded attributes 1388 nor P3 PC leif@ogre.com ASSI dmake/borland C (??) complaining about winsock.h missing 1428 enh P5 PC kmcclusk@netscape.com ASSI DDRAW bits compiled on NT don't display right on 95 1515 enh P2 All dbaron@dbaron.org NEW for dotted borders, want round dots instead of square dots 1570 enh P5 All leif@ogre.com ASSI RFE: Add getOption() and setOption() to OO layer 1574 enh P5 All leif@ogre.com ASSI RFE: new add() and modify() methods/functionality 1721 maj P1 Oth leif@ogre.com ASSI Can't compile/link with dynamic libs on AIX 1763 min P2 PC jkeiser@netscape.com REOP Frame border colours should be specifiable in CSS 1775 nor P2 PC kin@netscape.com NEW stream converters need to implement the new StreamConvert... 1781 nor P2 All kmcclusk@netscape.com ASSI 1px double border invisible on win32 (was: division of sp... 1785 nor P4 All joshua.xia@sun.com NEW prompt on status bar before starting up Java 1996 enh P4 All blaker@netscape.com NEW Support LONGDESC for IMG 2056 nor -- All block-and-inline@layout.bugs NEW display: run-in not implemented 2167 enh P5 All leif@ogre.com ASSI RFE: Add a synchronization method to Entry:: 2183 nor P5 PC khanson@netscape.com NEW windows localtime() bug exposed in javascript Date object 2212 nor P3 All table@layout.bugs NEW character alignment not implemented (align=char, charoff=... 2418 tri P3 All block-and-inline@layout.bugs NEW Problems with floating :first-letter 2652 enh P2 All mscott@netscape.com ASSI would like mailbox:// command to work in Navigator window. 2654 nor P2 All sspitzer@netscape.com NEW localPath does not take effect until restart 2657 enh P4 All ducarroz@netscape.com ASSI Intelligent Send preferences should apply to News 2662 enh P3 All ducarroz@netscape.com ASSI Allow "Linked Msg Type" to be Changed in Compose Msg window 2679 maj P2 All mscott@netscape.com ASSI Need to support submission protocol 2705 nor P3 Sun leif@ogre.com ASSI Problems with SHA encrypted passwords 2750 min P3 All harishd@netscape.com ASSI Line feeds after start tags and before end tags should be... 2769 nor P1 All nobody@mozilla.org NEW [LDAP] Auth with bogus credentials does not bind anonymously 2852 enh P3 Sun leif@ogre.com ASSI RFE: Better binary handling 2875 enh P5 All darin@netscape.com NEW Proxy: map HTTP 500 errors to necko errors (so Internet K... 2892 nor P4 All nobody@mozilla.org NEW [LDAP] RFE: Access to a local LDAP server in Off-Line mode 2894 nor

If it's so fast

[Anonymous Coward]

Then explain these open source FAILURES

1) The kweather applet in kde
2) The gtk file dialog
3) the latency in the arts sound sever
4) the lack of support for pci modems
5) the color scheme of games.slashdot.org

Re:If it's so fast

[Ricin]

"Then explain these open source FAILURES"

Lack of interest, lack of specific knowledge, lack of time, lack of income, lack of users, lack of specific hardware, lack of testers, lack of bugreports, lack of sleep....

They're just like humans you know.

Re:If it's so fast

[TheAwfulTruth]

I can explain that.

The original statement of "Always" is of course compeltely false. There is no such thing as "Always". In my personal experience with using such things as wxWindows etc, I find that debugging an app as a whole goes a lot SLOWER than when using something like MFC because wxWindows itself contains so many bugs that I am frequently debuggint IT instead of my own code. (BTW, MFC and MS's LIBC come with source, so debugging through it is pretty easy, compared to compeltely closed source)

Overall I find, then even though the MFC and .NET have their share of bugs, working with O.S. is frequently slower because of the "Fix it yourself" mentality you frequently run into with O.S. libraries and drivers.

OTOH, yes, sometimes Closed Source libraries will have a bug so severe that working around it takes a lot of effort.

So... 'Always"? Bullshit. I wouldn't even give it a "frequently". His, my or your experience in developing with O.S. or Closed Source will depend on exactly the bits of O.S. and C.S. that we use. but so far I do find that of all the O.S. and C.S. that I have used, the C.S. has been less trouble for my development debuggin wise. (Not to mention that my closed source debugger kicks the teeth out of GDB and KDevelop)

but I thought...

[H0NGK0NGPH00EY]

The one rule in the software engineering is that there are no rules.

I thought the first rule in software engineering was "you don't talk about software engineering."

Re:but I thought...

[deranged unix nut]

I thought the first rule in software engineering was "you don't talk about software engineering."

Nope, the first rule in software engineering is: "There is no software engineering."

Re:suprising?

[bladernr]

I could be suprising for a number of reasons:

1. Closed-source projects are often have a more structured process for development, along the lines of CMM, ISO 9001, established methodologies, etc. Things like CMM and ISO 9001 are assumed to result in higher quality (I seriously dispute that, but its the widely held assumption).

2. Many closed-source projects have customers with support contracts. Many open-source developers are not bound by such contracts and are not forced to fix bugs. It is assumed that suppliers with contracts to fix bugs will fix bugs (see Microsoft to see if that is a good assumption).

3. The quality of closed-source programmers is assumed to be higher. That is because the stereotype of the open source person is someone in college, unemployed, or in some other way with too much time on their hands. The sterotype of the closed-source (ie, corporate) programmer is that they went through a rigorous review and selection process to qualify to work on the project (that is assumed by people who have never worked in corporate IT).

Re:suprising?

[danny31415]

Because of the fact that it says ALWAYS. It seems like they didn't look very hard. I'm sure there is some open source project out there that debugged their code slower than some other closed source project.

ummm.....I hate subject lines.

[tx_kanuck]

w/o RTA, I would have to say that OOS is faster in debugging b/c you have:

1) More people who are looking for bugs.

2) Developers get public exposure and accolades for having good code (it's a pride thing)

3) There is no financial reason to not announce bugs.

4) Development cycle has little in the way of time constraints

5)A lot larger latitude in working conditions

Re:ummm.....I hate subject lines.

[Nuttles]

I think 5 is most important. I code at home twice as fast as work because I don't feel the pressure to produce every second.

Re:ummm.....I hate subject lines.

[DrVxD]

I code faster at home than at work primarily because the coffee is better...

cond-mat?

[the Atomic Rabbit]

Funny how these guys posted their paper to the cond-mat (condensed matter physics) section of the e-print archive, which, after all, has categories for computer science. The paper has nothing to do with condensed matter physics, aside from the fact that programmers are usually composed of solids and liquids.

Re:cond-mat?

[sket]

i guess the section was chosen because of the intended readership. since most computer scientist don't know about complex dynamical systems, power-laws and scale-free networks they wouldn't read that paper. in theoretical modelling of cond matter such phenomena are well known (eg spin glasses), as well as their application to other scientific fields like computer science or theoretical biology. of course it doesn't help much if such works are only noticed in the field of theoretical physics..

Thats because

[nother_nix_hacker]

Another, perhaps surprising conclusion is that debugging in open source projects is always faster than in closed source projects."

Thats because in closed source projects you have to dis-assemble the binary first! :)

Always ?

[FauxPasIII]

All generalizations are false.

Re:Always ?

[shaitand]

That's true. Closed source development is like a single processor pc, compared to open source development this is your massively parallel supercomputer. Now generally your massively parallel super computer is faster than a desktop pc, but it's NOT always faster... for instance, it's not faster if it's turned off.

Nice troll...

[siskbc]

...but of course there's a difference between not being ABLE to effectively debug something and not caring if you did.

Re:Who's surprised?

[Gherald]

the mortality rates are too high

Re: verified for correctness

[benjamindees]

You're missing the point. To a user, 'correct' means 'working'. If you write libraries, your users are actually developers, many times middle-men who don't use the application and don't care if it works. These developers may not look for and find problems, but, with OSS, the actual end-users or someone closer to them (sysadmins) usually will.

There were some errors in this paper

[jjeffries]

But they're fixed now, make sure to upgrade!

True...

[Psychor]

This is completely true. I've emailed Bill Gates many times asking him to fix the bugs in the copy of Windows 3.11 that I'm using, and he doesn't seem to have got around to making an updated version freely available yet!

Re:suprising?

[JamMasterJGorilla]

Why, may I ask, is this suprising (sic)?

It's surprising because it actually got published in something besides the Journal of Completely Obvious Discoveries. I hope they filed a patent on their method of coming to a conclusion.

Seems reasonable to me

[riptalon]

During development the closed source software will only be used by the developers, and in general the developers are not like their end users and may have little interest in actually using the software they have been payed to write. For open source however the software is likely to be available to users from a very early stage and the developers are likely to be active users of the software as well. It would be very surprising if the bugs were not squashed faster.

Once the software is released closed source has the problem that bugs will only be fixed if the producer sees profit in it. Major security bugs will be fixed "relatively" quickly, as they might impact future sales otherwise, but with closed source the producer may not fix known non-security critical bugs if they don't feel like it, and no one else can.

There is also a problem with bug reporting in the closed source world. Who actually reports closed source non-security critical bugs? There isn't a lot of incentive since they may not be fixed anyway and if they are the fix will likely just go into then next version (that could be a year or two away) and you will have to pay for. Also the fraction of the users that do not have a licenced copy are unlikely to report bugs.

Whatever the merits of this particular study's methodology the results are just plain common sense anyway.

Re:Seems reasonable to me

[deranged unix nut]

What you say is correct if you assume that the software development organization's management does not value software testing.

In an organization that values testing, there will be at least one tester per developer and the tester is responsible for representing the customer and ensuring that the software meets the needs of the user.

I have filed a sizable number bugs on a closed source project over the last few years, and most of those bugs were fixed before we released the product! Then again, I am paid a nice salary to find and file those bugs and my employer does care about the quality of the software that we release.

its the documentation!

[appleLaserWriter]

Successful OSS projects must be well documented in order to survive. Naming variables in an intuitive manner and providing insightful comments isn't about improving your annual review scores, it is about ensureing that others can and will read your code.

Companies like Microsoft need to introduce policies to create the same effect. Code reviews and extreme programming are good examples. They often degenerate into either a rubber stamp or a grudge match between different interpretations of Hungarian Notation.

Re:its the documentation!

[ClosedSource]

All that time spent making sure variable names are politically correct (Hungarian or otherwise) could be spend actually evaluating whether the code will function properly.

When was the last time you couldn't understand a section of code because they didn't follow the naming convention?

Re:its the documentation!

[mcrbids]

All that time spent making sure variable names are politically correct (Hungarian or otherwise) could be spend actually evaluating whether the code will function properly.

I frequently use a modifed form of Extreme Programming in projects, particularly where the issues that will be faced are not known at design time.

Typically, I focus on heavily documenting the data formats used, and code away from there. My rule of thumb is that anytime I feel the temptation to copy/paste, I break out the code into a function. As I do so, I revise variable names and leave notes.

Anytime code isn't immediately clear in these kinds of situations, I write all kinds of notes so that I understand what I'm reading when I get back.

You can certainly take naming conventions to a logical point of the rediculous, but compare these two pieces of code:

for ($c=0;$c$CC)
$cc=$c+$CC-5;
}

for ($dollars=0; $dollars$savings)
$mysavings=$dollars+$savings-$tax;
}

Now, it's clearly contrived. But the latter clearly has more relevant information than the former. (and I've seen all too much code similar to the former!)

-Ben

Re:its the documentation!

[ClosedSource]

I'm not against documentation, I just think a lot of energy is wasted on naming conventions.

If the issue were merely obfuscated names as your example suggests, I wouldn't have a problem with it. The problem comes when an attempt is made to embedded additional information in a name beyond its local meaning. Typically naming conventions are inconsistent, incomplete, and a distraction from more important issues.

Finding fixed bugs

[hysterion]

followed by a slow phase where most bugs have been fixed, hence, are hard to find

Duh. So it takes a "model of software bug dynamics" to see that it's a tad harder to find something which is no longer there ?

Their "model of iraqi WMD dynamics" is eagerly awaited.

The "many eyes" myth

[Anonymous Brave Guy]

A more likely explanation is the 'many eyes' that can review the code.

Many eyes can. How many actually do? Unless you're talking about the really big projects, probably very few indeed -- one, I suspect, in many cases.

It's not fair to cite mainstream developments like Linux or Mozilla as the way all open source is any more than it's fair to cite Microsoft's history on things like security and reliability as the way all closed source is.

Re:The "many eyes" myth

[Grant_Watson]

Many eyes can. How many actually do? Unless you're talking about the really big projects, probably very few indeed -- one, I suspect, in many cases.

I guess you can't eat open source, if OSS programmers are selling their second eyes.

Re:The "many eyes" myth

[Ramses0]

I've looked at a surprising number of program's source code. abcde, www-mechanize, ode physicis libraries, perl stuff, php stuff, python stuff, parts of the ogg utilities. Mostly because it doesn't have some feature that I want, or it does something that I'm interested in doing, and want to know how hard it is. It might not be much, but a cursory overview can tell you whether the author knew what they heck they were doing, and auditing even one code path (or even one function) can be helpful. Try it some time. If you're using debian: apt-get source abcde. I use it all the time as my "reference to how the hell bash programming works" :^)

--Robert

Re:The "many eyes" myth

[kbielefe]

How many actually do?

I'd say enough. I submitted a patch to fix a bug in a very small open source project that I was only evaluating for a couple of weeks. If I find a bug and upgrading doesn't help, the first thing I do is look in the source. It doesn't matter how big the project is. If I personally use it I want it to be bug free. I'm sure I'm not the only one who operates that way.

Now, I have never looked at the code wondering if perhaps there is a buffer overflow vulnerability in it. That doesn't interest me, but I know there are plenty of people who it does interest because we get a lot of security vulnerability patches without an exploit in the wild.

Re:The "many eyes" myth

[KidSock]

Many eyes can. How many actually do? Unless you're talking about the really big projects, probably very few indeed -- one, I suspect, in many cases.

Dispite the fact that the topic of this article is stupid I would have to disagree with your claim that the "many eyes" statement about OSS is a myth. I run a medium sized OSS project and only a very select few have ever contributed any worthwhile code but what is usefull is that occationally someone finds something genuinely important. A bug. A simple code change to get around a problem. Etc. It's actually one of the few things that I think isn't a myth about OSS. That and the benifits of having a large base of users against which to test the code. That's the real benifit. It's the claim that OSS projects benifit from many different code contributors that is a myth. I reject a lot of code that I'm sure would have otherwise been accepted other maintainers. It's the patchy collage of code that becomes a spineless inflexible blob.

Re:The "many eyes" myth

[CommandNotFound]

It's not fair to cite mainstream developments like Linux or Mozilla as the way all open source is any more than it's fair to cite Microsoft's history on things like security and reliability as the way all closed source is.

Actaully, in my experience I'm much more likely to dive into the code for a small app which is probably quite unknown. I rarely dive into the Mozilla or OpenOffice sources (for example) because they are so big and complex that it takes time just to get a build environment setup. Small projects like most sourceforge projects take ten minutes to get a build set up and completed.

Re:The "many eyes" myth

[dnoyeb]

disagree. I too have contributed to many open source projects without being a full member.

Some project to do with sound on linux.
JHotDraw
NetBeans
And about 2 others I can not recall.

This likely happens because everyones computer is different, and sometimes it does not work on their computer. Every so often a person gets a spare moment and/or really needs/wants that piece of software on their computer.

Re:The "many eyes" myth

[Anonymous Brave Guy]

If I find a bug and upgrading doesn't help, the first thing I do is look in the source. It doesn't matter how big the project is. If I personally use it I want it to be bug free.

Sure, but do you review the source code for every project you use, in its entirety, before you start? Or do you just look at it to fix bugs?

The proposition here, as I understand it, seems to be that it's easier to debug open source code because the code is more maintainable as a result of more people having access to it. But if no-one other than the original author and perhaps a very small number of reviewers actually looks at things routinely, then the theoretical advantage makes no difference in practice.

Re:The "many eyes" myth

[ProfKyne]

Hear hear. Someone on the PostgreSQL mailing list suggested that I investigate DbUnit, a utility for use with JUnit that restores a database to a known state after each unit test for situations in which mock objects are too inconvenient. It looked perfect... until I tried it with my database.

My schema's referential integrity constraints required the re-import of data to be done in a certain order. DbUnit officially only supports Oracle and Sybase (I think), and I suppose this wasn't a problem for those DBs... but I thought I had hit a wall that I could not cross with PostgreSQL. A simple look into the code showed that the change wouldn't be very hard to make. A few hours and a patch later, and DbUnit had new functionality that it did not have before. The point is, not only was I able to use the fact that the utility was open source to make it work better for me, but the change I made went into CVS and will hopefully be of use to others. It is not a myth.

Re:The "many eyes" myth

[kbielefe]

Of course I don't review the source for every project I use in its entirety. I have about 100 MB of compressed source installed on my gentoo machines right now. It takes me about an hour to thoroughly peer review a 15k source file at work. Of course, my method ensures that buggy open source code gets the most reviews. Code without known bugs is less likely to need review in the first place, or has already gone through review when it did contain known bugs in the past.

I don't think the argument is that the code is more maintainable (meaning easy to follow and change) because it's open source, although if you've ever had to maintain code that someone wrote thinking no one else would ever have to read it then you know what a difference "many eyes" makes. Of course, many people will routinely read your code if you develop at Microsoft (or another large closed source company), so their code should be just as maintainable. Remember, most open source developers and users have day jobs at closed source companies, so how can either group write substantially better code than the other?

The argument is that bugs get fixed faster in open source applications, not necessarily easier. By faster I mean the time between when the bug is discovered on an end user's machine and the time the bug is fixed on that user's machine.

For one, users have an incentive to report the bugs in the first place. They didn't pay anything, and want to give something back.

Second, knowledgable users include patches in their bug reports, or at least a pointer to the flawed function, making tracking down the bug much easier. Have you ever tried to fix a bug that you couldn't reproduce on your own system? Having a user that can reproduce it while running with full source capabilities in a debugger really helps.

Third, you can have a fix for an average bug installed on your system in a matter of a a few hours if you do it yourself, a few days if the developer fixes it and you get the cvs version, or a few weeks if you wait for the next official release, all without charge. Our company gets that kind of service with closed source software only on very expensive, somewhat specialized software that we use in our own development.

Most closed source software you have to wait months for a service pack or years for the next release, you usually have to pay for the privilege of having a bug fixed that shouldn't have been there in the first place, and pay for features you don't really want that introduce more bugs.

This whole process leads to more bug-free code than closed source actually installed on user's machines, which is where it counts. And don't even get me started on custom features.

Drink the kool-aid...

[HydeMan]

This is yet another example of someone trying to sell the OSS agenda. Statistics can easily lie, and I suspect that the person involved in the study was not a neutral party. A more important question is do people on /. EVER find fault in OSS? Surely it can't be a nirvana. Be honest, and get real.

Ego problem != closed source problem!

[Anonymous Brave Guy]

We have egos at work too. I am not supposed to TOUCH anything my coworkers do, it is THEIR code and THEIRS to command, hands off! Even when I sneak a peek and find obvious bugs, I am not allowed to fix them because it violates territoriality. This sort of code hording is impossible in OSS [...]

It should be impossible in a sensibly run closed source project as well. If you have a development team whose egos are such that they can't stand criticism and claim unique ownership, you are on the absolute lowest rung of the smart development process ladder. Teams full of such people will never produce good results, whether their source is closed or not.

As with many of the points raised in a thread like this, though, the problem is with industrial development processes and not actually with closed source. In a decent coding shop, no-one has sole knowledge or control of any area of code. In better shops, reviews are routine, and the good guys actively solicit feedback from their peers.

This is just another case (like the "many eyes" argument, and others) where the argument made in favour of open source looks great at first glance, but doesn't stand up to much scrutiny. To date, the most convincing argument I've seen for why some of the big open source projects have generally good code is still that those who do it frequently do so out of interest, rather than just to pay the rent (though certainly most of the work on the big OSS projects is done by pro's these days).

Re:Who's surprised?

[Sponge Bath]

What do you want, Windows nightlies?

Is that a euphamism for BSODs?

Re:Who's surprised?

[Sponge Bath]

Is that a euphamism for...

Shit! I've mispelled a word on Slashdot. I'll burn in hell for sure.

"Always" is the tip-off

[ClosedSource]

Any paper that concludes that one method is "always" better than another is almost certainly wrong. Be happy slashdotters, you can get away without reading the article this time.

Re:"Always" is the tip-off

[DrVxD]

That's because sweeping generalisations are always wrong...

Re:We are not brain-washed

[jefbed]

Open source is real, it is here, and it works. This sight is of course going to be filled with advocates of open source because it is part of the open source community. Open source in general has no universal agenda that is being "sold." Though slashdot is not quite a serious site, I doubt that they would post biased studies and false statistics (besides the polls). OSS is not nirvana, though Free Software comes closer. It must be acknowledged that there are universal problems with software development that cannot be solved with either OSS or proprietary methods. If you look closely at the history and accomplishments of OSS/Free Software then you will see its value. Not everyone is rich enough to buy quality proprietary software, nor does everyone want to do such.

Re:Who's surprised?

[raistphrk]

Windows nighties would frighten me even more. Imagining Craig Mundie modeling one of those would be enough to turn me off from sex for the rest of my life.

Re:suprising?

[fredrik70]

I think it's not only that OS projects got access to the source that does it. Many closed source projects are rushed and products that really should have been released as a beta is release as a gold version because of companies trying to cut cost, et, al. (early netscape and ms springs to mind), whereas OS projects are more released after the motto 'it's released when it's done'. THink of the stuff NASA does for example, closed source but they are *really* making sure that as good as all bugs are gone before they use it in a craft - costs be damned (this might change now with the cheaper-faster approach NASA had lately).

To quote Gene Spafford...

[DesScorp]

" Not always. A more likely explanation is the 'many eyes' that can review the code."

I went to a speech by Gene Spafford here a few years ago, when the subject of Linux code quality versus other systems (especially MS) came up. Someone mentioned Eric Raymond's "Thousand Eyeballs" theory, that more people looking at the code ensured better quality.

Spaff responded "that does no good if those thousand eyeballs are looking at things like networking your toaster instead of quality and security".

I don't think this point is emphasized enough. It's not enough that lots of people are looking at the code. You need lot's of people with training, expierience, and an eye for problems to look at the code. He pointed out that one of the biggest problems in development is that while people can learn C from a book, and even get good at it, they don't learn proper software engineering techniques, philosophies, and debugging skills that way.

In short, simply being open source and having lots of developers isn't a solution in itself.

Re:To quote Gene Spafford...

[AJWM]

Spaff responded "that does no good if those thousand eyeballs are looking at things like networking your toaster instead of quality and security".

On the contrary, it's likely to be somebody trying to do something with the code that the programmer never expected (like networking the toaster) that finds the problems. The programmer (assuming he's even halfway competent) will have coded against the problems he expects people to hit.

I remember one tester I had on a small project, she'd find ways to crash code that I would have sworn was uncrashable, and my response when seeing how she did it was almost invariably "why in the world did you want to do that?". (And then I went away and fixed it. Hey, it was Motif and C++ on a Pyramid, an ugly mix at best.)

Re:To quote Gene Spafford...

[deranged unix nut]

Sounds like the type of hell I put my developers through. I have been averaging roughly 2 bugs per day, while writing test automation, verifying fixes, generating reports, etc.

The customer probably would have only reported a dozen of those bugs if they weren't fixed, and they might only notice a couple dozen now that they are fixed and different from the previous version, but the developers fixed several hundred of the bugs that I found.

As for "Why in the world would I do THAT?", "because the code let me do it." Any good software tester should continually challenge the assumptions of the developers. The code needs to work correctly.

Except when written in Ada

[ebunga]

Seriously, grab gnat, go to http://www.adahome.com/ and read a tutorial or two, and start writing reliable, readable, and reusable code in the amount of time it takes to read through the tutorial.

Re:Except when written in Ada

[Ada95]

Ada 95 and the GNAT compiler are great but adahome.com is a Zombie/Cobweb site. AdaPower.com and AdaIC.org are much better Ada resources

Re:Debugging Software

[Anonymous Coward]

This may come as a shock to you but closed source developers do have access to source code, as unbelievable as that may sound. That means that they can actually step through the code with a debugger just like their open source friends.

Re:We are not brain-washed

[sheldon]

"Though slashdot is not quite a serious site, I doubt that they would post biased studies and false statistics (besides the polls)."

*COUGH*

I've been reading and posting to slashbot since 1997. You're obviously very new around here.

There's very little that is posted to slashbot relating to Open Source that shouldn't be picked up with a doggie doo scoop.

Re:suprising?

[Paleomacus]

Yeah saying something ALWAYS happens a certain way is

• always

a bad way to word things.

Mythical Man Month Myths

[Jerf]

Two of the three open projects you cite also happen to be bigger than most projects, with more developers working in || at the same time than most projects. Does that help them? Read "the mythical man-month" for the answer. (Hint: no).

You have seriously misread The Mythical Man Month, to the point of absurdity. The Mythical Man Month does emphatically not claim that more people can't do any more then fewer people.

What it says is that programmers are not interchangable, and as a result, adding programmers to an existing project will not get a 1-man-hour to 1-programmer-hour gain. While the old programmers are training the new programmers, and while the communication overhead increases, and while the new programmers are not yet settled in, productivity may fail to increase or may even decrease, over the next few months.

However, once the new guys are settled in and assuming a decent organization, the new organization can indeed move faster.

The Mythical Man Month's main point is to avoid treating programmers, or perhaps more accurately programmer-hours, interchangably. The conclusion based on that is the somewhat more famous part about not adding manpower to a late project because it will make it even more late.

Most strong open-source projects, and probably all the ones in this study, have a strong group of core people who are intimately familiar with what they need to know to continue improving the software. Other people may drift in but will find it difficult to contribute code back. (Seriously, just try to get a patch into Mozilla.) The Mythical Man Month does not apply to steady-state teams of people, only growing ones.

So yes, larger teams help those projects create their large software products. Go look at how much code is in Mozilla; no matter how wizard you are, I don't think a small team of people could even type that much code in anything less then a couple of months. How else do you think they could do it but with a large team?

Re:Mythical Man Month Myths

[ArmorFiend]

There are communication overheads for projects with more than 1 active programmer. You don't need branching, patching, etcetera when you're a team of one. You don't need to argue office politics about whether the UI team should be from the OS department or the application department. Good projects funnel developers off into their own little spheres of influence. When you encroach on another's sphere, you have to take time out to learn their API, or even their CPI, and that takes time.

Re:Mythical Man Month Myths

[Jerf]

You say that bit about "spheres of influence" like it's a bad thing, but it's a necessary part of getting a lot of people to work on one project. You can't get a hundred people to work without them; hell, you can't get four people to work without them. It reduces the communication from n^2 to something more like n log n, which is again why adding people can continue to benefit a project for a long time.

old practices

[Jerf]

I think those are probably fossil practices left over from pre-Internet days.

It is important to shield the developers from telephone calls and visits from the customer, because otherwise the developers will get nothing done, not necessarily because they have no time (time_in_day - time_dealing_with_customers may still be large), but because they are constantly being interrupted.

As long as the developer has discipline and the customer realizes that the answers won't be instant, though, there can be great value in having email access to the developers, for both parties.

Hopefully more companies will modernize their policies soon.

I have a c++ (unmanaged) project to get done.

[Sludge]

I develop with Cygwin and Emacs, but I compile and debug with Visual Studio 7.0. I believe that the Visual Studio debugger is unparalleled (lacking just Emacs integration! :) ), and there is nothing that can beat precompiled headers, a fast compiler (in the first place) and the visual debugging integration of Visual Studio.

I then boot to Linux and port my code. I've been writing portable code for half a decade, so I know what I'm doing, more or less. But, I can get more work done with Visual Studio, faster.

In case it makes a difference to your perception, I write end user apps, sometimes with heavy graphics requirements and GUI frontends.

Due to the nature of my work, I can't rely on masses to test everything before I ship.

In other news...

[saiya]

Water is always wet!

Oxford's College of Common Sense

[TitanBL]

"See, our model indicates that when you first start debugging your code there is more bugs than when you are almost finished. At which point the bugs the remaining bugs tend to be more difficult to find (less obvious), due to the fact that you have not found them yet. Furthermore, our models suggest that when working with a larger more diverse group of people, testing/debugging is accelerated.

Nah.... ;-)

Re:We are not brain-washed

[anthonyrcalgary]

What we have now with OSS is pretty much existance proof that it can work and a fuckload of anecdotes. We have exactly the same thing for closed source. The relative merits are going to take decades if not centuries to work out.

So?

[Cato the Elder]

Yes, open source users can take advantage of patches for newer versions sometimes. This doesn't obviate the need to support users of of older versions. It doesn't make debugging the problem faster.

I'm not saying there aren't advantages to open source. I'm just saying that the authors of the paper have done a poor job of proving faster debugging is one of them.

Re:OSS and being a commercial developer.

[archeopterix]

1) Competitors. If anyone can see your code, so can your competitor. It's always easy to improve on something when someone else already does the heavy lifting. Why should I spend $$$ on coding and have it OSS when my competitor will turn right around and see how I did something instantly? That right there destroys my investment. I'm in business to make money, not to help my competitor. Someone give me good reason why I should pay to help my enemy.

Valid point. My thoughts on this: Yes, your competitor can take your source and (more or less secretly) take your ideas or even pieces of code. Your hope is that the customers will choose you instead of him, because they also get the source and the right to modify it. Of course this doesn't always have to suffice.

2) Customer support. OK, my app is OSS. Some guy codes an derivative, and then releases the changes. Mr. Slashdot user (who has actually bought a licenced version from me, *gasp*) compiles it and has trouble somehow. He decides to ring ME up. This costs me money. Why should I have to waste resources now to screen who has trouble with my product, and who has trouble with a derivative not made by me?

I don't think it's a big problem - ask for hashes of the source tree. Of course the customer can lie to you and tell you the hashes of the original version, but eventually this will come out. I don't see why the customer shouldn't pay for that (support agreement should state this clearly - "you call me up on a bogus version, you pay").

OSS Debugging faster

[!Squalus]

The fact that OSS debugging is faster is no surprise at all. You want to see some slow debugging? Try working in a osftware company for a while, and you will understand why closed source is much slower.

The real reason are office politics, people who don't want anyone else to know what they do, and people who are busy skiving away their time.

It is an unwritten rule and unspoken truth that most closed source developers believe they have manegement by the short-hairs and that no one can see through waht they are saying. Not true, but people believe perception is reality, so it goes on.

I personally know of cases where developers purposely put bugs into software just for a little job security. I am not saying that is the norm at all - for most it certainly is not.

What is the norm is the obfuscation of time, effort, and code availability. That is almot truly ubiquitous. In other words:

1) We might be able to take a look at that some time in the future, but it could take significant time to debug that, and I am rather busy at the moment

2) That would take two to four weeks to correct. Divide by a factor of ten here if not greate to reach the truth (two to three hours or days even).

3) You can't see that code (to company employees trying to support a customer)

That specifically is why debugging takes longer in the proprietary world. There are artificial barriers to correcting mistakes. Those barriers are there for job security, and having little, if anything, to do with protecting intellectual property.

They usually cover up those fun little comments in code like: /* let's see how long it takes them to find this one */

or
# putting this line in because he keeps insisting
# that this wrong, when I know it is right
or
* This function makes a nice screen appear
* it doesn't do anything really
* but it seems to make Bob happy :)
or
* DOS isn't finished until Lotus 1-2-3 doesn't work

You get the idea. Ask a Product Manager about comments in code and they will shake nervously if you ask to see them, or just flat out deny you that access.

Open source code has funny comments as well (some very famous ones in fact), but at least it is open for all to see and make changes easily.

I am not saying that goes on where I work now, but I have seen it happen all too often in the closed source world. The ideas expressed therein are not necessarily those that the companies would want to get public attention.

The code isn't "clean enough" or "good enough" for public consumption.

Just something to think about.

Re:We are not brain-washed

[The Bungi]

Welcome to Slashdot!!!

Mythical Man Month Myths

[ArmorFiend]

Right. That was my point from the beginning. If communication overhead grows at (n log n) and the study consistantly picks bigger OSS projects than closed projects, then the OSS projects should theoretically have more problems quashing bugs. That's my point, not very deep at all!

you want

[Triumph The Insult C]

http://www.openbsd.org/goals.html

any of those goals should suit your needs

Bart Skinner?

[boots@work]

Oh, I remember him. Talk about clueless! On a bad day he could barely remember his own name. :-)