QA Under The Open Source Development Model

carrowood writes "A survey was conducted questioning open source developers from both large and small projects concerning their quality assurance practices. A research paper based on the survey result was just published in the Journal of Systems and Software. Some comparisions between QA practices of open vs closed source projects are made with some interesting observations. While on the whole it looks like open source QA can be as good as that in traditional software development, there were a few areas pointed out where the open source community does not do so well, such as regression testing and setting release dates. A thought provoking read."

Comments

No Deadlines does not mean No Pressure

[Commykilla]

All the comments so far have been arguing that "no set release date means more QA". From my experience with open source software, this is complete crap. In many cases, commercially used open source software is developed by organizations such as Red Hat and IBM who, we hope, have significant QA teams and resources.

But, for the projects contributed by individuals, with a single or small group of developers -- who wants to do testing? Everybody wants to be a developer, and the more new features, the more bang, the more people will be drawn to the program, so more often than not new features and frequent releases rather than testing are the focus. Most of the time, you can guess from the sheer frequency of builds with new features that QA is not being taken into account. This system works great for projects that enthusiasts use -- individuals request new features, play with the product, and in return, report bugs. This does not work for companies of any size who expect software to work properly right out of the box, where the "feedback loop" of reporting bugs is a last resort.

Open source programmers and commercial programmers alike are under pressure to release early and release often. And in both the open source world and the commercial world, there are groups that excel at QA and groups that don't take QA seriously enough.

Re:ISO

[stwrtpj]

Do any open source projects get audited for ISO 9001 compliance? They have a quality certification that many [enterprise] customers desire.

Interesting thing is, having been through the ISO 9001 rigamarole, I can tell you that it does NOT necessarily enhance quality. ISO 9001 has very little to do with quality, regardless of what their claims are to the contrary. ISO is all about process and repeatibility. It simply validates that you have a well-defined process. You can have a wonderful process, with everything down to the number of times the developers go to the bathroom and for how long documented and validated, and still have a product that's a piece of crap.

An organization I used to work for did the ISO thing. The software had no fewer problems after ISO than it did before. It's just a nice thing to put on my resume, really.

Scott Adams of Dilbert fame put it best: If you never went through an ISO audit, you probably don't know what ISO is about. If you did go through an ISO audit, you definitely don't know what it's about.