Slashdot Mirror
QA Under The Open Source Development Model
carrowood writes "A survey was conducted questioning open source developers from both large and small projects concerning their quality assurance practices. A
research paper based on the survey result was just published in the Journal of Systems and Software.
Some comparisions between QA practices of open vs closed source projects are made with some interesting observations. While on the whole it looks like open source QA can be as good as that in traditional software development, there were a few areas pointed out where the open source community does not do so well, such as regression testing and setting release dates.
A thought provoking read."
Do any open source projects get audited for ISO 9001 compliance? They have a quality certification that many [enterprise] customers desire.
I know at work ISO audits are both fun and exciting! (must contain laughter)
--------
Free your mind.
Considering the fact that open source software is often not commercial, it doesn't sound strange to me that release dates are not very strict. As there often is no budget, there is no maximum allowed time that may be spent on the feature set. Thus, software is released whenever the planned features have been built in, rather than cutting the desired feature set to fit the release date.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
Well, that's easy to understand. Commercial products are generally released to a schedule, whereas open source is released to a standard.
For example, anyone with half a clue in the IT profession is aware that you NEVER NEVER NEVER take on board a x.0 release of any Microsoft product - you ALWAYS ALWAYS ALWAYS wait for the first service pack as the release date was set by marketing types who wanted profile over quality. Even if the product is buggy people get their bonuses, and people talk about the product (no such thing as bad publicity).
Open source products are typically released by people who don't want to spend large amounts of time answering emails from users about problems, and madly fixing bugs that slipped through.
It's like back in the old days when if a vendor released a product, andthe product had problems, they fixed the problems for free rather than just make you wait for the next release and buy it. Yes kids, that did in fact happen.
I skimmed the article. I'm not sure what they were trying to prove. Release dates didn't jump out at me. As a software engineer, I don't see what release dates by you. Regression testing is obviously missing.
BUT HERE's the free idea. Maybe someone (aka doctoral candidate) could prove that because Free software is used by ALL possible users in whatever way they are going to use it within, say 4 months, that it doesn't matter if there's some obscure bug, in say, the Intellivision drivers for Linux, because no one uses those. I.e., software has an astronomical amount of states, any 1 of which could be broken, but after all (5 billion people) actually use the software, all HUMANLY possible states are VERIFIED.
Look at a company like MySQL AB vs. Microsoft. MySQL 4's release dates slipped and slipped, and when questioned, Monty responded that the software will be ready when it's ready. A company like Microsoft is driven by the marketing team. They set a launch date, and bugs or no bugs, the software will be launched with maximum fanfare on that date.
I think it's obvious which practice delivers the most stable software.
I like to make this point about ISO-9000: if they are not now ISO-9000 certified, McDonalds could be in about five minutes - they have a proceedure for everything. Does this mean that McD's good is "good quality"?
IF you have a proceedure, AND IF that proceedure included required analysis of failures, AND IF that proceedure requires improvements to the proceedures to correct the failures, THEN you MIGHT begin to approach quality over time.
However, far too many company's ISO proceedures fail to require analysis and actively discourage improvement to the proceedures ("You want to change the proceedure! OK, here's a ton of paperwork to fill out, and you will have to get the signatures of fourteen people who would rather not sign anything other than their own paychecks. Have Fun!").
ISO is really just a big peer-pressure MLM scam the way it is run now.
And yes, the company I work for is ISO 9001 qualified.
www.eFax.com are spammers
" That number looks like you pulled it from somewhere the Sun don't shine. "
... I just spend time writing it well in English."
... implementing one less than that gets much less so."
.. I'm shocked."
It's a guess. An approximation. I did not do any reasearch and arrive at that answer. But a simple look at any bugtrack should back me up. Most errors are due to memory leakage, buffer overflows and other artifacts of C programming.
" "literate" code is often the wrong approach, when I want to say things well in English I don't write the same thing in Japanese next to it
Literate programming means writing well in code. It means much more then comments and I urge you to some research into it.
"Interfaces in the Java sense I doubt buy you much from a lower bug count POV."
Interfaces force a strong contract. They make it harder to break the program.
"It's just not going to happen in the OpenSource world IMO, implementing someone's very good spec. isn't much fun
In that case then open source programs will continue to suffer from too many bugs.
" Duh, testing helps reduce bugs
Unit testing is different from regression testing and pre-release QA. By insisting on unit testing you will deliver cleaner code to the QA team.
War is necrophilia.
I don't know of ISOed OS software, but I am aware of organisations that have gone through a quality audit who use open source software. The main issue is having a test and internal release procedure, so you don't, for example, roll out Perl 5.8.1 without ensuring that your users are aware that their old version is being replaced. You *don't* need to get Larry Wall's personal inprimateur on the package, you just need to have a documented procedure.
In this way, security and QA are very much related. Zero security and no quality checks are fine, as long as those people using the system are aware of this and agree.
See my journal, I write things there