Slashdot Mirror
QA Under The Open Source Development Model
carrowood writes "A survey was conducted questioning open source developers from both large and small projects concerning their quality assurance practices. A
research paper based on the survey result was just published in the Journal of Systems and Software.
Some comparisions between QA practices of open vs closed source projects are made with some interesting observations. While on the whole it looks like open source QA can be as good as that in traditional software development, there were a few areas pointed out where the open source community does not do so well, such as regression testing and setting release dates.
A thought provoking read."
Considering the fact that open source software is often not commercial, it doesn't sound strange to me that release dates are not very strict. As there often is no budget, there is no maximum allowed time that may be spent on the feature set. Thus, software is released whenever the planned features have been built in, rather than cutting the desired feature set to fit the release date.
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
Ultimately, while development can, in certain cases, be done in a vacuum, QA cannot (and should not). It's by nature a collaborative and interactive process.
I have nothing but respect for the few (good) QA engineers I've worked with.
I suspect clients requiring ISO 9001 just go for commercial software. One part of OSS often is to refuse warranty and alike - IMO this does not quite fit into ISO 9001.
1. Introduction
As everyone knows, Open Source software is the wave of the future. With the market share of GNU/Linux and *BSD increasing every day, interest in Open Source Software is at an all time high.
Developing software within the Open Source model benefits everyone. People can take your code, improve it and then release it back to the community. This cycle continues and leads to the creation of far more stable software than the 'Closed Source' shops can ever hope to create.
So you're itching to create that Doom 3 killer but don't know where to start? Read on!
2. First Steps
The most important thing that any Open Source project needs is a Sourceforge page. There are tens of thousands of successful Open Source projects on Sourceforge; the support you receive here will be invaluable.
OK, so you've registered your Sourceforge project and set the status to '0: Pre-Thinking About It', what's next?
3. Don't Waste Time!
Now you need to set up your SourceForge homepage. Keep it plain and simple - don't use too many HTML tags, just knock something up in VI. Website editors like FrontPage and DreamWeaver just create bloated eye-candy - you need to get your message to the masses!
4. Ask For Help
Since you probably can't program at all you'll need to try and find some people who think they can. If your project is a game you'll probably need an artist too. Ask for help on your new Sourceforge pages. Here is an example to get you started:
Thousands of talented programmers and artists hang out at Sourceforge ready to devote their time to projects so you should get a team together in no time!5. The A-Team
So now you have your team together you are ready to change your projects status to '1: Pre-Bickering'. You will need to discuss your ideas with your team mates and see what value they can add to the project. You could use an Instant Messaging program like MSN for this, but since you run Linux you'll have to stick to e-mail.
Don't forget that YOU are in charge! If your team doesn't like the idea of giant robotic spiders just delete them from the project and move on. Someone else can fill their place and this is the beauty of Open Source development. The code might end up a bit messy and the graphics inconsistant - but it's still 'Free as in Speech'!
6. Getting Down To It
Now that you've found a team of right thinking people you're ready to start development. Be prepared for some delays though. Programming is a craft and can take years to learn. Your programmer may be a bit rusty but will probably be writing "hello world" programs after school in no time.
Closed Source games like Doom 3 use the graphics card to do all the hard stuff anyhow, so your programmer will just have to get the NVidia 'API' and it will be plain sailing! Giant robot spiders, here we come!
7. The Outcome
So it's been a few years, you still have no files released or in CVS. Your programmer can't get enough time on the PC because his mother won't let him use it after 8pm. Your artist has run off with a Thai She-Male. Your project is still at '1: Pre-Bickering'...
Congratulations! You now have a successful Open Source project on Sourceforge! Pat yourself on the back, think up another idea and do it all again! See how simple it is?
All the comments so far have been arguing that "no set release date means more QA". From my experience with open source software, this is complete crap. In many cases, commercially used open source software is developed by organizations such as Red Hat and IBM who, we hope, have significant QA teams and resources.
But, for the projects contributed by individuals, with a single or small group of developers -- who wants to do testing? Everybody wants to be a developer, and the more new features, the more bang, the more people will be drawn to the program, so more often than not new features and frequent releases rather than testing are the focus. Most of the time, you can guess from the sheer frequency of builds with new features that QA is not being taken into account. This system works great for projects that enthusiasts use -- individuals request new features, play with the product, and in return, report bugs. This does not work for companies of any size who expect software to work properly right out of the box, where the "feedback loop" of reporting bugs is a last resort.
Open source programmers and commercial programmers alike are under pressure to release early and release often. And in both the open source world and the commercial world, there are groups that excel at QA and groups that don't take QA seriously enough.
Communism was just a red herring.
Interesting thing is, having been through the ISO 9001 rigamarole, I can tell you that it does NOT necessarily enhance quality. ISO 9001 has very little to do with quality, regardless of what their claims are to the contrary. ISO is all about process and repeatibility. It simply validates that you have a well-defined process. You can have a wonderful process, with everything down to the number of times the developers go to the bathroom and for how long documented and validated, and still have a product that's a piece of crap.
An organization I used to work for did the ISO thing. The software had no fewer problems after ISO than it did before. It's just a nice thing to put on my resume, really.
Scott Adams of Dilbert fame put it best: If you never went through an ISO audit, you probably don't know what ISO is about. If you did go through an ISO audit, you definitely don't know what it's about.
Karma: Frotzed (mostly due to the Frobozz Magic Karma Company)
Other areas of problems are attributable to slovenly or "don't give a damn" attitudes--unused variables, unreachable code, "magic number" constants, and so on. Ignoring values returned by a function are very common. Maybe it is acceptable with a library function, but why return a value if you aren't going to use it? It's better to make the function into a procedure by returning void. On a more theoretical level, the use of weak typing even when the language allows for tighter specification of variables. Strong typing is designed to prevent such oddities and inadvertently multiplying a color by date.
However, in the end it all comes back to complexity. And that is where the biggest improvement in OSS quality can be obtained.