A Framework For Quality Assurance?

Midh Mulpuri asks: "These days licenses like GPL, MPL and QPL make it easy for developers to release software as open source without the hassle of having to write up "TERMS and CONDITIONS" for each project. While the software is released as open source, it is mostly released with no warranty whatsoever. That is reasonable considering the software is often 'free'. Major projects often find service providers offering fee-based support. But for most part, the burden of providing service falls on the community and the developers. Further, there is no quality assurance other than the reputation of the project (and the developers). While for many this is enough assurance, the lack of warranty and quality assurance might still leave some people uncomfortable. To make 'free' software more appealing we would need a better way to assure quality. The greatest quality assurance for software is the developer. A framework should be developed to make it easier for the developer to assure users of quality." Does Open Source really need a framework for Quality Assurance?

"This framework might include:

1. A "General Practice Agreement" or "Ethics Agreement" that states ground rules for open source projects and programmers in general. (I am using the word Agreement lacking a better word). This would be like the contracts that bind lawyers, doctors, architects and other professionals. Developers could sign the "Agreement" by placing a graphic on their site.
2. Self-test guidelines. The guidelines would include a checklist that needs to be completed before any production release.
3. Documenting results of the self-test on the project site and/or distributing the test suite with the software.

Most projects already have some form of testing. Documenting and publishing these tests would add to the credibility of the project. An ethics standard might be a more difficult issue. How would you come up with an ethics standard that is acceptable to all? A statement like "I shall document my code well" or " I shall make my software extensible and customizable" would be easy to adhere to. "I shall base my work on industry standards (when possible)" might not appeal to all programmers.

Such a framework would need the backing of prominent open source programmers to be developed and implemented. I believe that the assurance of quality would bring more respect to open source and open source programmers. If such a framework already exists (and escaped my well traveled mouse) please let me know. Otherwise, I request the Slashdot community to kick start the development of such a framework. What should a "General Practice Agreement" contain? What would be good self-test guidelines?"

Comments

Re:Why OSS doesn't need QA...

[Junks Jerzey]

In commercial software houses, changes are made all the time by people who don't understand the code. Often times, the person who wrote the code left several years ago, and noone is really even sure it works completely with the latest library versions. However, with free software, projects tend to be much smaller, and so figuring out what is going on is fairly trivial

Sorry, that's wishful thinking. In a commercial development house, there are people who work with the codebase day in and day out. Even if the original author has left (very common), there's always someone who can scribble on a white board for 10 minutes to set you straight. Often what looks like an obvious fix turns out to be incorrect, making you glad you had that 10 minutes of scribbling. Maybe the biggest problem with fixes to OSS is that you don't know the reputation of the person making the fix. Maybe it's from a careless programmer. Maybe it's from someone who can program but doesn't understand the importance of thorough testing. Who knows?

Re:Sorry, but "Quality Assurance" is a crock of sh

[Reality Master 101]

And these two numbers, hours tested and how many doing the testing are orders of magnitude greater for Linux than for any Microsoft OS. So while Microsoft spends orders of magnitude more dollars on "Quality Assurance" programs, Linux comes out more stable. You tell me. Which is the better indicator of product reliability?

You don't know what you're talking about. For some bizarre reason, Linux advocates simultaneously claim that Microsoft is the worst run software company in the world, and then go on to use Microsoft to prove the "superior" OSS development model.

There are other software companies, you know. Ones that make Linux quality look like the crap that it is. Yes, I said crap. Only in comparison to, say, Win/95 is Linux stable and reliable. It is riddled with security holes and has many bugs. You personally just don't see them because you don't stress your box in any sort of way.

Linux is not even close to the best version of Unix. Take a look at AIX sometime, or even Solaris. Got news for you: commercial Unix blows Linux out of the water. Or heck, look at the AS/400 if you want to see reliability.

The bottom line is that the OSS development model is far from proven, and in fact, has very few examples of producing top quality software. Apache is one. And I can't think of another. Almost all the best software is closed source and commercial. And one of the reasons is having a QA and testing staff.

--

Huh? Don't think so

[Vanders]

The best person to assure quality is the developer? No chance! I assume you've never worked in a commercial development enviroment?

As a QA tester, i can assure you that a developer is not the best person to assure quality. They just don't see their own bugs, or code that isn't to spec. (This may not always apply to OSS projects). The only person who can assure quality is a third party (Non developer), who does a full QA test on it. Trust me on this one.