QA != Testing

gManZboy writes "Original author of Make and IBM Researcher, Stu Feldman has written an overview of what should be (but is sadly perhaps not) familiar ground to many Slashdotters: Quality Assurance. He argues that QA is not equivalent to 'testing', and also addresses the oft-experienced (apparent) conflict between QA-advocates and 'buisiness goals.'"

Comments

Re:Capability Maturity Model

[mccalli]

the SEI was approached by the military a couple decades ago. The military had a problem; when it contracted out software development work, it would sometimes get back what it was looking for, it would sometimes get it on time...The SEI went about polling a large number of contractors, trying to see what was common amongst the ones who delivered. They found there was actually a very strong correllation between a number of processes and practices and high-quality under-budget software. The result is the Capability Maturity Model or CMM for short, which divides companies up into 5 "levels".

Yeah, I use it and am in a certified team. It's vastly overrated, and no substitue at all for people who know what they're doing. It might complement people who know what they're doing, but then such people would have come up with their own valid processes anyway, hence your initial correlation.

And it's hardly helped the US military come in on time and under-budget, now has it?

...but my dad has a good web site that deals with quality issues (IE only, unfortunately).

!

Cheers,
Ian

Re:Requirements?

[Eric Giguere]

For good reading on the design/requirements problem, I recommend Alan Cooper's The Inmates Are Running the Asylum. Talks a lot about how products can meet all their requirements and yet still fail because the requirements weren't right to begin with.

My experience with QA

[Anonymous Coward]

Normally I hate AC, but I have to in this case.

One of my main Day Jobs up to a few years ago was working in QA for Major Computer and Software Manufacturers.

The idea of:
Testing = finding bugs
QA = determining which features are required, and whether or not they work as intended for the end-user.

Is fine in theory, but rarely happens in practice. It usually ends up that QA is ignored and conflated into bug testing. And even then, it often doesn't matter.

Example: I was working on a team that developed an Important Piece of Software That Is Very Popular These Days. We Had No Specification.

None. After some terrifying meetings with the CEO, we somehow brought it to a 1.0 release. I didn't want to have to go through that little nightmare again, so at the 1.0 post mortem meeting, I asked "So, we built 1.0 without a spec - what exactly are we going to do next? What is the Specification for 2.0?"

The lead programmer looked right at me and said "The Spec for 2.0 is 1.0."

We had shipped 1.0 with over 850 bugs, with half a dozen known (if somewhat obscure) crashing bugs, and with several features "turned off" because we couldn't get them to work.

At that point I knew I had to get the fuck out of there. I wasn't going to spend over 2 hours a day driving just to help this rolling train wreck. I left as we shipped 2.0.

From there I went to a company That Was Also Extremely Famous (but now defunct) where QA was more of an expensive after thought. they hired a great team, but the engineers were so disjointed that the product kept changing every other month.

The stress level was so high at that company, of the 120 employees, half a dozen attempted suicide in the 9 months I worked there. At one point, there was such a row in basic engineering philosophy, two of the main programmers got into a fist fight. When the money dried up, we all got laid off.

We can go on and on about how important QA is, but the fact is, we're in a business that makes products, and when the business is more than a dozen people jammed in a garage or airless office space, the products tend to be driven by marketing droids. Left to their own devices, Engineers will produce complex objects that don't necessarily work or fulfil a worthwhile function, or do it in a way that is elegant and useful. Left to QA, the product never gets out the door, because Software Engineering *ISN'T*. SE is more like knitting or quilt making than an Engineered Science. Bridge Builders use Engineers - they have books full of equations that have been proven over the years and they use these solid tested things in a creative way to solve a problem : how to get from here to there.

When a bridge fails, or a building collapses, they just look at the evidence and compare it to the known working materials sciences, engineering principles, etc. and figure out how it failed.

With Software everything is written in a series of codes - at the machine level, we know what they do. But once you get into the meat and potatoes of software development, it all gets very wiggly very quickly. That's why TESTING is needed. And QA should be brought in even before the coding begins - when the UI is being developed, when the notion of the application's purpose and methods are being developed.

But, as noted above: if QA runs the show: it never ships, as there are always improvements to be made. Always.

So, you have the maketing droids who have the ear of the business managers who then set arbitrary and insane deadlines. The result? QA can't touch everything, or they conspire with Engineering that some sections are "good enough" and they let it go, so they can focus resources on testing problem areas, in order to meet the absurd deadline.

The end result is always the same: The public gets buggy software.

The only question is : How Buggy Is It?

They flip the crank, they do the late nights, they get the product out. QA and Eng do their littl