Ask Slashdot: Building a Software QA Framework?

New submitter DarkHorseman writes: I am looking into a new position with my employer and have the opportunity to work with the development and QA team to further the creation of a Quality Assurance Framework that will be used into the long-term future. This is software that has been in continuous development, in-house, for >10 years and is used company-wide (Fortune100, ~1000 locations, >10k users, different varieties based on discipline) as a repair toolset on a large variety of computers (high variability of SW/HW configuration). Now is the time to formalize the QA process. We have developed purpose-built tools and include vendor-specific applications based on business need. This framework will ideally provide a thorough and documentable means by which a team of testers could help to thoroughly ensure proper functionality before pushing the software to all locations. The information provided by Lynda.com along with other sources has been invaluable in understanding the software side of QA but I have seen very little in terms of actual creation of the framework of the process. What would you consider the best resources to prepare me to succeed? Even if your QA needs are for smaller projects, what advice do you have for formalizing the process?

Comments

Use your toolchain

[plover]

Presumably you are using modern tools to compile and build your software, manage source code, and manage your project work. Many of these tools will either incorporate or integrate with bug tracking software and testing frameworks. If there's a native bug tracker available, select it. If there's a native test framework available, use it.

What you need is a least-friction option, where testers, analysts, and developers can all see the bugs, write up the bugs, test the bugs, and fix the bugs. You don't need "The Most Advanced Framework Available Today", you don't need "The Best Test Tracking and Reporting Software Ever Produced", you need a solution that works well for all the people involved. Having a third party tool where the developer has to stop working, log in to the bug tracker, read the bug details, switch back to the development environment, make some changes, switch back to the bug tracker, write up the findings, switch to the test framework, execute a test, switch back ... All that switching is a huge productivity killer. The smoother the integration, the more effective and efficient the engineers will be - and that's where your expenses really lie.

Here's the problem. Some organizations say "hey, let's evaluate and buy the bestest test software out there" without giving a thought to the developers. So the QA department runs off on their own, buys a tool, and starts building tests in it that the developers can't run. If the developers can't run the tests, they don't know if they're fixing the problems correctly, so they waste tons of time. Worse, if a developer makes a change that breaks some test, they won't know until that result is reported to them, possibly days, weeks or even months later, depending on your QA cycles. During the intervening time, the developer continues to write code based on their original faulty change, creating technical dependencies on what may be a completely flawed base assumption. When the test finally reveals the flaw, the developer's choices are limited to: A) rewrite everything according to the better architecture uncovered by the flaw, or B) make a scabby patch so the test passes. If you choose A, the software's release will be expensively delayed. If you choose B once, you'll likely choose it again, you're incurring technical debt, all your software is likely to be crap, and no good developers will want to work for you. The correct answer is of course C) don't produce tests the developer's can't run themselves on demand, or tests that aren't automated as a part of the build process.

QA is a carreer killer. Don't touch the tarbaby.

[Ungrounded Lightning]

I am looking into a new position with my employer and have the opportunity to work with the development and QA team to further the creation of a Quality Assurance Framework that will be used into the long-term future.

You haven't taken the position yet? Don't!

Unlike other industries (such as the auto industry, where QA is often the highest ranking and most powerful professional short of plant manager, respected for their global understanding of both engineering and the math of probability and statistics, empowered to order things fixed and, if appropriate, stop the production until they are), the pointy-haired bosses of software development generally view QA people (even the write-the-tools types) as failed developers. "If you were good you'd be a developer." is a typical quote. A QA position on your resume is an albatros around your neck.

One of the better programming talents I know (and I know a lot of the big names), when starting out, took a QA position as a stop-loss. After that she couldn't get a developer position - even those where they CLAIMED to give her a developer position actually put her in the QA department.

QA is who they cut first, when things get tight. This is massively stupid - because it's who you need to get OUT of the jam. But understaffing QA shows up on the bottom line a couple years out, after the PHB has lined up his next positio, while understaffing development shows up quickly.