Ask Slashdot: Building a Software QA Framework?

New submitter DarkHorseman writes: I am looking into a new position with my employer and have the opportunity to work with the development and QA team to further the creation of a Quality Assurance Framework that will be used into the long-term future. This is software that has been in continuous development, in-house, for >10 years and is used company-wide (Fortune100, ~1000 locations, >10k users, different varieties based on discipline) as a repair toolset on a large variety of computers (high variability of SW/HW configuration). Now is the time to formalize the QA process. We have developed purpose-built tools and include vendor-specific applications based on business need. This framework will ideally provide a thorough and documentable means by which a team of testers could help to thoroughly ensure proper functionality before pushing the software to all locations. The information provided by Lynda.com along with other sources has been invaluable in understanding the software side of QA but I have seen very little in terms of actual creation of the framework of the process. What would you consider the best resources to prepare me to succeed? Even if your QA needs are for smaller projects, what advice do you have for formalizing the process?

Comments

You might as well ask...

[selectspec]

...how to build a software company.

Anyway, there is a lot to QA, but everything starts with the build and bug/feature tracking systems. QA is anchored to these two pillars. Formalizing (and documenting) how these systems work and interact is critical to correctly tracking issues, reproducibility and inter-QA-to-developer communication. A strong nightly build system coupled with a properly used tracking tool (like Bugzilla, Jira, etc) are pretty much the cornerstone of any framework.

The build system itself breaks down into the nightly automated build process as well as source revision control system. Testing should start before QA ever sees a build by testing the build after checkins and running developer implemented unit tests. Tagging release candidates and making sure that QA is working with the correct build is critical to optimizing QA efficiency. Nothing is worse than having QA run tests on the wrong build.

Next, QA testing itself gets divided into two major categories: automated and manual testing. Automation has a high initial cost of development, but pays off in the long run with lower labor, especially for mundane tasks that are too tedious to possible test by human.

Automation typically is closely linked into the build system. Manual testing breaks down into a bunch of different sub-categories, but all manual testing should follow documented test plans. Test plan documentation can get overboard, but at least you want a wiki of some kind (Confluence, etc) to document how development wants QA to perform the testing.

Manual testing is often where labor is the biggest factor, and thus it tends to get the most attention from management. These folks tend to be either "testers" good at testing vs. system integrators who are good at staging environments that look like real world customer environments. Much here of course depends on the nature of your product and your customer.

However, with both manual and automated testing, it is important to have infrastructure for creating clean environments to test with. This usually involves some form of virtualization. QA needs to be able to produce clean images of systems to test with. This part of QA has never been easier, unless you are a hardware company, and actually have to still use a lot of physical equipment instead of VMs.

Release quality is determined by comparing the list of open issues targeting a given milestone and their status. Again, a well integrated tool for issue tracking is critical in the process of release management.

Lastly, I would not wait 10 years to formalize a QA plan. Every software startup that I know of had a QA director starting to fill this plan out within a few months of getting seed money. QA is integral to software (and hardware) engineering.