Software Carpentry QMTest Testing Tool Released

soundsop writes: "The first tool resulting from the winners of a design competition by the Software Carpentry project has been released. The QMTest tool is a testing tool to replace software such as XUnit, Expect and DejaGnu. An issue tracking tool, called QMTrack (a la Bugzilla) is forthcoming. It looks like the winning design proposals for a config tool (autoconf replacement) and a build tool (make replacement) are not being implemented."

Comments

Re:What exactly does this do?

[Harumuka]

SC Test aims to produce regression testing software. This basically means that when software has new features added, regression testing tests to make sure it hasn't taken a step backwards (regressed). In plain english, regression testing tries to prevent new features from introducing bugs. A good introduction to regression testing is at AutomatedQA, although the software there is commerical their few words on regression testing are well worth reading.

This is what Webopedia has to say on the subject:

The selective retesting of a software system that has been modified to ensure that any bugs have been fixed and that no other previously-working functions have failed as a result of the reparations and that newly added features have not created problems with previous versions of the software. Also referred to as verification testing, regression testing is initiated after a programmer has attempted to fix a recognized problem or has added source code to a program that may have inadvertently introduced errors. It is a quality control measure to ensure that the newly-modified code still complies with its specified requirements and that unmodified code has not been affected by the maintenance activity.

Replacing Expect? It's not a testing tool.

[jdavidb]

Kudos to the QMTest folks; I'm looking forward
to the fruits of the software carpentry project. But, as Expect is an
automation tool, not simply a testing tool, I don't think it'll exactly be "replaced."
The well-known DejaGNU suite, written in Expect, might be a candidate for replacement, though.

I use Expect all the time, but have never used it for testing. I tried DejaGNU a time or two,
but never could figure it out. If you like Expect and like Perl, you might check out the
Expect.pm module project; it's really come to fruition recently, and
I've finally started doing that kind of thing in Perl instead of TCL/Expect.

an HTML user interface

[markj02]

I have written a lot of test cases in my time. I can't quite figure out why I would want to use QMTest.

First, I like writing test cases in a text editor, programmatically. It's tedious enough writing them in the first place, at least I can cut-and-paste and modify them quickly in an editor. Going through a web GUI does not seem like it's very efficient. Also, I don't particularly like using anything other than the implementation language and shell scripts for test cases; otherwise, people receiving the source code need to install additional tools. I also don't see anything in the white paper about support for the hard parts of testing, like configuration and compilation management for lots of extra C/C++ code, GUI testing, or web site testing (the latter usually require recording and playback).

Altogether, I'm not sure I ever felt I needed something like what QMTest seems to be doing. And the things that are actually difficult to test, it doesn't seem to provide useful support for. Can someone explain what I'm missing?