Slashdot Mirror

← Back to Stories (view on slashdot.org)

Automatic Functional Testing for Mac and Linux?

Posted by Cliff on from the true-software-testing-suites dept.

testYourGUIS asks: "I work with a company that tests consumer and business software for various venues. One of the programs we rely on heavily is called Rational Visual Test, which is a VB-like compiled programming language that works directly with Windows API to simulate mouse clicks, text entry, menu pulldowns, and so on. We use them to stress-test various applications, collect the results, and then send the vendors a spreadsheet showing what we found. Recently, pressure has started to produce the same results on Mac OS X and possibly Linux applications. I was wondering if anyone knew of such software that works on Mac OS X, and how to go about doing this for Linux?"

"For example, a vendor has a database interface, and wants to know how many times we can access a series of records, what the results show (are the right, blank, time out?), and how long it took for the search to take place during various times of the day, and so on. So we emulate one of their data entry people by logging onto the system, putting in some search parameter from a script, and then record what the text (or error message) was when it came back. Yes, we could connect directly to the SQL database without their front end, but it's the combination of their front end and the database we usually test. We have found many examples where were can connect raw, for instance, but the front end displays a timeout. We also test things like 'What happens if the user enters a colon or asterisk?' or 'What if the text entered in the box is over 256k?' Then we tell the vendor the results, and they hopefully fix them based on the data we gave them."

7 of 200 comments (clear)

  1. testing tools by Anonymous Coward · · Score: 5, Informative

    try OpenSTA. Its an open source automated testing application, similar to Rational's product.

    www.opensta.org

    There is a commercial company that provides support at www.opensta.com

  2. Hmm. by dasmegabyte · · Score: 5, Informative

    Automatically uses the API to interact with the machine? Yeah, they've got this for Mac...it's called AppleScript and it's been around for roughly 20 years.

    Don't be scared by the name "script"...there are visual tools for ascript and the language itself is so simple my mom could do it.

    --
    Hey freaks: now you're ju
  3. XRunner by micantos · · Score: 5, Informative

    Check this out for some pretty reasonable X-windows testing. Heard quite a few positive things about it.

  4. Re:In case it gets slashdotted by yorgo · · Score: 5, Informative

    Check out http://www.testingfaqs.org for a short list of possible contenders (GUI test drivers).

  5. Legos! by djtack · · Score: 5, Funny

    I've got it... you need Lego Mindstorms. Just build some robotics to push the mouse around, whack random keys, etc. Best of all, such a system would be fully platform-independant. I bet Microsoft is already doing this, only they call it "genetic programming".

  6. I've got two words for you... by pi_rules · · Score: 5, Funny

    Trained Monkeys.

  7. Applescript, QuicKeys for Mac OS X and OSA by plsuh · · Score: 5, Informative

    AppleScript is great, but the app that you are testing must be scriptable. Many are today, but some are not. Furthermore, in this situation you want to test the GUI, not just the underlying object model which is what AppleScript talks to.

    However, AppleScript is not the only scripting environment for the Mac. Underlying all of the native scripting languages on the Mac is something called the Open Scripting Architecture. This allows any OSA component (i.e. scripting language) to talk to any other OSA component.

    One OSA component is QuicKeys, a great product that actually does simulate mouse clicks, keystrokes, etc., and is completely scriptable. It integrates completely with AppleScript and via the power of the OSA, with shell scripts and Perl as well.

    On top of this, Apple provides a powerful GUI building tool for AppleScript called AppleScript Studio, which is free. Plus, AppleScripts can talk across a network to do RPC. Imagine, you can have an AppleScript Studio front end that drives scripts on a set of machines over the network (so that you can assess performance under load). The individual scripts rely on AppleScript to drive the logic while using QuicKeys sequences to drive the GUI.

    The only issue here is how easily you can assess results. There's no general way to check what the GUI is showing other than a pair of Mark I mod 0 eyeballs, since grabbing data out of the app being tested (even simulating a copy-paste) only talks to the underlying data structures and does not address GUI bugs. A possible way is to arrange the windows in a known manner, turn off the menu bar clock, take a screen shot, and use a graphical comparison tool to XOR the bitmap with a known good screen shot to highlight differences, but this won't work if you're working with variable or randomly generated data.

    Disclaimer: I work for Apple, but these are my opinions only and do not represent any sort of official endorsement.

    --Paul