Slashdot Mirror
Automatic Functional Testing for Mac and Linux?
"For example, a vendor has a database interface, and wants to know how many times we can access a series of records, what the results show (are the right, blank, time out?), and how long it took for the search to take place during various times of the day, and so on. So we emulate one of their data entry people by logging onto the system, putting in some search parameter from a script, and then record what the text (or error message) was when it came back. Yes, we could connect directly to the SQL database without their front end, but it's the combination of their front end and the database we usually test. We have found many examples where were can connect raw, for instance, but the front end displays a timeout. We also test things like 'What happens if the user enters a colon or asterisk?' or 'What if the text entered in the box is over 256k?' Then we tell the vendor the results, and they hopefully fix them based on the data we gave them."
you can do this with AppleScript, quite possibly the most underrated language of its kind.
Cretin - a powerful and flexible CD reencoder
try OpenSTA. Its an open source automated testing application, similar to Rational's product.
www.opensta.org
There is a commercial company that provides support at www.opensta.com
Automatically uses the API to interact with the machine? Yeah, they've got this for Mac...it's called AppleScript and it's been around for roughly 20 years.
Don't be scared by the name "script"...there are visual tools for ascript and the language itself is so simple my mom could do it.
Hey freaks: now you're ju
Check this out for some pretty reasonable X-windows testing. Heard quite a few positive things about it.
Obviously this would work on Linux, OS X, and Windows as well.
Hope this helps.
Check out http://www.testingfaqs.org for a short list of possible contenders (GUI test drivers).
I've got it... you need Lego Mindstorms. Just build some robotics to push the mouse around, whack random keys, etc. Best of all, such a system would be fully platform-independant. I bet Microsoft is already doing this, only they call it "genetic programming".
Actually, the company I am currently employed by is in the process of writing a system/gui test environment for this very purpose. One caveat-it's only for Java, but when we complete it (very soon), we will release it to the public. our address is www.atomicobject.com.
The name of the project will be HASTE, look for it in a month or two.
Expect allows us to do on Linux
(and a vareity of other operating systems) ?
DO NOT PANIC
you could always port it to OS X ... :-)
It's a PC you can use to drive keyboard / mouse input into another system. It comes with a video capture card to read the output and make decisions about whether the test was successful or not.
It's fully scriptable using a c-ish syntax, and comes with a nice IDE and runtime environment to get the work done. It seems like it may be a nice match for what you want to do.
Here is more info.
You think emacs is evil?! You've never used VM's XEDIT have you?!! That's evil, baby!
Trained Monkeys.
For Mac OS X have a look at F-Script. You can inspect a running program and call any method in that program and any method on any object in the Objective-C runtime. This should allow you to generate events or enter text in text fields, etc.
Of course this will only work for programs developed using Cocoa. Can't help you with Carbon apps. You might be able to write something using the Apple Event manager.
Last time I had to do this for Linux (and Gtk+) I had to roll my own.
I just posted a set of tools for automating X at my website that I recently created to make a bot that could play Yahoo! Spelldown. There are 2 parts to the tools, finding things on screen using a "visual grep" program that finds images inside of images, and xte, an app built on the XTest extension that allows scripting of moving the mouse, clicking, dragging, keypresses, etc. Together you can use them to make scripts that can identify buttons on screen or letter or words or numbers, whatever, and interact with programs. Still a pre-1.0 version, but it does everything listed above so far.
BTW, don't use xse, which is a wrapper for XSendEvent, if you roll your own. There are so many places where it just doesn't work, or doesn't work right, because the XSendEvent function just doesn't work reliably. I lost a lot of sleep over that before I gave up and just wrote xte.
Free Online Woodworking Resources Directory
AppleScript is great, but the app that you are testing must be scriptable. Many are today, but some are not. Furthermore, in this situation you want to test the GUI, not just the underlying object model which is what AppleScript talks to.
However, AppleScript is not the only scripting environment for the Mac. Underlying all of the native scripting languages on the Mac is something called the Open Scripting Architecture. This allows any OSA component (i.e. scripting language) to talk to any other OSA component.
One OSA component is QuicKeys, a great product that actually does simulate mouse clicks, keystrokes, etc., and is completely scriptable. It integrates completely with AppleScript and via the power of the OSA, with shell scripts and Perl as well.
On top of this, Apple provides a powerful GUI building tool for AppleScript called AppleScript Studio, which is free. Plus, AppleScripts can talk across a network to do RPC. Imagine, you can have an AppleScript Studio front end that drives scripts on a set of machines over the network (so that you can assess performance under load). The individual scripts rely on AppleScript to drive the logic while using QuicKeys sequences to drive the GUI.
The only issue here is how easily you can assess results. There's no general way to check what the GUI is showing other than a pair of Mark I mod 0 eyeballs, since grabbing data out of the app being tested (even simulating a copy-paste) only talks to the underlying data structures and does not address GUI bugs. A possible way is to arrange the windows in a known manner, turn off the menu bar clock, take a screen shot, and use a graphical comparison tool to XOR the bitmap with a known good screen shot to highlight differences, but this won't work if you're working with variable or randomly generated data.
Disclaimer: I work for Apple, but these are my opinions only and do not represent any sort of official endorsement.
--Paul
Holy piss, madd scientist, I can't believe you recommended that ;)
QADirector is not a testing language in itself. It is a harness from which to run the script generated by script writers. It used to be a Unix app, but when the switch was made to Wintel there were a few missteps. If you are looking for a test harness that is comprehensive (for Windows anyways) in its command-line options, look into QAD, otherwise I would go with Test Director by Mercury or maybe Segue's new product (I believe its called SilkPlan or something similar).
QARun on the other hand is a pain in the ass. If you are looking for automation, don't use it. It's designed for straight-up "record and playback" style testing. If you wish to deploy these tests to client machines (something which some, but not all people want to do), do not use QARun. You can look into XRunner (it is the predecessor of WinRunner) by Mercury if it is still being sold/supported for *nix testing. I honestly don't know about Mac, our product doesn't run on it.
If you want a serious evaluation document, I highly suggest heading over to http://www.qaforums.com and head to the Automation section. Post a query about the Mac and you'll probably receive a response from one of the moderators with a white paper that was written comparing all of the available tools.
DISCLAIMER: My group uses QARun, QAD, Segue Silktest, and WinRunner, so as not to be accused of bias.
"What we elect to call imagination is mere combination of things not heretofore combined." - Frank Norris
Let's see here... 1993-2002 is roughly 20 years?
It is if you're using base 5 - due to a tragic accident involving penut-butter, weasels, and a large bowling-ball; and thus only have one hand.
Moneyed corporations, non-working 'poor' and criminal prisoners are turning productive citizens into tax-slaves.