Slashdot Mirror
Automatic Functional Testing for Mac and Linux?
"For example, a vendor has a database interface, and wants to know how many times we can access a series of records, what the results show (are the right, blank, time out?), and how long it took for the search to take place during various times of the day, and so on. So we emulate one of their data entry people by logging onto the system, putting in some search parameter from a script, and then record what the text (or error message) was when it came back. Yes, we could connect directly to the SQL database without their front end, but it's the combination of their front end and the database we usually test. We have found many examples where were can connect raw, for instance, but the front end displays a timeout. We also test things like 'What happens if the user enters a colon or asterisk?' or 'What if the text entered in the box is over 256k?' Then we tell the vendor the results, and they hopefully fix them based on the data we gave them."
you can do this with AppleScript, quite possibly the most underrated language of its kind.
Cretin - a powerful and flexible CD reencoder
try OpenSTA. Its an open source automated testing application, similar to Rational's product.
www.opensta.org
There is a commercial company that provides support at www.opensta.com
Automatically uses the API to interact with the machine? Yeah, they've got this for Mac...it's called AppleScript and it's been around for roughly 20 years.
Don't be scared by the name "script"...there are visual tools for ascript and the language itself is so simple my mom could do it.
Hey freaks: now you're ju
Check this out for some pretty reasonable X-windows testing. Heard quite a few positive things about it.
Obviously this would work on Linux, OS X, and Windows as well.
Hope this helps.
Check out http://www.testingfaqs.org for a short list of possible contenders (GUI test drivers).
I've got it... you need Lego Mindstorms. Just build some robotics to push the mouse around, whack random keys, etc. Best of all, such a system would be fully platform-independant. I bet Microsoft is already doing this, only they call it "genetic programming".
Actually, the company I am currently employed by is in the process of writing a system/gui test environment for this very purpose. One caveat-it's only for Java, but when we complete it (very soon), we will release it to the public. our address is www.atomicobject.com.
The name of the project will be HASTE, look for it in a month or two.
Expect allows us to do on Linux
(and a vareity of other operating systems) ?
DO NOT PANIC
You might be able to use Rational Visual Test with a windows based X-server to control Linux (or any Unix) apps.
you could always port it to OS X ... :-)
It's a PC you can use to drive keyboard / mouse input into another system. It comes with a video capture card to read the output and make decisions about whether the test was successful or not.
It's fully scriptable using a c-ish syntax, and comes with a nice IDE and runtime environment to get the work done. It seems like it may be a nice match for what you want to do.
Here is more info.
You think emacs is evil?! You've never used VM's XEDIT have you?!! That's evil, baby!
Trained Monkeys.
Let a cat in the room and pretend you are doing some serious work.
Here's a website that talks about the language: http://expect.nist.gov/. I bet there's other sites so check google. O'Reiley also has a book about it. I still think I'd use Applescript under MacOS, but under Linux I'd give expect a try. I played with it a little under Solaris and it looked pretty useful. I was able to automate the setup for US Robotics modem racks and Cabletron ethernet hubs using expect. Never did get that stuff totally debugged and my script would go crazy, but that's not the language's fault.
Yeah, right. That paraphases exactly why we've used VHS for 20+ years instead of Beta. NOT! It is arguable that the Apple was a superior computing platform to the IBM PC of its day. The IBM-PC had the full corporate power and marketing strength of, well, IBM behind it while the Apple had the comparatively limited resources of two guys named Steve.
For the uninformed... Back in the days when the VCR was just being released on the world there were two competing formats, JVC's VHS format (which we all know) and Sony's Beta (a.k.a BetaMax). The latter was far superior but, alas, was first out marketed (a bit of history), then out litigated (Read case story). To date the majority of video production houses still use Beta because of it's superior image and audio quality.
My office has been taken over by iPod people.
You could probably roll your own rather quickly with the CGRemoteOperation API: /System/Library/Frameworks/ApplicationServices.fra mework/Frameworks/CoreGraphics.framework/Headers/C GRemoteOperation.h
The Future: Some assembly required; batteries not included.
For Mac OS X have a look at F-Script. You can inspect a running program and call any method in that program and any method on any object in the Objective-C runtime. This should allow you to generate events or enter text in text fields, etc.
Of course this will only work for programs developed using Cocoa. Can't help you with Carbon apps. You might be able to write something using the Apple Event manager.
Last time I had to do this for Linux (and Gtk+) I had to roll my own.
I'm assuming you're talking about java.awt.Robot. This, with BeanShell would allow test scripts for GUIs to be built.. A more sophisticated system would be able to capture AWT events, and store them in a file for later playback...
it's called AppleScript and it's been around for roughly 20 years.
Let's see here... 1993-2002 is roughly 20 years?
Has CowboyNeal been teaching you "new math"?
I just posted a set of tools for automating X at my website that I recently created to make a bot that could play Yahoo! Spelldown. There are 2 parts to the tools, finding things on screen using a "visual grep" program that finds images inside of images, and xte, an app built on the XTest extension that allows scripting of moving the mouse, clicking, dragging, keypresses, etc. Together you can use them to make scripts that can identify buttons on screen or letter or words or numbers, whatever, and interact with programs. Still a pre-1.0 version, but it does everything listed above so far.
BTW, don't use xse, which is a wrapper for XSendEvent, if you roll your own. There are so many places where it just doesn't work, or doesn't work right, because the XSendEvent function just doesn't work reliably. I lost a lot of sleep over that before I gave up and just wrote xte.
Free Online Woodworking Resources Directory
i think compuware's QARun or QADirector work on *nix
MARIJUANA, SHROOMS, X: ONLINE?! - E
Over the years I've asked Rational salespeople if they plan on porting some of their more popular debugging tools to Linux, such as Purify, Pure Coverage, Quantify, etc. The earliest responses to the question were, "What's Linux?" In later years, the response has simply been "No". I do not understand why, but it has been a major sticking point for me. More and more, people are developing software for Linux, and it's hard to understand why companies like Rational won't embrace it.
I'll tell you, it's a major hassle to develop a product for Linux while being forced to debug it on Solaris because that's the only place our debug tools will run. The first company to make decent Linux tools that are similar to Rational's will have my business. And Rational won't.
AppleScript is great, but the app that you are testing must be scriptable. Many are today, but some are not. Furthermore, in this situation you want to test the GUI, not just the underlying object model which is what AppleScript talks to.
However, AppleScript is not the only scripting environment for the Mac. Underlying all of the native scripting languages on the Mac is something called the Open Scripting Architecture. This allows any OSA component (i.e. scripting language) to talk to any other OSA component.
One OSA component is QuicKeys, a great product that actually does simulate mouse clicks, keystrokes, etc., and is completely scriptable. It integrates completely with AppleScript and via the power of the OSA, with shell scripts and Perl as well.
On top of this, Apple provides a powerful GUI building tool for AppleScript called AppleScript Studio, which is free. Plus, AppleScripts can talk across a network to do RPC. Imagine, you can have an AppleScript Studio front end that drives scripts on a set of machines over the network (so that you can assess performance under load). The individual scripts rely on AppleScript to drive the logic while using QuicKeys sequences to drive the GUI.
The only issue here is how easily you can assess results. There's no general way to check what the GUI is showing other than a pair of Mark I mod 0 eyeballs, since grabbing data out of the app being tested (even simulating a copy-paste) only talks to the underlying data structures and does not address GUI bugs. A possible way is to arrange the windows in a known manner, turn off the menu bar clock, take a screen shot, and use a graphical comparison tool to XOR the bitmap with a known good screen shot to highlight differences, but this won't work if you're working with variable or randomly generated data.
Disclaimer: I work for Apple, but these are my opinions only and do not represent any sort of official endorsement.
--Paul
DejaGnu is a TCL/Expect testing harness provided as free software by the Gnu Project. Not sure if it has tk hooks (which would allow you to test X-based programs) so you might want to check. If you're trying to test programs on the postscript display of a Mac instead of X, well, I don't know if it would work. Also, check to be sure there is a recent version of TCL/TK and Expect for your machine before you pursue this too far.
There is also LTP, the Linux Test Project, which may be of use to you as well. (Do a Google search for "linux test" and you'll find it quick.)
If you're not a TCL programmer, you're probably not going to like DejaGnu very much. If this is the case, good luck with the GUI tool search; you can expect to spend some serious money for decent test tools. (The pun is simply there. If you look at it too long or pay too much attention to it, it may or may not vanish. Just chalk it up to some crazy cat and don't worry about it.)
Best wishes,
Everything in the Universe sucks: It's the law!
Android runs on top of these and can be downloaded free for Linux and Unix
See the July 2001 issue of "Dr. Dobb's Journal"
for an overview of Android and how to download.
Plays back keyboard and mouse events for most UNIX implementations. (maybe even Mac OS X)
It is a little bit roll-your-own
Expect to spend some time writing perl to support your android scripts.
Letter To Iran
<g> IIRC, it's called Visual Test because it used to be Microsoft Visual Test and was bundled with Visual Studio. Then they sold it (back?) to Rational.
My company uses WinRunner for Windows GUI testing. I know that Mercury Interactive which makes WinRunner and LoadRunner another GUI Testing utility creates Unix version of their software in addition to the Windows versions. I don't know about Mac though.
Mercury Interactive
Hope that helps
Joe
I'm not completely sure about this, but I think you can use ATK (accessability toolkit) for sending events and getting widget contents to GTK apps. That works only onn GTK2, though, so perhaps it's not helpful to you, but at least I think the functionality is there.
AppleScript interfaces directly with the program at a lower level than the GUI -- it's essentially an alternate method of controlling the app, plugging into the same API as the GUI widgets do. This tests the app and everything below that, but it's not exactly the same as actually testing through the GUI itself, since you're bypassing it.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
No this is what apple means to :P
"Think Different"
--toq
I don't have moderator points, so I'll just post at 2 below this ;-)
You could roll your own based on VNC by creating your own VNC client that generates it's own scriptable mouse and keyboard events.
Then just run the standard server on the tested system.
-Peter