Testing Products for Web Applications?

hellbunnie asks: "I work with a team of developers many of whom spend much of their time writing web-based front-ends for DBs or other applications. Now, while we enjoy programming, we're pretty lazy when it comes to testing. Even if we weren't so lazy, I think we'd still miss a number of problems, 'cos there's just so many different screens that use any particular method/function that you might change. That means there's a lot to be tested after each change. So, my question is has anyone any experience with automated systems for testing web applications?"

"I've seen a lot of automated test suites advertised and I've always assumed that they were no substitute for careful testing by a human. However, as the number of web pages that we need to maintain grows, I've begun to wish that we had something that we could kick off at night, that would follow all links on our system and fill in values for the various forms it encountered, then when we arrived in the next morning there'd be some sort of report available detailing its findings. It could flag any pages that returned something obviously incorrect, such as a SQL error, a blank page or just the word 'error'.

Does such a thing exist or am I just engaging in wishful thinking to imagine that there might be something flexible enough to do the job? What do other people do to test their software?"

Comments

mercury

[psychopenguin]

Mercury Interactive - www.mercuryinteractive.com has some products that will do this. I used them for a short while and they seemed pretty good.

Re:mercury

[SpeakerOfTruth]

My company was using Mercury Interactive's WinRunner for a while, but has since switched to Seapine's QA Wizard. We found that with WinRunner, it took a long time to create the scripts - which sometimes took longer because it semi-frequently crashes. We ended up purchasing the advanced training class that WinRunner offers, but the training received only seemed rudimentary. It also requires that your QA people have the ability to program - which I am not sure is a fair requirement. After about 12-14 months of WinRunner, we gave up on it. At the moment, we are using QA Wizard. We haven't been using it for very long, but it does seem to be much easier for the QA people to create their test scripts and test the web pages.

If you didn't already know about it...

[Boss, Pointy Haired]

Web Application Stress Tool (freebie from M$)

http://webtool.rte.microsoft.com/

Cactus

[sterno]

Well if you are working in Java, I've used Cactus before with success. It's based on junit, and allows you to do unit testing on servlets/jsp's in a nicely automated way. As long as you take the time to create good test cases, it can do quite a good job.

Cactus or HTTPUnit

[revscat]

Both Cactus and HttpUnit allow you to do unit tests on web components. Both are extensions of JUnit. Cactus allows you to do unit tests of servlets and JSPs, while HttpUnit allows for unit tests of the resulting HTML code. (Cactus also integrates HttpUnit to a certain degree.)

Obviously, these tools are targeted at Java development. I have less experience with HttpUnit than with Cactus, but I imagine it could be used as a general test suite.

many open source test frameworks available

[consumer]

My experience with commercial load-testing apps is that they are outrageously expensive, a pain to program, don't really scale all that well, and mostly have to run on Windows with someone sitting at the mouse. There are some that work better than others, but the free stuff in this areas is quite good.

I recommend httperf and http_load for banging on lists of URLs really hard. At one place I worked, one of our developers rigged up some shell scripts that would play back log files through httperf and that worked pretty well.

If you want to record browser sessions for testing specific paths through the site, look at http-recorder or roboweb. There's also webchatpp, HTTP::WebTest, and HTTP::MonkeyWrench on CPAN. More info on this can be found on the mod_perl mailing list or on PerlMonks.

Get a good QA person

[gosand]

If you are a developer, do what you do best. If you want a tester, go out and find a good one. They are worth their weight in gold.

OK, maybe I am a little biased, as I have been in QA for 8 years. :-) But my comments still stand.

That said, we are currently using Rational's products to test our application, which includes a web piece. Hint: Don't use javascript if you plan on using Rational. They have SiteLoad, which I believe is free, but rest assured the rest of their products are NOT. Their licensing scheme is nothing short of trying to balance the budget of a small country. If you are wanting to implement their products in a big project, to handle requirements (Requisite Pro), Bugs (ClearQuest) and test plans (Test Manager), then prepare yourself for headaches. If you just want to get Rational Robot to record/playback user actions for testing, it is pretty solid. Rational purchased all different components of their system, so they aren't the smoothest to integrate. I have spent many hours with their phone support people.

I have also worked with Mercury and SilkTest, but to a lesser degree.

Oh, and if you are constantly changing critical code, you need to worry more about your development practices and not your testing.

Java Tools

[GOD_ALMIGHTY]

A couple of people have already mentioned the Jakarta project and cactus in particular.
I'd highly recommend picking the book:
Java Tools for eXtreme Programming

This is a great reference for all of the tools being mentioned and shows you how to integrate them into the development cycle if your using Java. You should be able to write the functional tests if your app is not written in Java.

As an aside, if your not developing these apps in Java, you really should look at using Tomcat, XDoclet and Struts for simple DB frontends, and then move to EJBs with JBoss, Jetty or Tomcat, Struts and XDoclet. If your lazy and don't want to write a lot of code, you'll love these tools. Reuse is high in Java, and the code generation tools like XDoclet take away most of the pain of using frameworks like EJB and Struts. Besides JSP taglibs allow me to have good looking pages made pretty by people who care about the differences between browsers for CSS, DHTML and what not.

Good Luck.

QA Wizard

[igiveup]

Seapine Software produces a product called QA Wizard that is a fully scriptable testing tool for web applications using Internet Explorer. Netscape/Java support is coming soon. A Windows application testing tool should be available by the end of the year, as well as a load testing tool.