Testing Products for Web Applications?

hellbunnie asks: "I work with a team of developers many of whom spend much of their time writing web-based front-ends for DBs or other applications. Now, while we enjoy programming, we're pretty lazy when it comes to testing. Even if we weren't so lazy, I think we'd still miss a number of problems, 'cos there's just so many different screens that use any particular method/function that you might change. That means there's a lot to be tested after each change. So, my question is has anyone any experience with automated systems for testing web applications?"

"I've seen a lot of automated test suites advertised and I've always assumed that they were no substitute for careful testing by a human. However, as the number of web pages that we need to maintain grows, I've begun to wish that we had something that we could kick off at night, that would follow all links on our system and fill in values for the various forms it encountered, then when we arrived in the next morning there'd be some sort of report available detailing its findings. It could flag any pages that returned something obviously incorrect, such as a SQL error, a blank page or just the word 'error'.

Does such a thing exist or am I just engaging in wishful thinking to imagine that there might be something flexible enough to do the job? What do other people do to test their software?"

Comments

Try using people and kids!

[mustangdavis]

I run a couple free web games ... and let me tell you ... if it has a security flaw, these people will find it! Hire a couple people that play my game! I'm sure they'll find any security flaws you may have!

Seriously, I don't know of any software that does that, but if you find one, I'M INTERESTED!

I don't know if you're looking for advice or not, but try putting in negative numbers or things like #(-3+1000) ... or end a SQL query in a text box and try to execute another query (or put in a sub query) ... and edit your query strings if you use GET (or build a query string and make sure that your program doesn't take a GET where it is looking for a POST) .... just a couple basics to try ... You might want to write a "validate_input" function for your forms as well ....

Hopefully that helps a little ...

More important ... Interface testing

[Anaphilius]

I used to work on such a system, and I did a lot of the testing for it as well. We found quite a few bugs, but those were overwhelmed by the number of interface design flaws we encountered. There were literally hundreds of unnecessary mouse-clicks per user per day in the original interface, simply because the programmers never had to use their own software for days at a time.

So I guess my point is, make sure you don't simply rely on automated testing. A bot won't get sick of clicking unnecessary buttons, and won't develop RSI injuries. Humans will, and you'll get great feedback because of it. At my old company, the programmers were very nice about fixing these flaws once I brought them to their attention, and grateful for our input.

Cheers,
Anaphilius

Re:Get a good QA person

[sohp]

if you are constantly changing critical code, you need to worry more about your development practices and not your testing.

Not true in many developement shops. With short iterations, refactoring, rigorous unit testing, collective code ownership, and continuous integration, code can be constantly changing but stable. Take for example the Mozilla Tinderbox. Development proceeds on many components and the builds and tests are run continuously. There are daily build smoketests (download a daily build and you'll see the smoketest menuitem), and sometimes things are broken for an hour or a day, but overall things just get better.

Embrace Change.

This misses the point of testing and QA altogether

[jea6]

Your proirity in testing shouldn't rely on automation necessarily because what you are bound to find is that the application works perfectly, when it's following the script you've programmed. When somebody on my team brings me code/functionality to review, the first thing I try to do is to "do the wrong thing" (eg. letters in a field to be interpreted as numeric). Thorough testing requires "unbridaled" human ingenugity.

Frankly, what you need are probably consistent programming methods (because your front-ends are probably being written by liberal arts majors who taught themselves --insert language here--), through error handling, documentation, a consistent testing mothodology, and much more upfront requirements analysis.

This stuff ain't cheap and you need to factor it into your pricing. I'd say that 10% to 20% of your budget should be QA and testing and you should insist that the budget be used for that. Too often QA time is used for actual development, leaving no QA.

Sigh..

[Inoshiro]

Your entire question, well, sucks. If you think you can test at the end of a product cycle, you're smoking the kind of crack cocain that leads to things like this.

When you write a function for your program, you need to write a test unit that is in the debug project. How it will work is that you write some tests in which you take an input, perform the operation, and test the output versus a contstant answer. Have one of these for each case that it handles in the unit. That way, you can always compile the test unit and examine its output versus the constant known-good value. That's good software engineering practice.

What you're asking, well, is a joke. Nothing's going to save your project if you've been just adding functionality without QAing at each step to verify correctness.

hellbunnia asks "I work with a team of developers who spend most of their time adding functionality to code. While we enjoy just cramming more code onto a source tree, we really never test anything. But even if we tested it, I think we'd miss a lot of bugs because we have no design policy. It's a lot to be tested, and it's all interrelated! So my question is, does anyone have a quick and easy solution that will save us from rewritting things with a proper design?"

"I've read a lot of freshmeat listings for testing, but I've always assumed that they were merely 'Hello, World' programs because nothing beats real testing by real humans. However, as the amount of code grows, I've begun to wish that we wrote a carefully designed set of unit tests as we added functionality, rather than trying to magically make it all work 2 weeks before our shipping deadline. I'm hoping we have some magic QA program which will do everything for us, except actually fix our squirrely code.

Does such a thing exist, or should I start updating my resume? How fucked am I?"

Re:Roll your own?

[andy@petdance.com]

Right Perl Modules?

• WWW::Mechanize for the automated fetching.
• for the HTML validation

both of which I just happen to maintain.