Slashdot Mirror

This. Mod parent up.

In (very) short, "testing is evaluating a product via experimentation" (see http://www.satisfice.com/blog/...). According to this definition, truly anyone can test. Anyone can "evaluate a product via experimentation".

However, formal, professional testing also has a purpose: to inform. That is, "testing provides information about the quality of a product so that others can make informed decisions."

So, formal, professional testing is "evaluating a product via experimentation - in order to inform". And /that/ requires "a modicum of skill and critical thinking".

Mod parent up.

http://www.developsense.com/bl... is a treasure-trove of testing (and other) information. Simply reading his (and similar) blogs is an quick, easy, and effective (and free!) way to learn about testing. Also, be sure to check out the blog of James Bach for the same reasons: http://www.satisfice.com/blog/.

You are confusing testing (evaluating the product by learning about it through experimentation) with what we call checking (evaluating functions in the product by algorithmic observations and decision rules). http://www.satisfice.com/blog/.... Checking can find lots of functional problems in the product, and it's an important thing to do. But that's not the same as testing the product. Knuth made this distinction using different words years ago: "Beware of using this product. I have merely proven it correct; I haven't tested it."

If your heart acquires strength, you will be able to remove blemishes from others without thinking evil of them.

Mohandas Gandhi

It is unusual for a software shop to be as immature in its tools and processes as your new employer. Rather than join in the chorus of voices condemning your employer, let me suggest an alternative.

Understand the reasons for the current situation. Professionals, especially engineers, usually have a rational basis for their choices. Perhaps they are disillusioned from having wasted time and energy in the past (see Test Automation Snake Oil). Perhaps they have a few heroic individuals who hold everything together and don't see the need for tools. I have no idea. I cannot wrap my brain around how someone would try to get by without revision control but that's immaterial. You have to understand that before you can either change it, or learn to live with it.

Read the IEEE paper, How to Be a Star Engineer. Then, be a star. Help your team see the value in some basic tools like version control. Introduce them. Train your peers. Proceed slowly and patiently. Talk to your managers and senior staff about the risks you can mitigate and be realistic about the costs of doing so. In other words, help your company do better software engineering.

It is very possible they hired you because you come from a disciplined engineering shop and can help them improve their practices.

Or you can take the coward's way out and flee before you try to teach anything or learn anything.

Cem Kaner wrote a Bill Of Rights for Software Customers: http://www.satisfice.com/kaner/?p=8

He introduces it with this:
"As the software infrastructure has been going through chaos, reporters (and others) have been called me several times to ask what our legal rights are now and whether we should all be able to sue Microsoft (or other vendors who ship defective software or software that fails in normal use).

"I'd rather stand back from the current crisis, consider the legal debates over the last 10 years, and make some modest suggestions that could go a long way toward restoring integrity and trust -- and consumer confidence, consumer excitement, and sales -- in this stalled marketplace."

1. Let the customer see the contract before the sale

2. Disclose known defects

3. The product (or information service) must live up to the manufacturer's and seller's claims.

See Cem's post for 4 through 10

Someone above recommended Edward Kit's book. Definitely worth reading, imo.

But the biggest reason I see for developers not being so good at testing, is that they are of the mind set that they try to show how the SUT does work. Not really in the least bit helpful, that is what your unit tests should have been doing. No, the mind set is 'In what ways does this fail?'. All software fails, you just have to find out how. That is probably one of the reasons you think so much of the literature is of a philosophical bent, because it is. There are techniques that can help you find the places where it'll break (wikipedia - boundary value analysis, equivalence partitions and software testing may help), but ultimately it is just a form of scientific method. Your SUT is a hypothesis, the only way to show validity is by attempting all you can to disprove it.

You may want to have a quick read of James Bach's site, and may even see if you can get yourself along to one of his trainings.

GUI test automation (Mercury, Compuware et al) - they are potentially useful, but you need to be able to understand how to test before you can write good automated tests. Unfortunately this is often not the case, hence the oft heard saying "Test automation, enabling you to test badly, quicker." Test automation is not the same as manual testing, you need to understand how it is different for it to be of value (beyond unit testing - you probably know how to unit test, hence you probably have at least an initial idea of how to automtate unit tests).

Feel free to ask more, or disagree, or just curse me some :)

And just to own up to my own bias, I've been a tester for around 15 years.

I am a software test engineer, I lead a small software test team, and am a huge advocate for automation when it is the best solution to the problem.

James Bach, a leading test automation consultant, wrote an excellent article[PDF] debunking some of the reckless assumtions folks make about test automation.

Generally speaking (and this is VERY context-dependent) I would like to have both skilled manual regression testing and regression suites written by skilled automation engineers, but if forced to pick one or the other, I will advocate for manual testing almost every time. Test automation excels in concert with manual testing, not in place of it.