Slashdot Mirror
Practical Software Testing Resources?
rhartness asks: "I've been a software engineer by profession for a few years and a programming enthusiast for much longer. As my experience has increased, so has the size of the projects that I have had to work on. My software testing method involves trying to do everything I can think of that the end user might try to do. Hopefully, this will break the application if there is a bug within my code. The current project that I am working on involves numerous tiers within a smart client environment. Trial and error testing is no longer sufficient — there is simply too much that could happen. Searching the Internet for software testing resources provides an abundant amount of information but it's often quite philosophical and verbose. What are some practical resources that Slashdot readers use for testing your software projects?"
Release it with a big BETA label. Let your customers find the bugs.
Read "The Art of Software Testing" by Glenford Myers. Even if you only get a few chapters in, you'll get a lot of ideas that will improve your testing. "Testing Computer Software" by Cem Kaner is another good one.
A huge step is to put in unit testing / regression testing that's run nightly, and idealy with every build. You should at least cover the basics, e.g., at the persistence level can you create/read/update/delete a record? If you load a parent object, do dependent objects get loaded as well? At the presentation layer you can verify that a missing field will set the right error message.
You still need to do additional testing, but this will catch the underlying errors that can cause flakiness -- and worse, bad workaround -- at the higher levels in your code.
For every complex problem there is an answer that is clear, simple, and wrong. -- H L Mencken
Step 1: write a systematic test plan. There are all kinds of acronyms and books out there on the subject - I doubt you need it. At the very least you can start with CRUD (Create, Read, Update, Delete) and go from there. Cover the success path, cover the failure paths.
Step 2: automate. This isn't optional if you're planning to maintain a project beyond the 1.0 release. For specifics, it depends on your project. Is it mostly a Java app (http://www.junit.org/)? Is a lot of the logic in the database (http://www.dbunit.org/)? Is it a web app, regardless of language (http://webtest.canoo.com, http://wtr.rubyforge.org/)?
Step 3: run your automated tests and laugh at how much easier it makes your life than manual testing.
ClutterMe.com - easiest site creation on the Net. Just click and type.
Given the additional information, I have a little more to say. It reinforces my impression that you are basically on the right track, there are just no silver bullets. However I think unit testing is probably more significant for you than I assumed at first. I think jt2190 expressed the value of unit tests very well.
I also agree with jt2190 about the significance of requirement changes. This is absolutely the area where you stand to make the most gains. If your boss does not have professional software development experience him/herself than he/she almost certainly does not appreciate how severe the consequences of changes are. Even if you tell them, they won't fully get it.
The fact that your boss is the brain child is a double edged sword, but mostly positive I think. Try to push more of the testing burden on your boss. After all, it's your boss who best understands how they want it to work. Explain that he/she can't just test the part that changed, but must as least give a cursory test to everything to check for side effects. That will help reinforce the importance of avoiding changes. Assuming your boss is as busy as most bosses are, it will also increase your chances of getting a dedicated tester hired (most likely from zero to slightly non-zero if you are the only dedicated developer).
When you get a new requirement or a requirement change, question it. Make sure you understand the underlying concern that is supposed to be solved by that requirement. This is an art, I'm sure you're getting better at it already. More importantly make sure THEY understand both the concern as well as the solution they are proposing. Specifying good requirements, just like good testing, is very hard and a lot of it comes from misidentifying the problem itself. If you have a good way of producing a mockup that allows them to explore the solution hands-on without needing to invest a lot of time to create it, it will go a long way. That's tricky as there are two closely related traps: the quick-and-dirty mockup evolves to production code without adequate planning or the quick-and-dirty mockup gets thrown out entirely despite the fact that some of it was valuable.
One thing I noticed you didn't say is that you are under pressure from your boss to reduce bugs. That leads to me to believe you realize how time consuming bug fixes can be and are self-motivated to find a solution. If so, that's a good sign.
One last thing that I should have mentioned initially as it applies to almost all situations, clean up that old code. It's inevitable you'll find crufty code as you go back to things. Sometimes it started that way for whatever reason (most often time pressure), sometimes it evolved that way. Cleaning it up (also known as refactoring) pays off 95% of the time. Unit tests are perfect when you want to change the code but not the behavior.