Slashdot Mirror
How Can I Make Testing Software More Stimulating?
An anonymous reader writes "I like writing software. In fact, I revel in it. However, one thing has always kept me back from being able to write the best software I possibly can: testing. I consider testing to be the absolute bane of my existence. It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it. I know I'm not that lazy, as I can spend hours on end writing software, but there's something about testing that makes my mind constantly want to wander off and think about something else. Does anyone have any tips on how I can make non-automated testing a little bit more stimulating so I can at least begin to form a habit of doing so?"
just imagine if you don't test giant sharks with lasers will attack you.. with that stimulate you enough?
Personally I feel that the dev should never do the testing of their own code as they are too close to the subject to test every angle.
:P
Sorry this doesn't answer you question
http://www.gibby.net.au
But you should be writing the test as you write the code
Add porn
Wear a different hat each day.
Does anyone have any tips on how I can make non-automated testing a little bit more stimulating so I can at least begin to form a habit of doing so?
No, I don't. I strongly think you're directing your effort the wrong way, and duplicating work if you're spending too much time on non-automated testing.
Software Engineers are not good at poking holes in their own work, so you should have someone else doing the bulk of that kind of testing anyway. You obviously need to do some cursory testing to avoid wasting someone else's time, but there are much better ways of directing your testing effort.
Focus on developing unit tests both before and during the development effort. Avoid developing your unit tests after writing the code though - your mind will be tainted with your approach, and you'll miss the obvious stuff. Not only do unit tests reveal bugs, the act of writing them will also help you get interfaces right, and help ensure a better overall design for your code.
Clearly there aren't enough bugs in the software you are testing. As an experienced C programmer I can help fix this problem...
http://michaelsmith.id.au
First off, don't do non-automated testing. It's unnecessary. Do Behavior Driven Development with Cucumber http://cukes.info./ It's massively more fun than unit testing.
Have you given Test Driven Development and NUnit a try? If you haven't heard of these things or taken one of the TDD frameworks out for a test drive then you owe it to yourself to invest some time. The software industry in general and the .NET community in particular is moving towards automated unit testing and Continuous Integration. You should get on this train too.
Allow yourself one porn^H^H^H^H^H download per bug found. Not only will this stimulate you, but you will be able to test with one hand.^H tied behind your back.
Imagine that the guy who wrote the code is the royal, snobbish jerk who always thinks his code is better than anything else. Realize that, at some point, he's going to make a change that will change the way the component behaves, and will yet again break your code that uses that component. Don't let him get away with it. Let your fingers pound as you write code to check for the smallest fault. Let your lips curl in a snarl as you feed it poisoned data. Relish in the thought that you can catch him in every slightest mistake. Think of all those things that might not think to catch in the future. Not just the important stuff that defines the role of the component, but those little things that some people might call "implementation details".
Also remember that he's the same jerk who wrote those unit tests that screwed you over all those times. Don't just pay it back; pay it forward.
tasks(723) drafts(105) languages(484) examples(29106)
If you're not testing your code, you're not doing your job. Find ways to automate your tests.
Dog is my co-pilot.
http://exogen.github.com/nose-achievements/
*
Instant Feedback
A suite of at least 50 tests takes less than a second to run.
*
Coffee Break
The suite takes between 5 and 15 minutes to run.
*
Take a Walk
The suite takes between 15 and 60 minutes to run.
*
Take a Nap
The suite takes between 1 and 5 hours to run.
*
Take a Vacation
The suite takes at least 3 days to run.
*
Coming soon!
Anticipation
One test in a suite of at least 10 is slow, and all the rest pass.
Time
*
Night Shift
Make a failing suite pass between midnight and 5am.
*
Punctuality
Make a failing suite pass at 9am (give or take a minute).
Failure
*
Complete Failure
All tests in a suite of 50 to 999 fail.
*
Epic Fail
All tests in a suite of at least 1,000 fail.
*
Minor Letdown
All tests in a suite of 10 to 99 pass...except the last.
*
Major Letdown
All tests in a suite of at least 100 pass...except the last.
*
Happy Ending
All tests in a suite of at least 10 fail...except the last.
*
Coming soon!
Heisenbug
Make a passing suite fail without changing anything.
Errors
*
Sausage Fingers
Cause at least 2 distinct syntax errors in a single run.
*
To Understand Recursion
Exceed the maximum recursion depth.
*
Coming soon!
To Err is Human
Cause all tests in a suite of at least 50 to fail with a single error.
Size
*
My God, It’s Full of Dots
The suite has at least 2,001 passing tests.
Frequency
*
Coming soon!
OCD
Run the suite at least 5 times in the spa
Reviewing just the first hour of video games.
http://en.wikipedia.org/wiki/Milgram_experiment
except for you, the shocks will be real, and you will be both tester and test subject
every time your code throws an exception, make it so you receive a small, incrementally growing shock
after about 10 or 20 bugs, and the growing voltage of the shocks, you should be breathing heavily and sweating and be focused like a laser on ensuring the integrity of your code
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
http://thedailywtf.com/Default.aspx
The threat that one day someone will post your code and or screen shots from your programs for everyone to ridicule should be motivation to either improve or write worse code.
---- Fight to protect your right to keep and arm bears! ummmm... ya I think that's right....
Wear a different hat each day.
I put on my robe and wizard hat.
I hate testing too! So I devised a little game with a compatriot in code: we unit-test each other's code. Each class is unit tested by the other person, and whoever finds the most bugs wins. What do you win? Usually pizza and beer. Free lunch or dinner can be a strong motivational reason, plus the chance to brag about trouncing the other person. Makes you a MUCH better "defensive" coder, always thinking about how/which parameters to validate, return codes, etc.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
"It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it."
Which sums up why software is so shitty today. I seriously hope that you don't write software for the areospace industry because I don't feel like falling out of the sky because you were too bored to test your code.
Every job has its boring moments, testing your code is one of those things that programmers must do. Should do, it encourages discipline and discipline is what makes good code. You can automate the testing to some degree but at some point you've got to poke it and prod it yourself because computers are stupider than even we are. If you can't hack that, find a different line of work.
Bibo Ergo Sum.
> The developer should unit test, and the test group should system test.
In my experience, the developer should participate in system testing too.
System testers have to know what the corner cases are. They can't guess them all (unless they're more skilled at each unit than the unit's developer is, in which case what's this product genius doing in the system test group!). Brute force takes too long for most systems, and even brute force requires knowledge of what variables to change and what ranges to span for each variable.
I used to program embedded systems for oil pipeline tools. (Not a sector with a glowing reputation for reliability right now, but bear with me.) The system testers had decades experience in pigging, hydraulics, tool operation, an ISO certification, but they weren't programmers. The whole team participated in system testing (or a member of each unit group did), so that we'd collectively know all the combinations worth testing.
(As for answering the question of the story, I think the asker really should have given a hint as to what sort of software it is.)
Expert in software patents or patent law? Contribute to the ESP wiki!
I do software QA for a living. And if you're not a tester, don't try to be. It's your job to write code that meets spec, runs clean, is efficient and effective. Write it well. Write it secure. Write it to handle errors from data, users, networks, etc. Double check that you validate input. Make sure it doesn't leak memory. Write good unit tests. Test it enough to make sure it works. Then give it to a tester.
Good software testers are a different breed. They are a sceptical, picky, pedantic, detail oriented bunch who take new code as a personal challenge to find the inevitable bugs. They will test your code a dozen different ways you would never think of. They will find bugs that could not possibly exist. They don't care that your shiney new whistle or bell will be the next big thing that will make you all rich. They care that it doesn't barf when you pass it a string with more than 256 characters. Including special characters. In German. Or Japanese. They care that when it's been running for 12 days straight with automated stuff beating on it that the memory usage hasn't ballooned. They care how it deals with data files 10 times larger than you say it should handle, or runs on a machine with half the ram it should have, or handles twice the workload it should - because somewhere out there is a user who will ask it to. They will chew it up, spit it out, and ask you to fix it. Then they will do it all again.
Testers are a strange bunch, and good ones are hard to find. Find some good ones and cultivate them. They are a lot cheaper than a ticked off client.
-- "Never underestimate the power of human stupidity." - R.A.H.
You don't need to make it more interesting, you need to make it part of your routine. It's like waking up at 6am to go for a run. The first few times you do it it'll be hard, but if you stick to it and do it every single day it will become just something you do. Start small - don't try to run a marathon in your first week, just don't make any excuses or invent reasons to skip it just this once. The fact that you care enough to ask slashdot suggests that it is important enough to you to make the effort. If you can make the change and stick to it you will probably find more fulfillment in your work.
Unexpect the expected!
I used to dislike testing until I learn how to implement code designed to be tested. Use a dependency injection frame work (that will keep you busy for a while) and write testable code. Writing elegant, readable code which scalable and testable is not an easy or boring task. If you can not automate the tests, you are probably do something wrong.
M0571y H@rml355.
The reality is that all software ships with bugs. Some known and some unknown. Typically it depends on how easy it is for the customer to find, what is the impact to the customer, cost to fix, and risk of regression. Given that software is typically patched after shipping, it means even more bugs get shipped rather than slipping the ship date.
"Here's how it works..."
That is, if you can write a test that looks like that. I work on compilers, so the trick is to write the example that tweaks just the feature that I want to check, and not 17 others. But, my test cases look like squirrelly little programs that could actually be inputs from especially peculiar users.
You ought to also think about demonstrating your error cases. "If you do this wrong thing, it says...." I am sort of a nut about error messages, partly because I once read the Apple Human Interface guidelines and thought they were a good thing. In particular, does the error message provide the user with information that will help him make the error message go away? When you decide that an error has occurred, what special information do you have that might help? My anti-favorite error message was "Bad SOCKS version number". Not "unexpected", not "too old and not-supported", but "BAD". Morally wrong, so unspeakable that not only can the BAD number not be named, even the good numbers cannot be named, because they might therefore name the BAD number by process of elimination. Truly unspeakable, the version number that shall not be named.
One word: Fluffers
It's inherently boring....
SO, Build hooks into the 'ware as you write it, and automate the testing.
Work smart, not hard.
Red
Actually it's just the opposite. It's like working in a candy factory and quickly getting completely bored/sick of the candy. I did one site where we were having problems with certain videos in different browsers and I swear to God that there was one clip of some really hot anal action that I got *really* sick of. After about the 3rd time I would watch about 5 seconds of it, see if the problem was manifesting, and then be back in the editor.
Now I watch whale documentaries to get my jollies. ... Wait! What?
Do you enjoy finding the weak points in things? Do you use new devices, maps, etc without reading the instructions because you just think you should just know how they should work? Do you find that alarm bells go off in your head when you read a phrase that could possibly interpreted in more than one way? Do you often use things in ways that they were never intended yet those ways seem the most logical to you? Do you possess powers of intuition that lead you unforseen vulnerabilities? Do you find the needle in the haystack? Coding isn't inherently more interesting than testing. Anyway, if you identify with any of these phrases, you can apply it to testing...
Unit tests should excercise the corner cases in your code. If you know what they are, write tests for them.
QA testing should break all design assumptions about how the software should be used. Having the programmer sitting there telling the QA guy what to click on (and I've seen that far too often) invalidates that. The most useful bugs are the ones where the QA guy says "I did what I thought would get the job done, and instead it formatted my hard drive", leaving the dev to sputter "but, but, you're not supposed to do that". Given enough users, every possible "stupid" thing you can do with your software will be done in the field, and you really want to know that you will at least fail safely in all those "but that's not how you're supposed to do it" cases.
Socialism: a lie told by totalitarians and believed by fools.
Read Slashdot instead.
Then you have hookers killing themselves from boredom.
Although... trying to explain away the dead hookers would make debugging a lot more interesting.
Democrats or Republicans. They are both taking us to the same place and they are not afraid of us anymore.
System testers have to know what the corner cases are. They can't guess them all
That's why you need proper documentation, like use cases, technical designs. Often (and preferably) written by analists and not the developers.
Preferably you have a setup like this:
Business analyst writes documentation based on requirements from the business.
Developers build the application, based on the documentation.
Testers write testcases based on the documentation and test the software as soon as it is released to them.
Testing is a profession too.
And there are many tools and methodologies (TMap, ISTQB, Testframe etc) to ensure proper test-coverage and to have anything meaningful to say about the quality of the tested application.
80 CC D8 AF AE D3 AB 54 B7 2E CE 67 C7
"but, but, you're not supposed to do that".
Yeah, that one still makes my *hair* stand up. Most devs are superb at narrow domain problem solving. Of course, reality and users tend not to accommodate them. I can't tell you how many times I've pushed the limits of a software package to get something done and it fails miserably.
So, yes, they will have 37 instances of the software running at once and yes, they will try and save one or more projects/files with the same name at the same time and yes, no matter how many times dev says it's their fault, they're wrong. Machines are designed to accommodate people, not the other way around.
Please do not read this sig. Thank you.
The requirements are vague, such as "difficulty of problems presented to the user must increase gradually over the course of the session," where difficulty isn't defined rigorously.
Uh, you define the requirement. If you've written code, you've *already* done that, you've just made the requirement inaccessible as its now encoded in a programming language. Write it down first and everyone will be happier.
The application's specification includes a physical simulation with random, pseudorandom, chaotic, or otherwise nonlinear behavior
That's not unit testing. Most code in a simulation is *not* random, it's fed inputs from other parts of the code, which ultimately result in the behaviour you describe.
Automating system test for something like this is interesting. First, you *must* design the code to be testable. That means being able to, for example, define random number seeds and so forth, so that a run can be reproduced.
Second, the implicit assumption is that there is *some* way to determine, based on inputs, if the outputs of your software are correct. If you can't do that, you have far bigger problems. Of course, those tests might be fuzzy ("value should be between X and Y", "value should be greater than zero", etc), but they're still tests.
Third, realize automated testing isn't the end-all and be-all of testing. For some systems, you will be forced to have some amount of manual testing (for example, integration testing large, complex software systems can be difficult to do in an automated fashion). That said, I'm willing to bet, for *most* systems, you can automate most if not all of the testing.
You know your automated test suite's coverage isn't 100%, and you're doing non-automated testing to find things that your automated test suite is missing.
Beg the question much? If your test suite is missing things, fill in the gaps. If you can't fill in the gaps, ask yourself why.