Slashdot Mirror
How Can I Make Testing Software More Stimulating?
An anonymous reader writes "I like writing software. In fact, I revel in it. However, one thing has always kept me back from being able to write the best software I possibly can: testing. I consider testing to be the absolute bane of my existence. It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it. I know I'm not that lazy, as I can spend hours on end writing software, but there's something about testing that makes my mind constantly want to wander off and think about something else. Does anyone have any tips on how I can make non-automated testing a little bit more stimulating so I can at least begin to form a habit of doing so?"
just imagine if you don't test giant sharks with lasers will attack you.. with that stimulate you enough?
Personally I feel that the dev should never do the testing of their own code as they are too close to the subject to test every angle.
:P
Sorry this doesn't answer you question
http://www.gibby.net.au
But you should be writing the test as you write the code
eom
Add porn
Wear a different hat each day.
I found that some of my best fun is to write software to test my software. Fortunately, I have written server code for a long time so I either get to write clients to exercise it or hardware device simulators to emulate devices the server talks to.
Loads of fun! Quite often, more fun than the SW I'm supposed to be writing.
-- Many men would appreciate a woman's mind more if they could fondle it
Does anyone have any tips on how I can make non-automated testing a little bit more stimulating so I can at least begin to form a habit of doing so?
No, I don't. I strongly think you're directing your effort the wrong way, and duplicating work if you're spending too much time on non-automated testing.
Software Engineers are not good at poking holes in their own work, so you should have someone else doing the bulk of that kind of testing anyway. You obviously need to do some cursory testing to avoid wasting someone else's time, but there are much better ways of directing your testing effort.
Focus on developing unit tests both before and during the development effort. Avoid developing your unit tests after writing the code though - your mind will be tainted with your approach, and you'll miss the obvious stuff. Not only do unit tests reveal bugs, the act of writing them will also help you get interfaces right, and help ensure a better overall design for your code.
Duh.
Clearly there aren't enough bugs in the software you are testing. As an experienced C programmer I can help fix this problem...
http://michaelsmith.id.au
First off, don't do non-automated testing. It's unnecessary. Do Behavior Driven Development with Cucumber http://cukes.info./ It's massively more fun than unit testing.
I know, I know, TDD is talked about ad nauseum, but I often find that when I've just written some logic, and I'm writing some tests for it, that I pick up quite a few bugs, and actually learn more about what I'm really writing.
Even so, some parts can still be boring, but at least, if you break it up and do it test-driven then you're not faced with writing 47-odd tests in one big lump, after you've written heaps of logic.
"Laugh while you can a-monkey boy!" - Dr Emilio Lizardo
people in the front rows may be exposed to some splash effects. some viewer experiences involve heart attacks, and conjunctivitis. (http://en.wikipedia.org/wiki/Conjunctivitis)
I've only done hobby programming for a couple of years, but if you find that testing isn't exactly your cup of dried leaves steeped in hot water, I would suggest viewing it as a challenge: If you love writing code, then it should follow that you like writing good code. The only way you can determine that is by trying to break said code. Sadly I don't usually need any additional helping breaking my code, as it often breaks itself, but by shirting it into a challenge against yourself, your code and motivation will benefit.
You need to write tests for your code so that testing can be fully automated, with the issuing of one command. If there are no ways to automate your testing, you need to write software that allows you to do that, and then use it. If it is useful enough, you can even sell that.
This
Get used to it, comrade.
I think it's just who you are, and you're probably wise to partner with others.
I'm quite the opposite. I like nothing better than to write a great benchmark, because I'm an optimization fiend. Also love to write a correctness testing benchmark, because it lets me check my optimizations for accuracy. However, I can't stand most high-level coding work, because I get horribly distracted. My brain gets distracted by what's happening on the metal when it doesn't matter at all, and then I get really bent out of shape about the aesthetics of the code, which for most jobs where you need to get things done doesn't matter much until you have something working, but for whatever reason, it's just hard for me to enjoy it.
Then go back and do it again, but this time play black-hat hacker. Get creative rather than methodical. Try sql statements everywhere that might touch a database. Hammer every communication method it uses with denial of service abuse and then crafty mis-information that only you know enough to design.
The point is, make a game of it and you'll enjoy it. You could always trade code with a co-worker and keep score of how many bugs you each find in eachother's code.
On the one hand you take life too seriously, and on the other, you do not take playful existence seriously enough. Seth
I have the same problem. If you figure it out please let me know.
Have you given Test Driven Development and NUnit a try? If you haven't heard of these things or taken one of the TDD frameworks out for a test drive then you owe it to yourself to invest some time. The software industry in general and the .NET community in particular is moving towards automated unit testing and Continuous Integration. You should get on this train too.
Allow yourself one porn^H^H^H^H^H download per bug found. Not only will this stimulate you, but you will be able to test with one hand.^H tied behind your back.
tele-dildonics hooked to the exception mechanism.
Imagine that the guy who wrote the code is the royal, snobbish jerk who always thinks his code is better than anything else. Realize that, at some point, he's going to make a change that will change the way the component behaves, and will yet again break your code that uses that component. Don't let him get away with it. Let your fingers pound as you write code to check for the smallest fault. Let your lips curl in a snarl as you feed it poisoned data. Relish in the thought that you can catch him in every slightest mistake. Think of all those things that might not think to catch in the future. Not just the important stuff that defines the role of the component, but those little things that some people might call "implementation details".
Also remember that he's the same jerk who wrote those unit tests that screwed you over all those times. Don't just pay it back; pay it forward.
tasks(723) drafts(105) languages(484) examples(29106)
Do it like the pros: release the software and issue patches.
The bug reports will be great fun.
I hit the save button and the program crashed. Thanks a lot asslick."
If you're not testing your code, you're not doing your job. Find ways to automate your tests.
Dog is my co-pilot.
It's the exponential that increases fast, exp(10) = 22026.4
TDD is the best approach. Write unit tests BEFORE you write the code. First of all you'll be motivated to get it done. Second of all you'll write MUCH more testable code and thus spend a lot less time on testing. In the long run you will actually get to write more fun stuff than if you didn't do it this way.
Honestly, not all testing will ever be fun, but there's really no point to writing bad broken software and untested software is invariably bad software. Honestly though whatever organization you're working in should be insisting on this kind of process anyway. First thing I do when I set up a team is get the unit testing process up and running.
"Malo periculosam, libertatem quam quietam servitutem." -- Jefferson
Its like asking "How is the most stimulating way to self file my taxes?" The answer is most likely there isn't unless you're of the rare few that do enjoy doing just that. Easiest way is to just hand it to someone else and get creative in forms of payment, maybe booze or some movie/music/game they want for doing it (everyone is different).
Attention... all grammer nazi"s! Is they're anything; wrong with: my post,
http://exogen.github.com/nose-achievements/
*
Instant Feedback
A suite of at least 50 tests takes less than a second to run.
*
Coffee Break
The suite takes between 5 and 15 minutes to run.
*
Take a Walk
The suite takes between 15 and 60 minutes to run.
*
Take a Nap
The suite takes between 1 and 5 hours to run.
*
Take a Vacation
The suite takes at least 3 days to run.
*
Coming soon!
Anticipation
One test in a suite of at least 10 is slow, and all the rest pass.
Time
*
Night Shift
Make a failing suite pass between midnight and 5am.
*
Punctuality
Make a failing suite pass at 9am (give or take a minute).
Failure
*
Complete Failure
All tests in a suite of 50 to 999 fail.
*
Epic Fail
All tests in a suite of at least 1,000 fail.
*
Minor Letdown
All tests in a suite of 10 to 99 pass...except the last.
*
Major Letdown
All tests in a suite of at least 100 pass...except the last.
*
Happy Ending
All tests in a suite of at least 10 fail...except the last.
*
Coming soon!
Heisenbug
Make a passing suite fail without changing anything.
Errors
*
Sausage Fingers
Cause at least 2 distinct syntax errors in a single run.
*
To Understand Recursion
Exceed the maximum recursion depth.
*
Coming soon!
To Err is Human
Cause all tests in a suite of at least 50 to fail with a single error.
Size
*
My God, It’s Full of Dots
The suite has at least 2,001 passing tests.
Frequency
*
Coming soon!
OCD
Run the suite at least 5 times in the spa
Reviewing just the first hour of video games.
http://en.wikipedia.org/wiki/Milgram_experiment
except for you, the shocks will be real, and you will be both tester and test subject
every time your code throws an exception, make it so you receive a small, incrementally growing shock
after about 10 or 20 bugs, and the growing voltage of the shocks, you should be breathing heavily and sweating and be focused like a laser on ensuring the integrity of your code
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
I hate going over old, poorly commented code.
I hate testing code and looking for problems.
I hate debugging inexplicable errors.
What to these all have in common? If you write good, smart code with the right tools you'll not have to do as much of them. Others here have mentioned automated testing and I think that's great. I am no big expert, but improving the way I write code and the tools I write it with reduces the onerous parts mentioned above. I don't do all my projects with automated testing (I probably should) but it helps when you have it.
So, as far as it works for me, I would look first at the way that you program (big bites vs tiny bites, languages, structure) before I would look at the way you test. Testing always sucks. Then again, I'm an idiot.....
...... and idiots rule the world....
http://thedailywtf.com/Default.aspx
The threat that one day someone will post your code and or screen shots from your programs for everyone to ridicule should be motivation to either improve or write worse code.
---- Fight to protect your right to keep and arm bears! ummmm... ya I think that's right....
Create some bugs in the unit test and hunt them down!
Before becoming test-driven I would manually test my stuff, using my judgement to determine the scope of what I'd test. When I became test-driven I saw how writing those tests was like writing the code on the other side of the coin. The design is so clear and beautiful when it's exercised in a suite of tests. I enjoy writing (unit/transactional) tests because it proves how awesome my code is, and is much better at keeping it that way than I am while I'm busy building something more interesting.
--- Bigger bits, softer blocks, tighter ASCII.
Write a test framework that lets you expand the scope of automated testing.
Build better bugs, experience more fruitful testing.
Ever hear the expression "I'd rather write programs that help me write programs, than write programs?" Maybe you can get some enjoyment out of writing the code that helps you test the code you're writing. Write a perl script to test your C code, etc.
My friend Debbie Ann is so promiscuous, instead of an appointment book she needs a package manager
You've just written some cool new fangled program for something. Maybe it automates a process. Maybe it creates a new interface for something. Maybe it's the next app killer. And best of all, you wrote a test suite for it during the development.
But does the test suite itself work right? How do you know that your test suite tests correctly, and completely? Did you also write a test suite test, too?
now we need to go OSS in diesel cars
Wear a different hat each day.
I put on my robe and wizard hat.
I hate testing too! So I devised a little game with a compatriot in code: we unit-test each other's code. Each class is unit tested by the other person, and whoever finds the most bugs wins. What do you win? Usually pizza and beer. Free lunch or dinner can be a strong motivational reason, plus the chance to brag about trouncing the other person. Makes you a MUCH better "defensive" coder, always thinking about how/which parameters to validate, return codes, etc.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
I'm a test developer right now, so this may skew my opinion. When I develop any code, whether it's tests, frameworks, utilities, or actual development, I always consider testing of my own code a stimulating factor. Do you enjoy knowing that your code works as you intended? I do. In my opinion good design is reflected by good testing. If your tests are bad then you won't know if your design is good either. This includes performance testing.
Try this: do no testing at all for a month and see how annoyed you get by receiving a million defect reports.
An even better game... rig your machine up to an explosive device and program it so that if you get the same error message three times, BOOM!
That's exciting, right?
Help me fix my brother's injured butt!
"It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it."
Which sums up why software is so shitty today. I seriously hope that you don't write software for the areospace industry because I don't feel like falling out of the sky because you were too bored to test your code.
Every job has its boring moments, testing your code is one of those things that programmers must do. Should do, it encourages discipline and discipline is what makes good code. You can automate the testing to some degree but at some point you've got to poke it and prod it yourself because computers are stupider than even we are. If you can't hack that, find a different line of work.
Bibo Ergo Sum.
Outsource testing to India. That's their punishment for taking our programming jobs ;-P
Table-ized A.I.
No, there is no way to make testing interesting. I hate it too. Its a "necessary evil" like grooming a dog. The dog survives without it, but that tick infestation can really stress your life out with bugs coming from all directions and showing up in places you never could have imagined.
As a software analyst and a QA engineer, if you think of the software you write as if it were someone else's (preferably someone you know is a poor coder), test the software with the objective of breaking it, and finding those loopholes which will allow you to use the software in ways it wasn't designed to be used. It makes a very monotonous task so much more enjoyable.
For me the interest is raised if I write a few high level tests BEFORE doing any coding. Then code.
I suspect the heightened interest is the pleasure of writing to a spec and meeting a goal.
So often (80% of the time?) I then early on save a good bit of rewriting.
That was Zen, this is Tao
> The developer should unit test, and the test group should system test.
In my experience, the developer should participate in system testing too.
System testers have to know what the corner cases are. They can't guess them all (unless they're more skilled at each unit than the unit's developer is, in which case what's this product genius doing in the system test group!). Brute force takes too long for most systems, and even brute force requires knowledge of what variables to change and what ranges to span for each variable.
I used to program embedded systems for oil pipeline tools. (Not a sector with a glowing reputation for reliability right now, but bear with me.) The system testers had decades experience in pigging, hydraulics, tool operation, an ISO certification, but they weren't programmers. The whole team participated in system testing (or a member of each unit group did), so that we'd collectively know all the combinations worth testing.
(As for answering the question of the story, I think the asker really should have given a hint as to what sort of software it is.)
Expert in software patents or patent law? Contribute to the ESP wiki!
I do software QA for a living. And if you're not a tester, don't try to be. It's your job to write code that meets spec, runs clean, is efficient and effective. Write it well. Write it secure. Write it to handle errors from data, users, networks, etc. Double check that you validate input. Make sure it doesn't leak memory. Write good unit tests. Test it enough to make sure it works. Then give it to a tester.
Good software testers are a different breed. They are a sceptical, picky, pedantic, detail oriented bunch who take new code as a personal challenge to find the inevitable bugs. They will test your code a dozen different ways you would never think of. They will find bugs that could not possibly exist. They don't care that your shiney new whistle or bell will be the next big thing that will make you all rich. They care that it doesn't barf when you pass it a string with more than 256 characters. Including special characters. In German. Or Japanese. They care that when it's been running for 12 days straight with automated stuff beating on it that the memory usage hasn't ballooned. They care how it deals with data files 10 times larger than you say it should handle, or runs on a machine with half the ram it should have, or handles twice the workload it should - because somewhere out there is a user who will ask it to. They will chew it up, spit it out, and ask you to fix it. Then they will do it all again.
Testers are a strange bunch, and good ones are hard to find. Find some good ones and cultivate them. They are a lot cheaper than a ticked off client.
-- "Never underestimate the power of human stupidity." - R.A.H.
You don't need to make it more interesting, you need to make it part of your routine. It's like waking up at 6am to go for a run. The first few times you do it it'll be hard, but if you stick to it and do it every single day it will become just something you do. Start small - don't try to run a marathon in your first week, just don't make any excuses or invent reasons to skip it just this once. The fact that you care enough to ask slashdot suggests that it is important enough to you to make the effort. If you can make the change and stick to it you will probably find more fulfillment in your work.
Unexpect the expected!
I like writing software. In fact, I revel in it. However, one thing has always kept me back from being able to write the best software I possibly can: testing.
It is part of your job... testing that is. Just do it. Not every part of your job is supposed to be tantalizing or pleasant. If it were, it wouldn't be a job. It's just part of what a job is. And doing those things that are necessary (even if they suck) is part of being professional. Testing does suck balls sometimes, but hey, if something sucks, do it more until it stops sucking (either because you became proficient or numb.)
I consider testing to be the absolute bane of my existence.
Your outlook on things certainly won't help in making things better, will it?
It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it.
In that case, please do the world a favor and stop writing code. People who do this kind of shit is the type of people who say "done" even though all they have done is to check if their code compile and works with the happy cases. People like that are the bane of proper software engineering. By not doing this crucial part of your job, you are being unprofessional and unethical, and you should return a fraction of every paycheck proportional to the time you should have invested in testing.
Before looking for making things interesting, you should try having some work ethics.
This Ask Slashdot was very informative. I'm serious. I liked this one. It's a good question.
Soulskill +1 : Relevant ; +1 : Motivational
Someone flopped a steamer in the gene pool.
Do your unit testing, but otherwise: Hire a QA engineer. If you test your own stuff you won't see the mistakes.
Shoes for Industry. Shoes for the Dead.
"I know I'm not that lazy, as I can spend hours on end writing software, but there's something about testing that makes my mind constantly want to wander off and think about something else."
Well, actually, that is kind of lazy :-)
Seriously: you do the parts you enjoy and you skip the parts you don't. That, my friend, is lazy. Or at least undisciplined. And if you are getting paid to do it ... well, that's just unprofessional.
I'm not trying to be a jerk, it's just that I have been at this for 15 years, and I constantly run into people who don't bother to test. It pays off in the long run, it makes for better software, and it makes you a better developer ... you just have to have some professional pride and buckle down. I mean you know that you need to, you know it would make better software, and yet you can't seem to find the interest/sel-interest/professional pride/whatever to do it? Don't you see a problem there?
"Doubt your doubts and believe your beliefs." -- Switchfoot, Ode to Chin
I used to dislike testing until I learn how to implement code designed to be tested. Use a dependency injection frame work (that will keep you busy for a while) and write testable code. Writing elegant, readable code which scalable and testable is not an easy or boring task. If you can not automate the tests, you are probably do something wrong.
M0571y H@rml355.
The reality is that all software ships with bugs. Some known and some unknown. Typically it depends on how easy it is for the customer to find, what is the impact to the customer, cost to fix, and risk of regression. Given that software is typically patched after shipping, it means even more bugs get shipped rather than slipping the ship date.
"Here's how it works..."
That is, if you can write a test that looks like that. I work on compilers, so the trick is to write the example that tweaks just the feature that I want to check, and not 17 others. But, my test cases look like squirrelly little programs that could actually be inputs from especially peculiar users.
You ought to also think about demonstrating your error cases. "If you do this wrong thing, it says...." I am sort of a nut about error messages, partly because I once read the Apple Human Interface guidelines and thought they were a good thing. In particular, does the error message provide the user with information that will help him make the error message go away? When you decide that an error has occurred, what special information do you have that might help? My anti-favorite error message was "Bad SOCKS version number". Not "unexpected", not "too old and not-supported", but "BAD". Morally wrong, so unspeakable that not only can the BAD number not be named, even the good numbers cannot be named, because they might therefore name the BAD number by process of elimination. Truly unspeakable, the version number that shall not be named.
You come off and leave and let someone else raise the child..
Seriously, someone who cannot test his own code cannot write good code.
I noticed this many times even after Microsoft figured they can never possibly test all possible uses of Windows and kinda gave away Windows 7 beta. Think about it, the Microsoft as we know gives away free, usable, fully functioning OS knowing some people will hack the "time bomb".
Lets talk about OS X. High level developer merely changes wallpaper to something cool (they don't even do it), installs XCode, all OS X updates and codes. There is no resident software, no UI hacks (thanks to Apple, you gotta hack to have themes), nothing hogging CPU every 5 min, all files in place, no kind of insane settings (I even seen one sharing entire home folder to everyone). It is not a "test" machine, it is a Developer machine.
As guy doesn't mention what kind of software he writes, I can only assume things but it is the general problem. End user machines are overlooked. For example, nobody says "lets go to macupdate.com, download and install top 10 software and see how my software gets along with them", including Apple it is :)
For testing popular software, you need end users and "general" types, not high technical types. You gotta be nice to them yourself and make sure _everyone_ on IRC channel, forums, bug report replies are nice. I would even kickban the first ever so-called "supporter" who treats others like shit. That is one thing makes me really stay away from testing many open source software let alone using them. You know, you gotta kickban the first genius who seriously says "Where is the patch?" to already frustrated user. If they did it, many open source software would be in different place now.
It's inherently boring....
SO, Build hooks into the 'ware as you write it, and automate the testing.
Work smart, not hard.
Red
Actually it's just the opposite. It's like working in a candy factory and quickly getting completely bored/sick of the candy. I did one site where we were having problems with certain videos in different browsers and I swear to God that there was one clip of some really hot anal action that I got *really* sick of. After about the 3rd time I would watch about 5 seconds of it, see if the problem was manifesting, and then be back in the editor.
Now I watch whale documentaries to get my jollies. ... Wait! What?
.. your genitalia?
Pride.
Never happened. True story.
Do you enjoy finding the weak points in things? Do you use new devices, maps, etc without reading the instructions because you just think you should just know how they should work? Do you find that alarm bells go off in your head when you read a phrase that could possibly interpreted in more than one way? Do you often use things in ways that they were never intended yet those ways seem the most logical to you? Do you possess powers of intuition that lead you unforseen vulnerabilities? Do you find the needle in the haystack? Coding isn't inherently more interesting than testing. Anyway, if you identify with any of these phrases, you can apply it to testing...
Unit tests should excercise the corner cases in your code. If you know what they are, write tests for them.
QA testing should break all design assumptions about how the software should be used. Having the programmer sitting there telling the QA guy what to click on (and I've seen that far too often) invalidates that. The most useful bugs are the ones where the QA guy says "I did what I thought would get the job done, and instead it formatted my hard drive", leaving the dev to sputter "but, but, you're not supposed to do that". Given enough users, every possible "stupid" thing you can do with your software will be done in the field, and you really want to know that you will at least fail safely in all those "but that's not how you're supposed to do it" cases.
Socialism: a lie told by totalitarians and believed by fools.
First off, try to avoid testing your own work. If your company won't hire proper QA staff (which is stupid of them, but never mind that), then trade off with a co-worker: you'll test his stuff, he tests yours.
Now, your goal is to seek out the flaws in whatever you're testing. Be experimental - try doing things that are stupid and/or malicious. If you see a text input box, try putting in nothing, or putting in "Robert'); DROP TABLE Students; --", or putting in huge numbers of copies of the letter "A", or tossing in some Unicode surprises to see what happens. If you see a radio button, try every combination you can think of. If you see a date field, try to screw it up with different formats or something that isn't date info at all. You're doing the equivalent of taking a sledgehammer and whacking it as hard as you can manage to make sure that you don't do any damage, so have fun with it.
That said, the best testers are the most anal-retentive people I've ever found working in a tech-related field. They will work out exactly what all the cases are, and what the desired behavior is in each case, and if they don't get exactly what they're expecting they inform you of exactly what they did and what they expected. The best developers, on the other hand, tend to be too focused on the next great new thing to get excited about Test Case 1042-B.
I am officially gone from
Read Slashdot instead.
Pop some Ritalin so the person in question 'chills out' and can focus? Or as an alternative to red bull?
Ritalin is actually an upper (to those who aren't afflicted with ADD/ADHD, which I imagine most developers aren't) and would be counter productive in most situations. Unless you are going with the idea many soccer moms do, so exhausted they need something to keep them going, so they hit up little Billy's stash of uppers they don't actually need.
This is not the funny you're looking for.
Why do you hate testing?
Is it because you're goal oriented (meaning you can't get motivated to spend time writing unit test code)? If so you need to do a little growing up and realise that ALL untested code is garbage.
Or is it because you're doing it by hand (which means it's boring and laborious)? In which case the solution is obivous - you like coding, so write the code to do the job for you.
Either way you need to adjust your attitude or find a different job. A coder saying they dont' test is like a pilot saying he can't be bothered with paper work or a landing checklist.
These posts express my own personal views, not those of my employer
My comment will probably not apply to the coding you are doing: it's an old-fashioned, 50-year-old view of programming. Software starts with a statement of what it's supposed to do, in English and without any technical jargon. Ideally, the users of the software participate in the process; sometimes the marketing department proxy for the user. The next step is to determine the solution, again in English without a bunch of jargon. You break the solution into smaller pieces, then divide again, until you have nice compact functional descriptions of the building blocks. For each building block, you define the input and output associated with that building block, plus any saved state. Once you have those building blocks defined, you can write test cases that supply test input, check test output, and determine if the state is where it should be. THEN you write the code to actually do it, checking at intervals to see if your test cases run properly. Once the building blocks all work, you start assembling the building blocked into the larger blocks -- you wrote test cases for the larger blocks, right? Lather, rinse, repeat, until you have put the whole thing together. Then you debug the initial specification, identifying corner cases and adjusting the solution to take care of them. Once your solution is correct according to the high-level test cases, it's ready for beta testing. In other words, you do the boring parts first, then start being a wizard. Most importantly, though, you can show your wizardry works at each step of the way.
Thanks for the job! I work in SW QA at (big company).
There is NOTHING that will make manual testing suck less. It just does. I have written many a test plan for the manual test monkeys (both stateside and offshore) and I have performed many a manual test. It just sucks. There is no way around it.
I suggest heavy stimulant use or psychoactive drugs (warning, your employer might not be as friendly to my suggestions).
Step 1 - Write a small story on how the user will use the software.
Step 2 - Execute the steps in the story.
Step 3 - Find defects. Resolve them
Step 4 - Repeat and rinse.
Step 5 Helps document your system for future use
E.g. (for a simple UI driven system. Might not work for all types of software)
Step 1. John clicks on profile page
Step 2. John changes address
Step 3. John changes email ID
Step 4. John saves profile.
Step 5. New address and email ID is saved in database and displays on screen.
O this learning! What a thing it is - William Shakespeare
What? Am I the only one that drinks a shot for each bug I find? I keep trying to get a job at Microsoft so I can have myself a real party.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Often times, particularly in boring java web apps, the actual product code is less exciting than the unit tests I write to test them. Example:
A product needed to limit the number of events that were fired given some maximum rate of events/minute. The events can be fired from many threads simultaneously, but all threads contribute to the same calculated rate that must be monitored. The "RateQueue" as I called it was actually kind of neat, very compact and uses java atomic variables for concurrency. But once it was written, what's a good way to test it?
I wrote a little junit harness that can spawn threads initiated with a string that encodes a sequence of event firings, and each character of the string represents some configurable number of seconds. So a thread with "---x---x---------xxx-------" only fires 5 events in the time frame given. Now testing is fun, because I can create a bunch of different event profiles, and by visually comparing them in a text editor I can predict where the code should throttle the events, and then test for these conditions based on combinations of profiles. Even cooler, I can make the event profiles fill arbitrarily large time spans by compounding them fractally, where each "-" above would be replaced with a full string of "-", and each "x" would be replaced with the original string, and so on. Fun!
while [ 1 ]; do echo -n -e "\xe2\x95\xb$((($RANDOM&1)+1))"; done
Lest you think I'm a Ruby fanboy and dismiss me out-of-hand, try this or at least read about Behavior-Driven Development (BDD), as opposed to Test-Driven Development (TDD).
Then, learn Ruby and some of the common testing methodologies, like Shoulda, Cucumber, mocks, and RSpec.
Whatever language, OS, and framework you use, you just might change how you look at non-automated tests.
This is the only way I know to test my code... however painful, the advantage of the approach is that the stupidity of the developer gets known only among the two of me.
If you cannot get a split personality, then I don't think you are the appropriate person to test your code; even more, I think that testing your code is the most harmful for the future of the code... like any child, it might be OK and have a life, but will never be a great code.
Questions raise, answers kill. Raise questions to stay alive.
I know I'm not that lazy, as I can spend hours on end writing software,
Maybe you should go testing software for a while and pick up some skills... Doing the fun thing is always easy.
Everyone says you should unit test, and it's a good idea, but aside from that, what I try to do is write code in small pieces, and then compile and test. I try to not write more than 100 lines without being able to test it. You can't always do it, but when you can write in small pieces and test often, it does help eliminate bugs.
Qxe4
It's not boring when you think of all the bullshit and wasted time you'll avoid by testing properly from the start.
Adderal or Ritalin...
Or, you could turn testing into a drinking game!
No, I will not work for your startup
As a jr. level sw engineer, I abhorred testing. But I wrote the code, made sure it worked and moved on to the next feature. I was merely testing features as I went, until I made a change to the code that affected the code in more than one place.. then I was hunting bugs down everywhere because I had a function that was modified ever so slightly and it broke every thing that relied on one corner case.
Then a few years ago, I got my first taste of XP (Extreme programming) It was a completely new experience and I hated it. Agile, pair programming, writing unit tests... all new things. I no longer do XP, but I am better for having done it. What I learned was my test cases are insurance against future breakage. What I have discovered is that those unit tests are invaluable. A few weeks ago, I had to modify a core function because I identified a timing assumption that won't be a valid assumption in the next release of the code. I changed two core functions, added another, and could only pass it based on limited testing because there were no unit tests. If I had unit tests, I could see that prior concerns would pass or fail. Because that's what unit tests are: they are the validation of assumption and concerns. Any tricky code path can be replicated in a test case, so you know it still works (or doesn't). But you need that unit test infrastructure set up.
However system level testing - beyond engineering test - needs to be done by a non-engineering team, and a that team can only have minimal contact. Else leakage of assumptions and what not will happen. You want your testing team to be most user-like, but smart enough to break your code and report on your defects.
Unit testing doesn't have to suck. there Are frameworks out there. NUnit (.Net), and QtTest (C++) make it easy. (The Qt stuff is trivial)
There are two additional benefits of having good unit test infrastructure:
1. it can also be automated. Your nightly build system can also run the tests after building.
2. writing unit-testable code forces you to not put everything into GUI event handlers, but rather, develop an independent controller (as in MVC) which makes abstracting the interface easier if say, you want a local and web GUI. It also serves to keep the GUI issues out of the actual application logic. And that means, you'll get a script-able version of your application... not just for testing!
So has any of this made it more "fun"? No, but it has made me more confident about making changes. And it's made my code better. And it has shortened our time to market. (Fewer bugs make it out of engineering, system testing cycle is shorter, but oddly, we're not spending much more time in engineering (beyond getting the testing infrastructure set up)
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
I used to work in a group that tested each other's code. It was great. The best times I ever had as a programmer was in coming up with fiendish tests that would reduce a colleague's code to a smoldering pile of ash. Testing your own stuff is boring and you're bound to have blind spots to what to test since you wrote the code in the first place. Make it a challenge where you win if you kill somebody else's code, though, and it becomes a fun game and highly motivating.
... Example: someone asks me to code up something in OpenMP to do XYZ efficiently, on a platform with 256G memory. My first effort *should* work, but it keeps blowing out memory. So I code versions 2, 3, 4, and 5. In the meantime the requirements have changed, because the person I'm working for is also a scientist, and she's learned something new, and would like slightly different capabilities. I finally get something that appears to work, and then I wrote a few system-level test....
So, the history of humanity could end with: On 24th of July, 2023: Testing Device A147 achieved Singularity status. It quickly reprocessed in a sustained manner unknown to humans the entire mass of Earth. Humanity ended and a new node was born?
Shh.
Software development is a profession, treat it as such. You're paid to do a good job, so do it. If you can't find satisfaction in testing the code, find satisfaction in knowing that you've completed a quality piece of software (which includes testing and documentation). You'll enjoy your job a lot more if you approach it like this, and others will recognise the effort.
I use automated script systematically to test every features I wrote, making the job much easier.
And I do it immediately, or even while writing it, while it's fresh in my mind and usually easy to do, checking for normal and abnormal conditions. The script test for new and also the old functions, making sure everything is still OK.
Being proud of my code is a good motivator to ensure that no bugs are found.
Testing or any other way to make sure your code works correctly when it's delivered is boring - a bit like washing dishes or cleaning up your room. I can't see many ways around that or make it more stimulating. Testing becomes an attractive option only if the consequences of bugs and maintenance are more painful than the test themselves.
Lucky you, that you work in an environment, where you can get away with delivering untested stuff - savour it while you can.
In case you need to deliver better quality, I found that testing early and testing often makes testing almost bearable. Write some small functional unit - test it and don't move on to the next thing before it was tested successfully. It's a bit like tidying up after you when finished with one thing before getting out the next toys to play something else. It's not fun, but it makes the whole testing bearable because it goes fast enough every time. You will seem to go a little slower producing code, but the it will have the advantage, that it works.
Have someone else test it, preferably someone who doesn't know much about how it 'should' work. You do this for the same reason you have someone else proofread what you write: if you expect something, you're more likely to see that rather than what's really there.
Cultivate some beta testers who'll bash on your code for cheap. As in beer.
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
Then you have hookers killing themselves from boredom.
Although... trying to explain away the dead hookers would make debugging a lot more interesting.
Democrats or Republicans. They are both taking us to the same place and they are not afraid of us anymore.
You should be exposed to the full ridicule of users who are frustrated with the errors you've foisted upon them. Or do what I do, use testing as a way to avoid attending to writing coherent documentation.
Keep Doing Good.
I have created a connection with the company sales force. Whenever I introduce a new product feature, I invite one of the sales people to try it out - and I watch them like a hawk (but without commenting). Sales people are generally one step DOWN from real users, so if you can make it work for sales people, you are in a pretty good state. Sales people also have a real hunger for new things they can use to sell, so they are eager to get involved.
This approach removes much of the detail testing - all you need is to make sure you can actually demo what you have done successfully, then let the sales team pull it apart.
A connection with the sales force is also useful on another level. I often go along on pre-sales calls and listen to the customer's comments and watch the sales people demo our products. It gives me loads and loads of input and ideas for how to improve our UIs and workflow. Off topic I know, but something I highly recommend.
Meus subcriptio est nocens Latin quoniam bardus populus reputo is sanus callidus
Testing is a big subject, my take was to write a full automated test, one that puts the database in a zero state and do as many business scenarios it can, simulate the pass of time, simulate external inputs in the most realistic fashion so you don't have any surprise in the deployment.
The Test Team can use the tools developed to speed up their job(put the system in X state can be made by your automated test framework). they test manually all the system too, specially the UI and report bugs,
Designing the system so you can easily test and replicate states, developing the tools and putting it all together with the team was a task i found stimulating.
If you have a problem forcing yourself to do the non-automated 'tests' then you need to consider that the problem is that you are doing non-automated tests.
There are three types of tests in computer software:
Unit testing (automated)
UI testing (manually done by people OTHER than yourself to throw the unexpected at it)
and the third ...
bad manual tests that should have been automated.
Theres no such thing as non-automated testing outside the user interface aspect. I program cause I like to solve problems, the easiest way to motivate myself to test is simply turning the tests into another challenge. Figure out how to automate tests because there isn't anything you can't automate.
One of my current main projects is an MS Outlook plugin that modifies messages as they are sent, and does special things to replies from other users of this software.
To write the testing for that, I had to come up with a way to automate the GUI interactions with outlook, simulate both an SMTP and POP3 server to facilitate getting test data in and out of Outlook in a way it expects and to verify the plugin is doing what its supposed to do.
The only testing that isn't automated is the kind were you're determining if people actually LIKE using your application and to determine if what you've done to the UI actually make sense to people who don't write code for a living and know exactly how it works.
Finally, just do it. Its part of being a professional. It sucks, its the shitty part of the job, but its part of the job. You either take both and do them well or you become the boss and delegate both coding and testing to someone who can do it.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
Try not writing boring software?
...situated in your chair seat. Each failed test increases the voltage exponentially.
They really suck at test GUIs or anything visual
Then make sure your application is accessible to people using a screen reader, and then write the automated tests using your operating system's accessibility API.
First off, don't do non-automated testing. It's unnecessary.
How is non-automated testing unnecessary in the following cases?
Here's a tip for you developers developers developers. Test together, get some buddies to test your stuff together over a beer. Chat along. It's fun, almost like a game of chess or anything. Sometimes it gets boring, but still fun because of the company.
^_^ Group hugs?
Each bug you find, you take a shot. This makes finding bugs more interesting and gives you motivation to find them. After a few bugs you will hit the Ballmer Peak (http://www.xkcd.com/323/) but any more after that will cause problems...
Use sikuli to test guis. Or at least to do gui unit testing, sikuli can be applied to websites, webapplications, traditional desktop applications etc, heck even command-line. We even use sikuli at work to do system upkeep, i.e. clearing out junk from the system.
What do you mean, non-automated testing?
Injecting random, but smart, inputs and automatically checking the outcome is much more stimulating than the boring stuff you are talking about. This will stimulate your mind and your code much more than going over a list of events one by one.
In my experience around 80% of the bugs that my random tests detect happen in a combination of events that I would have never thought of testing. I also make sure that the simple events, a tester would think of, would have a reasonable chance of being tested.
So, what's the magic? Pure random testing does not work, it has almost zero chance of hitting bugs. Instead, think of corner cases in your input and give them a higher probability (e.g. zero sized string, MAX_INT, MAX_INT-1 and so on). Finally, generate different tests with a different "entropy" otherwise you are testing the same thing over and over again. This "entropy" thing is tricky to explain and to notice. For example, if your input is a bool[1000] and you randomize your elements independently (half True and half False) then, practically, you'd have a zero chance of getting 1000 elements of the same value (which could trigger something interesting)
The final thing is to check your outputs by validating invariants in your code, either both from inside by using assertions and from outside, finding good invariants is like a riddle. The details are too complicated for one post.
Sure you don't want to write the test because you are done with that stuff and you want to move on.
If you write the test first then you don't end up in this situation.
The test will be a beacon on the horizon to shoot for, and a validation of your specification (you DID write a specification?!?)
Put a bottle of something fun on the shelf, open it when the test passes. Motivation!
Game logic works. Anything that needs to be repeated 50 or 100 times suddenly becomes a goal when you attach an achievement to it. Make it social and people will start to complete on testing your software :D
If it's done properly with enough feedback for the user and enough obscure achievements, it will work.
Actually I think this is something Microsoft should adapt for the future versions of Windows ..
Etc ;)
GeoKone.NET
You answered your own question: Automate it.
If you can't automate it, this is likely because your presentation layer and logic layer are not properly separated- which calls for refactoring (although it's easier and far less time-consuming to do it right from the start). Depending on how picky you are, your program code should ideally be completely controllable by your test code.
There are more reasons to do this than testing alone. It will allow you to create alternative interfaces for your program. A command-line interface will make it possible to script any task that your program can perform; a completely stand-alone user interface will make it possible to provide different skins for your program; or text-mode menu-driven interfaces, or user interfaces for the visually impaired.
Surely this whole world of extra possibilities is more motivating than manual testing? (Which, as other posters pointed out, should be done by someone else *anyway*...)
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book
I thought you said you like developing software.
Well, not testing software is doing a half-ass job.
Don't you take any pride in what you create?
I too can't get myself to follow practices such as test-first, but I found a method that is not as ambitious, yet guarantees robust code and only few errors. Whenever I write a method, I think about what I am assuming at runtime and codify these assumptions as pre- and post-conditions at the beginning and the end of a method. These assumptions can be quite complete and I learned that I implicitly make them when writing code anyway, all I do is to write them out as assert statements. Then I devise a set of component tests that trigger different call-flows, hoping to cover a wide variety of states.
It really provides a great sense of certainty and I learned that I can skip a lot of error handling in the method by just asserting a fact. When I am confident that a function does what it is supposed to do, I recompile without debugging and the preprocessor will drop the asserts. I have made good experiences with this approach and certainty that a method does what I think it does far outweighs the additional overhead of writing the post- and pre-conditions and provides the means to keep your internal model and the actual model of the code aligned.
something clever to make me stand out!
Drugs.
Alternatively if you are not into illicit substances, try beer.
Might not help you find all your bugs, but at least the whole experience might be more interesting.
Admit nothing. Deny Everything. Make Counter-accusations.
If the prospect of producing the best software you can and testing it isn't stimulating enough, why not take up lion-taming?
To have a right to do a thing is not at all the same as to be right in doing it
I find that can make a lot of things much more entertaining - it is the way it makes patterns of colour wander across the screen and everything else. Failing that, perhaps read on below.
It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else, or even worse, if I absolutely have to test, I do a very poor job at it. I know I'm not that lazy, as I can spend hours on end writing software, but there's something about testing that makes my mind constantly want to wander off and think about something else. Does anyone have any tips on how I can make non-automated testing a little bit more stimulating so I can at least begin to form a habit of doing so?
I can sympathise with that entirely; but I think it is a matter of attitude - any task can be seen either as a chore or an interesting challenge. If you start on a job expecting yourself to do badly, then of course you will; But think of it like this: when you write a program, you have to plan how you will handle ranges of input values, how to react to this or that event etc - this is no different from planning how to test something, and if you think about how you can test your code afterwards from the very beginning, you can make that part easier.
When I start on a biggish application, I identify the basic bits of functionality, which I implement in the form of simple command line tools. At the end I join them together in a graphical interface that is little more than a frontend to call commandline utilities. Why? Because it is easy to make automatic test scripts for each of the commandline programs, and it is easy to predict what the outcome of each test should be.
It helps if you make notes on the way, as if you were going to teach somebody else via email; things like "The reason I do it this way is ...", and "when you do like this, it should react like that". You have to do this while you are writing the code, because that is when you have this hot idea in your mind.
System testers have to know what the corner cases are. They can't guess them all
That's why you need proper documentation, like use cases, technical designs. Often (and preferably) written by analists and not the developers.
Preferably you have a setup like this:
Business analyst writes documentation based on requirements from the business.
Developers build the application, based on the documentation.
Testers write testcases based on the documentation and test the software as soon as it is released to them.
Testing is a profession too.
And there are many tools and methodologies (TMap, ISTQB, Testframe etc) to ensure proper test-coverage and to have anything meaningful to say about the quality of the tested application.
80 CC D8 AF AE D3 AB 54 B7 2E CE 67 C7
Every piece of code you write, you basically test already. It is just how do you test it works in the first place?
I used to run the software and test it directly after writing. This is a bad idea, much better idea is to write the unit test as you write the actual code instead of starting the software, this way you actually end up being more productive and not starting the software and navigating it all the time. There are of course some pieces of code that is hard or almost impossible to write unit test on some platforms, usually the UI behavior and such.
From a professional tester point of view, let someone else do it. It's wise to do a developer test but actual functional testing is best done if someone else does it (a dedicated functional tester for instance). Firstly, that other person has a different perspective of the piece of software because it reads the requirements and hasn't seen the development process you as a developer did (the functional tester will press buttons differently). ;)
Secondly, you don't have to spend lots of time figuring out how and what to test. Which is more efficient as you can spend that time on developing other software. :)
More about structured testing here:
http://www.ie.sogeti.com/Resources--Downloads/Methodologies/TMap-Test-Management-Approach/
Software quality becomes more important everyday and only just recently more companies are hiring dedicated functional testers to test that, which you find a tedious task. :)
So you like programming more than testing? Well, write a program that generates and executes the tests. You'll get to do some more programming. Writing code that iterates through tests systematically can be quite a challenge. For example, it's not easy to write code that systematically generates a lot of varied, syntactically and semantically valid C++ programs for testing a C++ compiler. (Of course you'll also want to do the easier task of generating a lot of invalid programs if you're dong that).
I took a look at how much work I had to do after coding without tests, and how much I had to do with proper tests. I quickly realized that not only was coding with tests faster (in the long run) but it was a lot more fun.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
Write the software for a programmable dildo, and test it on yourself. It will stimulate you all the way to release :-)
Implement test pads in your code, and automate all the testing. Manual tests are pretty much a joke anyhow, since people make mistakes. A test which isn't automated is 1) a drain on your soul 2) irreproducible and 3) a loss of long-term value.
-I like my women like I like my tea: green-
If you write software that you personally use, that makes some testing (smoke-tests, for example) really easy. At least for the use cases that you are actively using.
"but, but, you're not supposed to do that".
Yeah, that one still makes my *hair* stand up. Most devs are superb at narrow domain problem solving. Of course, reality and users tend not to accommodate them. I can't tell you how many times I've pushed the limits of a software package to get something done and it fails miserably.
So, yes, they will have 37 instances of the software running at once and yes, they will try and save one or more projects/files with the same name at the same time and yes, no matter how many times dev says it's their fault, they're wrong. Machines are designed to accommodate people, not the other way around.
Please do not read this sig. Thank you.
You're new here, aren't you?
When persuade a Beowulf cluster of petrified Natalie Portmans to pour hot grits down your pants, then maybe we'll review your advice.
BTW: posting as Anonymous Coward does more to lower the overall level of this site.
Can we get a "-1 Wrong" moderation option?
Fine yourself a thousand dollars for every bug you ship, ten thousand for every BSOD a customers sees. Each time.
Hire an old programmer without a job to scream obscenities at you for shipping rip-off jack*** software.
Grow up.
``Tension, apprehension & dissension have begun!'' - Duffy Wyg&, in Alfred Bester's _The Demolished Man_
I myself am in the midst of QA of software (web site) I've spent the last year writing. It's all well unit-tested but there's plenty more integration testing of the new functionality and worse, regression testing. Ugh.
There's no one else available to test (well, there's another dev in the same boat as me). I firmly believe that I'm the worst tester of my own code for all the reasons already stated. Other Dev and I will test each other's code to the extent possible, but that will help only so much.
I have a plan for today: I'll find a good podcast or something, one that will demand my attention (maybe In Our Time). I'll try to find a sweet spot where I'm giving 5-10% of my attention to my web site interactions. I figure that if I make sure to simply continue interacting--even if only randomly--I can approximate the mind set of the typical user. In the meantime I might learn something from the podcast or at least be somewhat entertained. It's win-win.
I'll let you know how it goes.
There's a line from the book "The Art of Software Testing" from Glenford Myers that has stuck in my head: "A successful test finds bugs".
In other words, you redefine success as "I found a bug". After all, if you test and don't find a bug, all you've done is waste time. You haven't improved the product. You haven't "proved correctness". The only way to improve the program is to find and fix bugs.
Since we're all driven to be successful, once you get into the mindset of "success is finding bugs" you will be astonished at how many bugs you find.
ADD here.
to those who aren't afflicted with ADD/ADHD, which I imagine most developers aren't
I have to say you are exactly right that I could never be a coder full time. I took some classes, aced them (well, the tests anyway), but the work is way too boring, worse, the teams in college were like 3 people who each said "oh, I'll code the about screen..." and me who made a functional DB front end for some class DB. Testing however, was the worst of it all.
How much is your data worth? Back it up now.
... by doing more analysis and design before you write the software. Automate as much of you software processes as possible. If you can model your analysis, test the analysis models, and generate the analysis models directly into code, you will eliminate a large amount of test and development time. You also get to avoid the mind-numbing repetition involved in a lot of the aspects of every-day 3GL coding life. (Some of which can also be avoided by developing good reuse libraries, but not all.)
The post states that the author knows how to test and does attempt to test. (his mind wanders off during the process) I believe the answer is to improve or work on your cognitive abilities. (meta thinking if you will)
Tony Robbins has some easy to listen to motivational material that could help. Focusing on the task at hand without becoming distracted is the key.
Try testing other peoples code and when you come across a very obvious bug and you cannot understand how the programmer missed it, you will find more motivation to test.
Nothing can take the place of QA testing but I find that performing my own simplistic tests saves everyone involved time.
If you are testing your own code, you're doing it wrong.
You said it best:
It is so boring and un-stimulating that I usually skip it entirely, pass the testing off to someone else...
My work has an entire department of software test engineers, many of whom can't write a line of code (non-automated testing, obviously). The best testers we have are the ones who think like users and aren't biased by their inner-geekness.
I'm pretty sure authors don't proof their own work, so why would it be different for people who write thousands of lines of hard to read text on a computer screen for 8-12 hours a day?
leaving the dev to sputter "but, but, you're not supposed to do that".
My favorite quote on this subject: "If there is a button there, somebody is going to push it." Over and over again, I have been surprised by users who have put normally safe options together in ways that result in disaster. As a developer, it is my responsibility to make sure that the self destruct button is NOT right next to the refresh button, and that the self-destruct button has an obvious warning message before the countdown starts. Even though user insanity is not something you can test for, you can minimize its impact with an effective user interface design.
Unit testing has saved my ass numerous times. One test in particular pumps a bunch of random data into the system and writes out every possible output format with said random data. Garbage in, garbage out, but its repeatable garbage, and each release establishes a baseline for these output files. Prior to the next release, I repeat the test, and diff the output against the baseline. Any changes that can't be attributed to fixes in the new release are bugs that need to be tracked down. This test is good for catching problems from small code changes that ripple to unintended places.
I am not your blowing wind, I am the lightning.
Uh, you define the requirement.
Given the placement of objects in a Super Mario Bros. level, and the behaviors of those objects, how would I go about automatically assessing the difficulty of directing a character through the level?
Most code in a simulation is *not* random, it's fed inputs from other parts of the code, which ultimately result in the behaviour you describe.
Low-level code in a simulation is testable. But at a certain stage, the mock framework can be substantially bigger than the piece of code that you're testing. What is a typical ratio between product lines of code and test framework lines of code?
Second, the implicit assumption is that there is *some* way to determine, based on inputs, if the outputs of your software are correct.
For some outputs, there exists a way to demonstrate that the output is correct. For some outputs, there exists a way to demonstrate that the output is incorrect. For other outputs, the question is intractable. For example, consider a random level generator for Super Mario Bros. In this case, levels outside the player's chosen difficulty range are incorrect.
Dishwasha wrote:
First off, don't do non-automated testing. It's unnecessary.
Abcd1234 wrote:
Third, realize automated testing isn't the end-all and be-all of testing.
Then you appear to disagree with Dishwasha.
Beg the question much? If your test suite is missing things, fill in the gaps.
Acceptance testing helps find gaps to fill.
How do I write code to test the test suite itself for coverage and correctness?
You should ask this on Stack Overflow, or better yet, see if there's an existing question that maps closely to this one and engage in the discussion there. Its less full of howler monkeys and asshats than Slashdot, and generally gets more useful and serious answers if your question is programming related.
Jherico
What can the average user can do to ensure his security? "Nothing, you're screwed"
"but, but, you're not supposed to do that".
Yeah, that one still makes my *hair* stand up.
Still one of my favorite moments, 3x years later, is the stunned sputter from the summer student (who'd written a unix program to interpret an octal dump from 1ESS error messages) when, on my first run of it, I redirected a unix directory into it and it crashed immediately...
His introduction to diabolical testing.
That's one way to make testing fun.
You can entertain yourself by putting in some cheesy data (but don't tread into material that might be offensive..). If you're working with some other testers, encourage them to do the same and your life will be a hell of a lot less boring. I.E. if you're working on a forum or something, make a post "I put on my wizard robe and hat" (just saw that a few posts ago, almost made me spill my coffee). I do a significant amount of testing at my job, and even though I'm often the only one testing, I still laugh sometimes just thinking of what someone else might think if they saw this or that
I work on some software where the acronym "cat" is sometimes used and my project manager hates cats, and especially lolcats. So naturally I uploaded a picture of keyboard cat as the profile pic for the primary test user / admin account for the application (so every time they logged in, they saw that picture). At the next conference call one of the consultants said "Hey I really like that picture of the cat!" and another one half jokingly said "Hey, maybe we can use that pic as the company logo!". The PM rolled his eyes and said "..If I could shoot you all right now over the phone, I would..". Best conference call ever!
Global warming and other natural disasters are a direct effect of the shrinking number of pirates - Gospel of the FSM
As a developer, it is my responsibility to make sure that the self destruct button is NOT right next to the refresh button, and that the self-destruct button has an obvious warning message before the countdown starts.
The 'obvious' warning still doesn't work.
One of our major apps has a 3 level delete.
"If you delete this, it is gone forever" OK/Cancel
"Do you really want to delete this?" Y/N
"Do you really want to delete this? It will be gone forever." Y/N
Weekly, we get calls - 'Can you restore x, from 2 weeks ago?'
FML
I'm a veteran tester, and we all think we could write a book on the subject. Fortunately, O'Reilly wrote such a book already ('Beautiful Testing'). So I'll just make one other comment not yet covered:
Not all testing can be automated: I'm thinking of UI testing (or verifying all bells n' whistles are in place). It doesn't take long to check, and it's not even the most critical part of the testing process (functional UAT is, imo); but it's the first and last component of the app the end-user sees.
"He who questions training trains himself at asking questions." - The Sphinx, Mystery Men (1999)
So that when the app falls over at 3am, they can pinpoint the line of code and committer in the repository.
Imagine they have a list of people who caused them to receive phone calls at 3am, and they are sorting the list by number of calls. Imagine they have admin access to the human resources databases and know where all the developers live.
Imagine you've seen "Guns & Ammo" magazines and pirated copies of "Rampage" lying around in their offices.
Or... sharks with lasers.
Deleted
Find other people to find bugs in your stuff, you find bugs in their stuff.
Go fix,
repeat.
Testing your own software is a bit like trying to tickle yourself....
doesn't work so well.
Sure, I can test all the places where I know something might go wrong (but have already coded for), where I will miss testing are the places I never thought to look or write code for...
You can't know how much you don't know (and usually, the more people think they know is only representative of how much they don't begin to grasp the whole picture).
If you had signed your name, we could avoid hiring you to write any software.. Testing is about creating the highest quality software. If you have any pride in your work, you would want it to be bug free. I have known programmers who thought they were too good to test their own code. They were assholes. If debugging is the art of removing bugs, programming must be the art of putting them in. When people tell me they have found a bug in my software, I get all happy and thank them, because each bug found and corrected puts you that much closer to a perfect product. I don't think there is any magic bullet that makes debugging more fun, or entertaining. But producing quality software is rewarding in many ways.I can hardly believe you asked the question. I can hardly believe they thought the question was worthy of being on today's list.
Soul Skill,
Your question is so totally not an issue. Software testing protocols have been set up for decades now, and that makes them basically an ancient crossroads on a land bridge that existed before the Last Glacial Maximum. That happened sometime toward the end of the Pleistocene.
Go out, find a date with a human (man or woman), and chug some bottles of beer. Testing isn't the tyrant you make it; in fact, testing is something like the feeling that builds up just before a male orgasm: You know something's happening, but you're not entirely sure what may come out. Still, any excuse will serve a tyrant. I read that somewhere.
If you can't find a date, come to Punta Gorda. I'll make you crab cakes with mango chutney and an Old Fashioned, and we'll test the waters, which are easier to test than code. Otherwise you may become Soul's Kill, and we don't want that. We don't want that at all. So very messy.
Best,
Lune Keltkar
How do I make testing software more stimulating? Do you mean that you would find a way to connect it to a vibrator? Or do you mean How do I make the testing of software more stimulating? Here I would understand, how to relieve boredom.
Leslie Satenstein Montreal Quebec Canada
Yeah, I figured a job that requires a lot of abstract thought where you have to envision lots of steps/objects in your head might be problematic to someone with an attention deficit.
This is not the funny you're looking for.
For every bug you miss, you give back 1/10 of your monthly salary. That ought to make it interesting enough.
+++OK ATH
Automate it instead
JMeter
LoadRunner
*Unit
etc.
interesting ... seems the most common thread here is folks only looking at this as "QA" or GUI Test Monkeys ... anyone here ever heard of Quality Engineering ...???
... QE is always more predictive. if one takes an engineering approach to the test, verification, work-flow, user experience, and overall quality of the product, boredom just fades away. if one continues to look at the skills required as "post result" and not part of the whole, sure, i guess one can get bored ...
... this entire endeavor is far from boring, IMHO ... :)
QA is always more reactive
i've been in this field (QA & QE) for 20 years. i'm still not bored.
oh, and i write/develop/design software and hardware+software as well
hope you find your way!