How Do You Prove Software Testing Saves Money?

Design from the beginning is important too. by ls671 · 2011-01-02 20:44 · Score: 3, Insightful

I would say the most important thing to save money is a good design and programming guidelines for the application.

Test suites are obviously a good idea but it may end up as a tedious task if the application as poor or no real starting design and if everybody just did things their ways.

I have seen many applications that had grown larger than expected that typically cost 10 times more than it should to maintain, bug fix and implement change requests.

It is sometimes possible to fit existing code in a new clean design and that saves a lot of money if the application is going to be used for a long time.

Let's hope your application has proper design ! ;-)

--
Everything I write is lies, read between the lines.

Re:Design from the beginning is important too. by Kjella · 2011-01-02 21:54 · Score: 2

Then design wise, the thing to do would be for every problem found, see what can be re-factored and improved right there and then, and do it immediately (with test cases to prevent regression).
Well, in my experience it goes like this: You're looking to refactor the code because the code already is quirky. Some 99% of the time, that's just because it's sloppy coding, poor structure and logic or it is simply undefined or irrelevant, the corner cases are never reached, the function is always called with data of the correct format and so on. Then in 1% of the cases, the application depends on this quirky behavior and all in all works correctly - or at least works - until you clean the code. If you have to assume that everything that is there should be there, you end up with code almost as bad as what you started with. This is the problem trying to reverse engineer the intended behavior from existing code.
That is why you have to find some kind of border where you have pretty solid control on all the ways this code is called, it can be a module or a group of classes. Then you preserve the exterior while you build a new and clean interior, where you can safely remove the quirks because you're sure nothing depends on them. The kind of "micro-refactoring" that you seem to suggest he could do along with bug-squashing just doesn't have much effect in my experience. You need to have a certain scale to it before you get any real clean-up effect in the middle.

--
Live today, because you never know what tomorrow brings
Re:Design from the beginning is important too. by mcvos · 2011-01-02 22:11 · Score: 4, Insightful

Well, in my experience it goes like this: You're looking to refactor the code because the code already is quirky. Some 99% of the time, that's just because it's sloppy coding, poor structure and logic or it is simply undefined or irrelevant, the corner cases are never reached, the function is always called with data of the correct format and so on. Then in 1% of the cases, the application depends on this quirky behavior and all in all works correctly - or at least works - until you clean the code. If you have to assume that everything that is there should be there, you end up with code almost as bad as what you started with. This is the problem trying to reverse engineer the intended behavior from existing code.
1%? I think even if the code is buggy, it's quite likely that the code now depends on those bugs in various ways.
In any case, the right way to refactor is to make sure you've got proper tests for the code you're going to refactor. That way, after refactoring, you can check whether the functionality is still unchanged.
So yeah, the submitter has a problem. What he really needs from a code-quality point of few is pretty big and expensive. If there's no money for it, but there is money for continuing as before, then I guess that's the only option here.

First things first by Anrego · 2011-01-02 20:46 · Score: 5, Insightful

Can anyone offer suggestions for how to convince the owner that setting up a test suite is in his own best interest?

Are you sure that it will be?

but I'm sure that in the long run it would save time and money

How much time and how much money?

What you need here is hard numbers to back up your opinion. Specifically, and I hate to say the word, but you need metrics.

- How much do these bugs cost (these are a mixruee of hard numbers (dev time, etc) and "fuzzy" numbers (customer relations and so forth)?
- How much is setting up this test suite going to cost? How much is it going to cost to use it?
- How effective do you think it will be (look at existing bugs.. would they have been caught)?

That last one is especially important. Testing is good, but it doesn't solve all problems and there are other options. Really take a good look at whether previous bugs would have been found in your proposed system.

Once you have this information, the math then becomes pretty simple.

If your program is modular-ish .. it might be possible to do a trial. This is probably a good idea anyway. There are different ways to do testing, some of which are incompatible with certain development mindsets. Best to find that out early on a small piece vice the entire code base.

Re:First things first by Splab · 2011-01-02 22:42 · Score: 4, Insightful

Also, putting testing infrastructure into an existing system is a huge undertaking. And it requires a very strict handling for all future commits - if you just slip a little on the test suite you are basically invalidating the whole thing.
Many a times I've tried to implement test suites, both as request from employers and also for own sanity sake - and every time it has failed because the workload was simply too overwhelming to support both testing and development.
In the end, the basic economics of it has dictated that it is cheaper to hot fix than to make it right the first time around.
Re:First things first by arth1 · 2011-01-03 00:01 · Score: 2

Also, testing introduces bugs. Both because there will be bugs in the test code itself, and because of bias - even when not meaning to, new code will be skewed towards passing the test case, and the tests skewed towards validating the code instead of finding bugs. I've seen examples of bugs that wouldn't have existed if it wasn't for the testing.
Then there's time wasted on testing things that simply don't go wrong, or where the impact would be so huge that it doesn't require a test case - if it prevents the software from running in the first case. Or where the impact of the bug is so small that it doesn't really matter, and rewriting the code is too expensive compared to the theoretical risk of triggering the bug (like a system not allowing someone to be have a name with all digits, or a rounding error only showing up at speeds over 1000 mph).
Yes, testing is overall good for quality, but can increase costs quite substantially.
If the same bugs keep reappearing, perhaps the bug fixer could be better at commenting when fixing bugs -- not only in comment blocks above routines (because they can be missed), but inline comments about what was fixed, and more importantly why?
Re:First things first by balloonhead · 2011-01-03 00:19 · Score: 3, Funny

Why not just edit the splash screen with a big Google-esque 'Beta!" stamped over the application title? Problem solved.

--
This idea was invented by Shampoo.
Re:First things first by Glonoinha · 2011-01-03 00:38 · Score: 4, Insightful

In the end, the basic economics of it has dictated that it is cheaper to hot fix than to make it right the first time around.
What the OP failed to provide is the nature of the bugs his users are experiencing, the impact to the users and to the business, and the costs associated with the manifestation of those bugs.
If we're talking about user interface glitches where a button is not aligned with the others, the color of the button when pressed is wrong - then it is probably way cheaper to hot fix than to insure 100% quality the first pass.
If we're talking about workflow and data errors in a financial application with regulatory implications, or bugs that bring the system down for an hour and the company loses five figures in revenue for each hour of downtime, that's a different story.
Testing doesn't necessarily save money. Saves pride and credibility more often than not, but as others are saying it isn't necessarily the most fiscally prudent choice. Neither is a wholesale re-write of 10 year old code (even though that's our first inclination when we inherit code.)

--
Glonoinha the MebiByte Slayer
Re:First things first by Lumpy · 2011-01-03 01:00 · Score: 4, Insightful

Exactly, the cost of not testing may be a IT helpdesk position that can be eliminated by doing better software testing. It can also be loss of productivity, loss of work quality, etc....
Honestly, just because it's software does not make this hard, replace the word software with "business process" or "machine" and you have the exact same problems. Would a company buy a $250,000 machine without research or testing to see if it was the best fit for their solution? No.. yet many buy the crap Ticket system that is BMC Remedy for close to the same price just because everyone else does.
testing is not a IT process it's a business process. Lack of software testing is simply a indicator of a failure of the management in the company.

--
Do not look at laser with remaining good eye.
Re:First things first by rtaylor · 2011-01-03 01:21 · Score: 4, Informative

Incidentally, if anyone out there has suggestions on how to reliably test for race conditions, please speak up.
It's not easy but I have had good luck with pretty simple load generators and having the system put in random (from very long to zero) delays in the processes. Find lots of race conditions (short delays) and poor or missing interlocking (long delays).
The tricky part is making it replayable by recording the process step, random pieces of data used, and delays to a log which can be re-executed to prove the bug exists and prove the correction.
For really important systems I've run the random load generator for a month before sending the product out.

--
Rod Taylor
Re:First things first by theshowmecanuck · 2011-01-03 02:58 · Score: 2

It is my experience that test driven development is good in that it makes sure the code is unit tested. But unit tests do not take the place of user testing. What many people fail to take into consideration is that just because the application works doesn't mean it is doing what it is supposed to. For example, just because it displays the results of a financial calculation, doesn't mean the calculation is correct. Many times it requires someone with in depth knowledge of the business in order to make that determination. As much as developers like to think they are experts in a business, they rarely have the knowledge of someone with many years experience in it. Think of it like a poorly written SQL statement. It will likely return a result. It passes the unit test. But as we know about SQL... result sets from complex queries can be wrong or incomplete unless you validate the data. i.e. just because the code works, doesn't make it right... debugging logic is way harder than debugging syntax.

--
-- I ignore anonymous replies to my comments and postings.
Re:First things first by JustinOpinion · 2011-01-03 03:06 · Score: 2

Can anyone offer suggestions for how to convince the owner that setting up a test suite is in his own best interest?
Are you sure that it will be?
This is the key, I think.

There is a generally-held belief among coders that "doing it right, the first time" and "rewriting this mess" will save money in the long-term, and that managers are idiots for not seeing that. This can, of course, be true. But it isn't always true, and coders are sometimes projecting their OCD-desire to have nice code (and sometimes suffering from "I didn't write it so it must be crap" syndrome) and assuming that this will translate into the dollars and cents that the company cares about.

Sometimes it's worth it; sometimes it's really not.

The thing about money is that it is both non-linear (double the money doesn't necessarily have double the value; sometimes it has more than double because you can overcome barriers; sometimes it has less because of diminishing returns, etc.) and temporally varying (inflation, time-value-of-money, etc.). Because of this, it can actually make economic sense to do something in a half-baked way, and "pay the price" later on (in terms of higher support costs, or even having to totally re-do a task/project). For example, in cases where you "absolutely need it now" (the value of having it finished soon becomes larger than down-the-road problems) or because you can't spare the cash right now (the value of using that money to do something else right now is larger than the down-the-road problems). (If you want a physics analogy, notice that money is not a conservative force-field: it is a path-dependent process...)

I'm not saying that it always makes sense to do slipshod work now and suffer the consequences later. There are plenty of dumb managers who over-value short-term gains compared to long-term. But that doesn't mean that the optimal solution is to spend massive effort up-front; there is such a thing as being too much of a perfectionist. And, importantly, the right answer will vary wildly depending on circumstances and the current state of the business. A startup may need a product to show (anything!) in order to secure more money. Doing it "right" will mean bankruptcy, which is far worse than having to keep fixing and maintaining a piece of shoddy code for years to come. A very well-established company, on the other hand, may do serious damage to their reputation if they release something buggy; and can probably afford to delay a release.

Actually figuring out the cost/benefit is not simple. In principle this is what good managers and good accountants are there to do: to figure out how best to allocate the finite resources. If you think you've found a way to reduce costs by implementing testing, then by all means show them the data that supports your case. However don't assume that just because testing will make the product better, that it actually makes sense from a business perspective.
Re:First things first by jimrthy · 2011-01-03 03:37 · Score: 2

Rewriting from scratch is almost always the wrong thing to do.
Things You Should Never Do, Part I
Yeah, it's an "appeal to authority" argument. But it's been my experience that he's absolutely correct.
The only ones who win in this scenario are the consultants who are getting paid by the hour. Well, and your competitors.
I totally understand the urge when you inherit code from someone else. Heck, a lot of times, I feel the same urge when I go back to look at code I wrote 6 months ago. There's a reason that snarled-up nightmare was written the way it was. Make sure you understand that reason before you just scrap it and decide to start over.
Otherwise, you're just going to wind up with a newer snarled-up nightmare. It'll just have a bunch of bugs that were fixed (and left uncommented) in the existing version.

Why will it save money? by SuperKendall · 2011-01-02 20:57 · Score: 3, Insightful

You need to think very hard about exactly how any why this will save the business money. What do YOU think will be gained, for your software, from a technical standpoint?

Many people just think testing will make things better. But I have seen testing make things better, and I have also seen testing make software far harder to maintain and later to deliver.

What kinds of problems are you seeing that you think testing will solve? Do you see a lot of maintenance causing other errors? Do you want to do this in preparation for overhauling some part of the system?

You see bugs come back every 2-3 months, but is preventing that really worse than the overhead of maintaining and adhering to a test suite that all of the developers would have to do? That alone may not be enough of a reason to impose the overhead that testing can cause.

Since you are fixing bugs you have access to the codebase. Start a test framework yourself, of bugs you cover, and run it against the code day by day. See how often it catches issues, vs. how often you have to alter the tests because the code has changed in ways that are valid. Then you will have more real world metrics about problems solved vs. costs...

--
"There is more worth loving than we have strength to love." - Brian Jay Stanley

start small by sugarmotor · 2011-01-02 21:00 · Score: 5, Insightful

Just start small, with "failing tests" for each new bug. Bug fixed, test passes. Keep expanding the test coverage.

--
http://stephan.sugarmotor.org

Re:start small by VirexEye · 2011-01-02 21:32 · Score: 5, Insightful

+1 to moding parent up.
You won't get far convincing a product owner that you should spend months writing tests for the entire system.
Convincing someone that you should write unit tests for all new functionality to help guarantee the bug fix/new feature will continue to always work into the future is a much easier sell.
Re:start small by johnjaydk · 2011-01-03 00:42 · Score: 5, Informative

Excellent advice. Don't try to start a BIG, EXPENSIVE testing program. Have a look at the book: Working Effectively with Legacy Code by Michael C. Feathers

--
TCAP-Abort
Re:start small by msclrhd · 2011-01-03 00:55 · Score: 3, Informative

#include "myclass.h"
#include "assert.h"
void test_bug123()
{
myclass a(10);
assert(a.value() == 10);
}
int main()
{
test_bug123();
}
---
How hard was it to create that test framework? You don't need something that is overly complex, just something that will pass if the test succeeds or fails if it doesn't. You can then improve the test cases/framework as you go along -- look at the Wine project for example. They didn't create http://test.winehq.org/data/ over night, it was built up gradually, starting with getting the tests running as part of the build and slowly defining a wine test API as needed.
Collecting metrics and reporting them should be done automatically, but not initially.

Maybe it won't save money by initialE · 2011-01-02 21:01 · Score: 3, Insightful

Perhaps the app is just more important to you than it is to the company? First take your ego out of the equation, i guess, then make your case before the boss. Just sayin'. Also if you have a time sheet of how much time you've spent on this app so far it would be helpful in drumming up some numbers, plus a few scenarios where the failure of the app causes lost time for the company as a whole.

--
Starbucks, Harbuckle of Breath.

Cost-Benefit Time! by wild_berry · 2011-01-02 21:02 · Score: 5, Interesting

We've had to justify creating auto test suites where I work.

Over the last decade our product has grown from one code-base into three strands, each with separate customer foci, and we've had a healthy amount of staff turnover so that there are still brilliant, creative and skilled people working on it but some of the original knowledge has left us.

We found* numbers to justify that automated testing of existing features, applied later will protect against regressive changes. Even where there are complicated features which were not modular in design, or which lack good interfaces, the tests have saved us massive amounts of time testing by hand. The real win is hidden under something we didn't realise until later: creating the tests have forced us to really document what the features are and how they work**, sometimes from a unit-test level, sometimes at the interface level and sometimes in a top-to-bottom vertical slice. Once you have a record of what your software does, in a computer which is skilled at remembering exactly and repeating exactly what some former staff member told it a couple of years ago, you have a decent reason to be confident that your bug fixes won't cause more harm than good.

*: ballpark figures / educated guesses / made up.
**: We favour working code over comprehensive documentation, until our agile team is reassigned to other projects or leaves the company.

Eventually you need a solution by nanospook · 2011-01-02 21:03 · Score: 3, Interesting

The metrics is a good idea. This will allow the owner to see what it costs him to not have good QA. However, eventually, the conversation will float to how expensive will it be to have QA? Like any security situation, you will want to find the happy medium where the trade off in time/money balances the risk. Given you only have a few hundred users you might wish to experiment by adding QA incrementally. Also, you might find why those bugs are coming back? Poor documentation? Or are developers not maintaining up to date code bases? Process is just as important as QA when it comes to fixing bugs.

--
Have you fscked your local propeller head today?

Re:WHY do you have to prove software testing saves by Rosco+P.+Coltrane · 2011-01-02 21:19 · Score: 4, Insightful

WHY do you have to prove software testing saves money? Even a cheap and nasty electrical appliance from China is tested despite there being a lot of motivation to cut corners and not do so. Why should software be any different?

Rest assured that Chinese companies test their products precisely *because* it saves money. If it didn't, they wouldn't; they don't have a commitment to good quality for its own sake.

An untested product leads to high rates of return (lost money), customer dissatisfaction and brand name damage (HUGE loss of money in the medium to long term). Nobody puts up with bad products anymore. Software is one of the last kind of products where it's still somewhat accepted but people are quickly becoming intolerant to bugs nowadays.

Still, what is true of physical products (that extensive testing on top of proper design and good manufacturing practices) may not be true of software. I.e. the question is: is laborious and careful design and implementation with minimal testing more or less expensive than quicker, laxer design and coding and a strong test/correction feedback process? I'm not sure the answer is clear-cut. As a (former) programmer, I can see argument for both approaches.

--
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash

Re:Wrong Question tag appropriate by Rosco+P.+Coltrane · 2011-01-02 21:34 · Score: 2

Yeah, walk up to your boss and tell him you know more than he does. Great career move...

Have you heard of diplomacy? You may know more than he does, but he doesn't want to hear it. Instead, you should convince him that what *you* want to do is *his* idea.

--
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash

Paying technical debt. by SanityInAnarchy · 2011-01-02 21:35 · Score: 3, Insightful

As others are saying, if you can get hard numbers, get them. Nothing like actual evidence to back up your claim.

However, there's a more important reason for doing this which is harder to find solid evidence for -- the concept of "paying down your technical debt." Often, we hack software together relatively quickly, without fully testing, etc -- every time we do that, we're effectively borrowing against the future stability and maintainability of the product, or realistically, against our own debugging and refactoring time in the future.

Like all debt, it has interest -- the longer you let it fester, the worse of a mess it's going to be. So like all debt, it makes sense to pay it down when you can -- a little refactoring now can save a lot of headaches later.

So, it may be difficult to get hard numbers -- though again, those are better. But if you can't do that, sell it as an investment. Invest a little time right now at least putting the automated tests in place, and maybe start with just some regression testing. Twenty minutes writing a regression test will almost certainly save you an hour the first time that bug appears -- and those are pessimistic numbers (though also made up on the spot -- find your own actual facts!)

--
Don't thank God, thank a doctor!

Re:There's your problem... by SanityInAnarchy · 2011-01-02 21:37 · Score: 2

No, real developers use real version control software. Even SVN would be an upgrade, let alone Git, Mercurial, bzr, etc.

--
Don't thank God, thank a doctor!

Re:WHY do you have to prove software testing saves by Kjella · 2011-01-02 22:20 · Score: 3, Insightful

Still, what is true of physical products (that extensive testing on top of proper design and good manufacturing practices) may not be true of software. I.e. the question is: is laborious and careful design and implementation with minimal testing more or less expensive than quicker, laxer design and coding and a strong test/correction feedback process? I'm not sure the answer is clear-cut. As a (former) programmer, I can see argument for both approaches.

Didn't you just answer your own question: extensive testing on top of proper design and good manufacturing practices

For the first I'll just quote Knuth: "I have only proved it correct, not tried it." And if you send piles of shit to testing/QA and expect them to be the impenetrable barrier between you and the customer, you're equally deluded.

I've found that for anything that should last a while and be stable, there should be a test case. It's so easy to subtly break something. However, I've found writing a proper test suite that deals with databases, network communication+++ and not just the application itself is pretty hard.

--
Live today, because you never know what tomorrow brings

Develop a mathematical model of costs by Paul+Johnson · 2011-01-02 22:37 · Score: 3, Informative

Basically you have to develop a mathematical model of the costs of the current situation, and compare it with a mathematical model of the costs of using tests. As part of this you will have to produce a plan for introducing tests, with the costs for each step itemised. Use the best numbers available, but don't worry if some of those numbers are "best guess". Just don't try to hide the fact. Put both models in a spreadsheet and come up with a number for how long it will take to recoup the initial investment (break even). Don't forget to discount future cash flows. In MBA-speak this is known as a "business case".

--
You are lost in a twisty maze of little standards, all different.

Re:There's your problem... by Vegard · 2011-01-02 23:42 · Score: 2

I have seen the same happen with SVN. Trust me, people who don't understand version control systems will *always* find a way to fuck up! :)

Overall cost may not be "in his own best interest" by dtmos · 2011-01-02 23:44 · Score: 4, Insightful

If you truly are working for a small software development company, cost may not be the issue. Frequently (I may even say universally) the issue facing the owners of small companies is not cost, or even profitability, but cash flow -- making sure that there is enough money coming in to make the next big expense coming up (frequently payroll). In small companies there typically aren't the cash reserves available to spend on unanticipated expenses or program delays. The boss may even agree with you that the overall cost would be reduced if software testing were introduced, but may not "want to invest the time and money" because the company does not have the cash available to make the investment.

For example, the software you're working on needs to ship by the end of January to receive payment from the customer by the end of February, so that payroll can be made in March. If bug-fixing stops in January for development of the bug-fixing program, the customer doesn't get his software until the end of February, so he doesn't pay until the end of March and so payroll is missed that month. Having fewer bugs in the long term has to be balanced by the need to stay in business until the "long term" is reached.

It's like anything else in finance: You have better options and get a better return on your investment if you invest $100,000 than if you invest $1,000, but if you don't have $100,000, it doesn't much matter -- you do what you can with the resources you have available. Similarly, you get a better return if you're able to invest your money for a year than if you must have it back in a week. (The guy who said "time is money" actually knew something.) This trade between cash flow and long-term efficiency (leading, one hopes, to profitability), i.e., knowing when and how much to invest, is the real art of business management.

The solution to your problem? IMHO, incremental investment. I agree with those above who suggest implementing tests for new bugs found. This should enable you to begin to work more efficiently over time, without substantially delaying current work, while collecting metrics (I know, I know...) that can help quantify the cost of the bug re-work. Once a substantial body of tested bugs has been collected, one can institute a formal testing program (preferably by including it in the next project, so that it is a planned expense) as a cost reduction, since by then it should speed up development work over the ad hoc method then in place.

Re:WHY do you have to prove software testing saves by tibit · 2011-01-03 00:08 · Score: 2

Nobody puts up with bad products anymore.

-- Ha, ha, ha. The U.S. market obviously demands plenty bad, nearly worthless crap, if one were to judge from what's on the market.

There are simple things like cast-iron hand-cranked meat grinders where the market is saturated with extremely poorly made chinese crap, with absolutely no alternatives. I know, I've tried to buy a new one -- ended up getting a used Porkert instead.

Another simple thing: portable DVD players. I've already had to fix two of them because they had liquids spilled on them. It's not a rare occurence -- they are often used by kids who either don't care or just forget themselves and get unlucky. While the manufacturing quality is perfectly acceptable, the design quality just sucks big time. The clearances between many parts seem randomly chosen, so that either things that rub on each other seize, or they rattle. An no, this isn't a molding issue, it's by design. Molding on the units I looked at was by the book. You can tell that many small cross-section plastic parts, prone to breakage, were designed by people who have no clue or no experience in mechanical design. There are plenty of changes that one could make that would not increase the material costs, but would make those devices much more rugged.

It seems that many U.S. companies that brand products imported from China have no engineering on the other side of the pond, just marketing and purchasing people.

--
A successful API design takes a mixture of software design and pedagogy.

Re:But does it? Not always. by vlm · 2011-01-03 00:14 · Score: 2

Civilization V was released with a ton of bugs, many of which were (are!) showstopping crashers, as long as you got to a few hundred turns. Add a trillion smaller, non-crasher bugs, and you have one of the buggiest releases of all time. And yet, between brick-and-mortar and Valve/Steam, they sold about a million units. That's about $50M in the bank.

Testing? What testing?

The key difference is Civ5 is a short term experience, its just something you play until Civ6 comes out, assuming they survive.

On the other hand, the original post

We have one app that is used by a few hundred clients and was initially developed by a few undergrads about 10 years ago.

implies every sale is vital since there have only been a couple hundred, and its just one app, not a series.

In other words, everyone knows civ5 is super buggy and as long as its fun for awhile, who cares. On the other hand, over the previous decade word has gotten around about the original poster software thus only a couple hundred sales.

Quality control a decade ago might have turned them into a billion dollar company over the previous decade, rather than their current pitiful state. If, perhaps, they're planning on winding down next year, debugging for one eleventh the lifetime of the SW is probably pointless.

--
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger

Re:WHY do you have to prove software testing saves by gbjbaanb · 2011-01-03 00:17 · Score: 3, Insightful

Software is one of the last kind of products where it's still somewhat accepted but people are quickly becoming intolerant to bugs nowadays.

in consumer devices perhaps, but that's a different area to most software. For most software sold, especially to business users, bugs are accepted as one of those things that happen. Now, really buggy software will eventually get replaced but in most cases the important thing is to have a good (or excellent) customer service that responds to, and fixes the bugs. If you have that, your customers will not care so much about bugs that they find. you just need to be responsive, and considerate in your communication with them. Once you have that in place, you'll often find that its an asset to you, the more bugs the customer finds, the more they feel loved. And customers like feeling that you care for their business - especially when its some executive/supervisor reporting them, and some grunt who has to actually use your PoS software. Its how the big names can make such poor enterprise software and still have it selling in billions of dollars a year in maintenance/support payments.

I think I might have just explained how buggy software is good for your business. I feel dirty.

Lose The Job by assertation · 2011-01-03 01:19 · Score: 4, Insightful

If you are catching shit when a bug happens and they refuse to set up formal testing get out of the job now.

If you aren't catching shit of any kind, then look at the bugs as helping you keep employed. If it annoys you, go find another job.

I've worked for similar cheapskates. They aren't likely to change.

Maybe something from this NIST report? by mpsmps · 2011-01-03 01:22 · Score: 3, Informative

The Economic Impacts of Inadequate Infrastructure for Software Testing finds an average ROI for software testing somewhere between about 100% and 1000%.

Re:Wrong Question tag appropriate by sosume · 2011-01-03 01:40 · Score: 2

obligatory big bang theory quote incoming ...

Sheldon's Mom: Now, you listen here. I have being telling you since you were four years old, it's okay to be smarter than everybody but you can't go around pointing it out.

Sheldon: Why not?

Sheldon's Mom: Because people don't like it. You don't remember all the ass kickings you got from the neighbor kids. Now lets get cracking. Shower, shirt, shoes and let's shove off.

Sheldon: There wouldn't have been any ass kickings if that stupid death ray had worked.

It Doesn't Save Money by stewbacca · 2011-01-03 08:09 · Score: 2

Testing doesn't "save" money any more or less than "training" saves money. They both cost the company money.

Without either, however, you are less likely to receive future revenue because your software will garner the reputation of being bad (because it wasn't tested) and being hard to use (because there is no training).

There's always the old adage that you've got to spend money to make money. I think that holds true in this case.

Slashdot Mirror

How Do You Prove Software Testing Saves Money?

36 of 312 comments (clear)