Re:Testing - The Anti Quality Process
on
QA != Testing
·
· Score: 1
I am not sure where you get the idea that I criticized Microsoft for not making universally usable software. The web page I put up is accessable to anyone that uses windows. The same applies to MS Word. Linux users have to use some other word processor.
My web page was put together to keep me amused and as a resource for looking for work. I don't know what costs might be born by others as a result.
Quality (or the lack of it) is an economic issue. If someone thought it was worth the cost to buy MS Windows just to read my web page, I would be very flattered but it is not likely to happen. The vast majority of PCs have Windows and MS IE.
Try to keep a focus on the topic which is the relationship between quality and testing. Its the message that is important, not the package. If it means a lot to you to look at my site, go visit a friend who has windows.
Re:Testing - The Anti Quality Process
on
QA != Testing
·
· Score: 1
I am Quartz's father. I found the replies that were generated by my son's posting are very interesting. My appologies to those who do not have MS Internet Explorer but contrary to what my son claims, I have no intention of changing the page. It just isn't worth the effort to hand craft it in HTML that everyone can read.
The objective of the presentation was to show how futile it is to attempt to improve the reliability of a software system of any significant size by backend testing. The impact of any economically feasible test program on the typical defect ridden system is not significant. It then goes on to show how to use some reasonably easy to collect metrics to produce a verification plan that will have a significant impact and that is economical (assuming a reasonable but not exceedingly onorous or rigerous development process which is not changed).
One other idea that is very important is the concept of quality as the lack of waste ($)and therefore that quality improvement is the reduction of waste. Waste is not just waste in the development organization. I suspect that Microsoft caused more waste with its operating systems and office tools than they spent on their development. The waste occured through lost productivity on the part of the users.
For those who didn't like the analogy built of jelly beans (those blue ones sure taste good!), how about an analogy based on another food group. Mushrooms! (which unlike software defects can be eaten or sold).
Consider a wooded park with many trails that fork and rejoin (like the logical paths through a piece of software). On one warm summer morning there are N mushrooms growing in the park. There is a person who walks the same path in the park every morning and who likes muchrooms. On this morning he/she find X mushrooms along the regular path. On a second summer morning there are 2N muchrooms in the park. The person follows the same path and finds 2X muchrooms (the assumption being that the path is representative of the park as a whole).
Case 1: Fraction found X/N
number left in the park (N-X)
Case 2: Fraction found 2X/2N
number left in the park (2N-2X)
If this analogy holds true, then a given amount of test effort would produce the same fraction of the defects in a software system, regardless of the number present and the more defects a given amount of testing produces, the more defects will be left in the system. The analogy holds not just for software but for blue jellybeans as well.
Given that an equal effort will find an equal fraction of the defects, then the analogy can be used to estimate the total defect load at the start of test. If 2E units of test effort were exepended and the first E units found X defects and the second E units found Y defects, then the equation
X/N = y/(N-X) should hold true (YX). This can be solved for N. Try it on the JellyBean model examples. It gives much better estimates than the extraplation of the historical data.
There is only one way to get a reliable system out of test. That involves putting a reliable system into test. Once the reliability is determined, additional testing is waste. If there are defects, other far more economical methods of finding them exist.
Quality is free. But it is not a gift. You have to make an investment and you have to use common sense and maybe a little arithmatic. Testing is intuitively seductive because it looks like it is the right thing to ds. But it doesn't deliver the goods.
Slashdot Mirror
I am not sure where you get the idea that I criticized Microsoft for not making universally usable software. The web page I put up is accessable to anyone that uses windows. The same applies to MS Word. Linux users have to use some other word processor.
My web page was put together to keep me amused and as a resource for looking for work. I don't know what costs might be born by others as a result.
Quality (or the lack of it) is an economic issue. If someone thought it was worth the cost to buy MS Windows just to read my web page, I would be very flattered but it is not likely to happen. The vast majority of PCs have Windows and MS IE.
Try to keep a focus on the topic which is the relationship between quality and testing. Its the message that is important, not the package. If it means a lot to you to look at my site, go visit a friend who has windows.
I am Quartz's father. I found the replies that were generated by my son's posting are very interesting. My appologies to those who do not have MS Internet Explorer but contrary to what my son claims, I have no intention of changing the page. It just isn't worth the effort to hand craft it in HTML that everyone can read. The objective of the presentation was to show how futile it is to attempt to improve the reliability of a software system of any significant size by backend testing. The impact of any economically feasible test program on the typical defect ridden system is not significant. It then goes on to show how to use some reasonably easy to collect metrics to produce a verification plan that will have a significant impact and that is economical (assuming a reasonable but not exceedingly onorous or rigerous development process which is not changed). One other idea that is very important is the concept of quality as the lack of waste ($)and therefore that quality improvement is the reduction of waste. Waste is not just waste in the development organization. I suspect that Microsoft caused more waste with its operating systems and office tools than they spent on their development. The waste occured through lost productivity on the part of the users. For those who didn't like the analogy built of jelly beans (those blue ones sure taste good!), how about an analogy based on another food group. Mushrooms! (which unlike software defects can be eaten or sold). Consider a wooded park with many trails that fork and rejoin (like the logical paths through a piece of software). On one warm summer morning there are N mushrooms growing in the park. There is a person who walks the same path in the park every morning and who likes muchrooms. On this morning he/she find X mushrooms along the regular path. On a second summer morning there are 2N muchrooms in the park. The person follows the same path and finds 2X muchrooms (the assumption being that the path is representative of the park as a whole). Case 1: Fraction found X/N number left in the park (N-X) Case 2: Fraction found 2X/2N number left in the park (2N-2X) If this analogy holds true, then a given amount of test effort would produce the same fraction of the defects in a software system, regardless of the number present and the more defects a given amount of testing produces, the more defects will be left in the system. The analogy holds not just for software but for blue jellybeans as well. Given that an equal effort will find an equal fraction of the defects, then the analogy can be used to estimate the total defect load at the start of test. If 2E units of test effort were exepended and the first E units found X defects and the second E units found Y defects, then the equation X/N = y/(N-X) should hold true (YX). This can be solved for N. Try it on the JellyBean model examples. It gives much better estimates than the extraplation of the historical data. There is only one way to get a reliable system out of test. That involves putting a reliable system into test. Once the reliability is determined, additional testing is waste. If there are defects, other far more economical methods of finding them exist. Quality is free. But it is not a gift. You have to make an investment and you have to use common sense and maybe a little arithmatic. Testing is intuitively seductive because it looks like it is the right thing to ds. But it doesn't deliver the goods.