Given the software development technologies and practices that we have today, are software warrantees currently practical?
I contend that they are not.
Physical manufacturing has been playing their game for hundreds of years. We've been "manufacturing" software for just 40 years. Manufacturing and manufactured products are constrained by physical laws. Software, and software systems, are free of this limitation. Most physical products have fewer parts than significant software systems and are much easier to test. All in all, it's easier to raise the quality of a physical product than it is to raise the quality of software. And "easier" equates to "cheaper."
The other example you cited, GTech, is a vertical application with very specific requirements for success and performance. They also have a direct monetary motivation to improve the quality of their software (i.e., fewer errors == lower penalties). Any resources they commit to improving their software translates directly to an improved bottom line.
I would guess that if your were to examine the software they use to achieve their 0.3% penalty rate, you would likely find that the software itself is less complex than, say, M$ Word. It probably has fewer KLOC, has no GUI, doesn't have to contend with backward compatibility and probably supports no interoperability with other software. Plus, GTech has probably committed many, many thousands of hours to writing, testing and debugging this one application.
M$ could do the same thing for M$ Office. But who wants to pay $5,000 for M$ Office?
Also, regarding the "AS-IS" warning: nearly every EULA that I've actually taken the time to read is nothing more than an AS-IS warning wrapped in Lawyer-Speak. This is the problem. Most (all?) software today is delivered "as is," without guarantees and without hope for a consumer to seek retribution for any damages that arise from using the software.
To offer a warrantee, the warrantor (is that a word?;-), e.g., Micro$oft, must have some reasonable expectation that if they get sued, they will be able to defend their position. If they are sued, and they have to pay penalites, those penalties are necessarily passed to the consumer as higher product prices.
Now, of course, no software company wants to be sued. So, to keep from being sued, they'll have to raise the quality of the software they deliver. This process is not free; it's certainly not cheap.
Improving software quality requires more front-end work (analysis, design, etc.), improved development techniques and tools, and stronger verification techniques (automated testing, monitoring and improving path coverage, etc.) This all costs money. Since most companies are in the game for profit, if all these costs were embeded in the development process, the only way to maintain margins would be to raise the price of the delivered software.
There is another possibility, too: don't improve the software, expect to be sued and expect to pay damages. Now, to maintain the bottom line, send the attorneys, accountants and statisticians into a room, crank some numbers and come up with a new price structure for the product line that will maintain the current profit margins, and still pay for the anticipated damages.
I argued elsewhere in this thread (Liability Laws Impossible for Software (today)) that, while software warrantees are surely desireable, using current technology, I question whether they are practical.
To me, at least, the problem is not that we should offer software warrantees. I think everyone agrees warrantees are desireable. From where I sit, the problem is that current technologies and practices make providing software warrantees prohibitively expensive and therefore make them a practical impossibility. At least today.
Until we as an industry can make "better" software at lower cost, the whole discussion of software warrantees is, again, in practical terms, moot.
Finally, this argument only applies to consumer and business software. Man-rated systems (avionics, etc.) have no room for error and we should spend every penny needed to keep from inadvertently killing someone. However, while using M$ Word, most people's lives are not at risk. Their jobs, maybe, not not their lives.;-)
Maybe, under the influence of software lemon laws, the faces of Open Source and Free Software might be altered in America. However, there are six billion other humans who live outside the US. Of them, many millions develop software. Within that group are many thousands who contribute to the Open Source/Free Software movements. If Open Source/Free Software changed, or even died, in America, it would surely live on outside the US.
Sadly, we sheltered Americans often forget the fact that there's a big, wide, world outside of the good, ol' US of A.
Speaking only for myself, of course: thank you for the reminder.
Here is what I believe is the core issue in this discussion: as we practice software development today, protecting ourselves from software liability laws is a practical impossibility. If Software Lemon Laws existed, we'd probably all be out of a job, and/or software would very expensive to own. Plus, the Open Source movement as we know it would probably dry up.
Many of the posts in this discussion focus on the results of, or the viability of, Lemon Laws on the Open Source or the Closed Source software development industries. Futher, many posters have held out automobile manufacturing as a example, both to defend and assault the effects of liability laws on the software industry.
I submit that automobile manufacturing (or most any other manufacturing process) is orders of magnitude less complex than building significant software systems. So much so that I contend that it's currently next to impossible to know with 100% assurance that your software is correct (using current techniques and technologies). Software is too complex (or our testing software and techniques are too inadequate) to test to the point that we can be sure that we won't be sued, or that if we are sued our position is defensible. Also, given the current state of affairs in the liability law arena, if you can be sued, eventually you will be sued.
The beauty of manufacturing is that manufacturing, and the products produced, are constrained by physical laws. Parts fit together in specific ways, they exist in space, they take up room and they interact according to known physical (or chemical or nuclear) laws. Under these conditions it's "easy" (relatively speaking) to model an entire vehicle in software, or in engineering diagrams, before you ever start tooling a plant. Boeing's 777 was entirely built and tested in CAD/CAM before being manufactured.
How many "parts" are there in, say, a Ford SUV (including fasteners)? 25K? 50K? 100K? By today's software standards, this is a relatively simple system.
If we liken a line of code to a "part" in a vehicle (and by "parts" I would even include screws, nuts and washers), when was the last time that you ran across a significant piece of software (an OS, a word processor, a CAD/CAM system, a accounting package) that has less than 100K "parts" (aka, lines of code)? Most significant software, the kind that would require Lemon Law protection, is significantly larger than 100KLOC. But size is just the beginning of the problem.
When was the last time that you were able to model and test a 100KLOC (or 500KLOC or 1MLOC) software system before "manufacturing" it? A new car, or a new airplance, can be almost completely modeled and tested in virtual space before seeing the light of physical space. Not so for software systems, at least not ones that the average business can afford. (not that any software ever really "sees the light of physcial space," anyway;-)
Additionally, software doesn't function just as the "parts" that are "manufactured." Some "parts" don't exist until the software is executed (i.e., files, objects and other data). So, how do you test something 100% (or at least to a level that makes your lawyers happy) that has 250K (or 500K or 1M) parts, when you can't touch those parts, or even anticipate 100% what all the parts will look like? If manufacturing worked this way (where parts are created and destroyed, modified and manipulated when the product is sent to the field), would Henry Ford ever have been able to create an assembly line. Likely not.
Add to this idea that 100% test coverage of every logical branch, and every permutation of data manipulation, approaches an N-Complete problem. Currently, problems that are N-Complete are considered intractable. As a problem approaches "N-Completeness," it also approaches insoluability (using current technologies). Though 100% testing is not a true N-Complete problem, it is one that is difficult to manage and address -- and doing so ain't cheap.
Finally, stir in the real X-Factor: our users (God bless 'em;-). Developers: how many times have your users come to you and said "your software is broken," when, in reality, they were using the software for something that you'd never intended for it to do? Once, or twice, I'd guess. When this happened, who was "at fault?"
When software is "driven," unlike SUV's, it isn't constrained by physical laws. There's little risk in "trying something new." On the other hand, SUV drivers understand that "trying something new" (driving off a cliff, taking a corner at high speed, backing into a closed garage door, leaving the windows open during a rain storm, locking your kids inside in the heat of the day) normally has some obvious physical consequence. This is usually not the case for software (unless your software is controlling a CAT scanner...;-).
With business software, users "try stuff," they get creative, they push the envelope. When you push the envelope in an SUV, it falls over. When a user pushes the envelope in an SUV and it tips over, who's liable? When a user "pushes the envelope" using software and the software "tips over," sending a gigabyte of data to data heaven, who's liable?
Our SUV driver, like every wheeled-vehicle user, is constrained by physical laws. We accept that it may be irresponsible for a driver to "push the envelope" in an SUV. The consequences are obvious and well known to all wheeled-vehicles users. If the consequences are not obvious to the driver at first, they become so quickly. Still, is Ford liable for someone dumping his Excursion into a ditch, even when the user exceeded the design parameters of the system? Not so much.
However, when a software user "pushes the envelope" the consequences are usually not rooted in physics. So, when an intrepid user tries something new, and flushes precious data down the bit-toilet, who's really at fault? The user? Or the "manufacturer" for not anticipating this particular use of the system? A number of a factors would affect assessing blame, and asessing blame could happen in court -- and court ain't cheap.
(Please note that man-rated systems like space shuttle and airliner avionics, nuclear power plant control systems, PET/CAT/MRI control software are, and must be, held to a higher standard than business software and much of what I'm saying doesn't apply there -- but that level of quality ain't cheap, either.)
We could build business software, today, that would better stand the challenges of Lemon Laws, but that would drive up the costs of development. However, would end-users really want to pay $5,000 for M$ Office so that they're assured that it's fully tested -- at least to the point that M$, and it's lawyers, believe they could withstand the assault of a nation full liability attorneys?
What about the Open Source software? Granted, in Open Source bugs are shallower due to the greater number of eyes-balls scrutinizing the code. However, would any of us be willing to spend the time needed to truly test software (peer reviews are effective, but only go so far) to ensure that it'd survive it's day in court? Not likely. Beside, as already noted, since Open Source software is essentially free (without cost), open source developers may not be liable in court.
So, where does this leave us?
To me it seems that Lemon Laws for software are a practical impossibility, at least using today's technologies and practices. Raising the quality bar for software such that it could survive litigation would significantly raise the cost of software itself, likely making software prohibitively expensive. Further, if such laws were enacted, in the current climate (sue! sue! sue!) there would be law-suits. Maybe lots of them. Such activity would sap profits from the industry in the form of legal defense costs. These profits would have to be replaced, further increasing the cost of software to the end-user.
As for Open Source software, as mentioned in other posts in this discussion, Open Source would probably not be liable (or at least "sue-able") under Software Lemon Laws. Hence, there would be little incentive to raise the quality bar sufficiently to protect against litigation.
If it were determined that the Open Source community is liable, since it's primarily a volunteer work force, the work force would dissipate out of fear of litigation. I don't know of too many Open Source developers who would be willing to lose their homes and cars to Software Lemon Law litigation.
The future may change this situation. Who knows? Hopefully, it will. Otoh..... maybe Lemon Laws would force us to get our feces in one sock.....
I was able to scrounge up a couple of references that speak to some of these issues, both for and against:
Finally, please note that I don't believe that Software Lemon Laws are inherently bad. What I believe is that they are currently bad for the industry. The industry would not likely withstand the costs associated with protecting itself against Lemon Laws. At least not yet. I remain hopeful that the picture will change in the future.
Thanks for listening...;-)
P.S.: this is my first post at/., please be gentle... hehehe
Slashdot Mirror
Given the software development technologies and practices that we have today, are software warrantees currently practical?
I contend that they are not.
Physical manufacturing has been playing their game for hundreds of years. We've been "manufacturing" software for just 40 years. Manufacturing and manufactured products are constrained by physical laws. Software, and software systems, are free of this limitation. Most physical products have fewer parts than significant software systems and are much easier to test. All in all, it's easier to raise the quality of a physical product than it is to raise the quality of software. And "easier" equates to "cheaper."
The other example you cited, GTech, is a vertical application with very specific requirements for success and performance. They also have a direct monetary motivation to improve the quality of their software (i.e., fewer errors == lower penalties). Any resources they commit to improving their software translates directly to an improved bottom line.
I would guess that if your were to examine the software they use to achieve their 0.3% penalty rate, you would likely find that the software itself is less complex than, say, M$ Word. It probably has fewer KLOC, has no GUI, doesn't have to contend with backward compatibility and probably supports no interoperability with other software. Plus, GTech has probably committed many, many thousands of hours to writing, testing and debugging this one application.
M$ could do the same thing for M$ Office. But who wants to pay $5,000 for M$ Office?
Also, regarding the "AS-IS" warning: nearly every EULA that I've actually taken the time to read is nothing more than an AS-IS warning wrapped in Lawyer-Speak. This is the problem. Most (all?) software today is delivered "as is," without guarantees and without hope for a consumer to seek retribution for any damages that arise from using the software.
To offer a warrantee, the warrantor (is that a word? ;-), e.g., Micro$oft, must have some reasonable expectation that if they get sued, they will be able to defend their position. If they are sued, and they have to pay penalites, those penalties are necessarily passed to the consumer as higher product prices.
Now, of course, no software company wants to be sued. So, to keep from being sued, they'll have to raise the quality of the software they deliver. This process is not free; it's certainly not cheap.
Improving software quality requires more front-end work (analysis, design, etc.), improved development techniques and tools, and stronger verification techniques (automated testing, monitoring and improving path coverage, etc.) This all costs money. Since most companies are in the game for profit, if all these costs were embeded in the development process, the only way to maintain margins would be to raise the price of the delivered software.
There is another possibility, too: don't improve the software, expect to be sued and expect to pay damages. Now, to maintain the bottom line, send the attorneys, accountants and statisticians into a room, crank some numbers and come up with a new price structure for the product line that will maintain the current profit margins, and still pay for the anticipated damages.
I argued elsewhere in this thread (Liability Laws Impossible for Software (today)) that, while software warrantees are surely desireable, using current technology, I question whether they are practical.
To me, at least, the problem is not that we should offer software warrantees. I think everyone agrees warrantees are desireable. From where I sit, the problem is that current technologies and practices make providing software warrantees prohibitively expensive and therefore make them a practical impossibility. At least today.
Until we as an industry can make "better" software at lower cost, the whole discussion of software warrantees is, again, in practical terms, moot.
Finally, this argument only applies to consumer and business software. Man-rated systems (avionics, etc.) have no room for error and we should spend every penny needed to keep from inadvertently killing someone. However, while using M$ Word, most people's lives are not at risk. Their jobs, maybe, not not their lives. ;-)
Just my $.02. ;-)
Touché...
Nicely said, and so very true....
Maybe, under the influence of software lemon laws, the faces of Open Source and Free Software might be altered in America. However, there are six billion other humans who live outside the US. Of them, many millions develop software. Within that group are many thousands who contribute to the Open Source/Free Software movements. If Open Source/Free Software changed, or even died, in America, it would surely live on outside the US.
Sadly, we sheltered Americans often forget the fact that there's a big, wide, world outside of the good, ol' US of A.
Speaking only for myself, of course: thank you for the reminder.
Here is what I believe is the core issue in this discussion: as we practice software development today, protecting ourselves from software liability laws is a practical impossibility. If Software Lemon Laws existed, we'd probably all be out of a job, and/or software would very expensive to own. Plus, the Open Source movement as we know it would probably dry up.
Many of the posts in this discussion focus on the results of, or the viability of, Lemon Laws on the Open Source or the Closed Source software development industries. Futher, many posters have held out automobile manufacturing as a example, both to defend and assault the effects of liability laws on the software industry.
I submit that automobile manufacturing (or most any other manufacturing process) is orders of magnitude less complex than building significant software systems. So much so that I contend that it's currently next to impossible to know with 100% assurance that your software is correct (using current techniques and technologies). Software is too complex (or our testing software and techniques are too inadequate) to test to the point that we can be sure that we won't be sued, or that if we are sued our position is defensible. Also, given the current state of affairs in the liability law arena, if you can be sued, eventually you will be sued.
The beauty of manufacturing is that manufacturing, and the products produced, are constrained by physical laws. Parts fit together in specific ways, they exist in space, they take up room and they interact according to known physical (or chemical or nuclear) laws. Under these conditions it's "easy" (relatively speaking) to model an entire vehicle in software, or in engineering diagrams, before you ever start tooling a plant. Boeing's 777 was entirely built and tested in CAD/CAM before being manufactured.
How many "parts" are there in, say, a Ford SUV (including fasteners)? 25K? 50K? 100K? By today's software standards, this is a relatively simple system.
If we liken a line of code to a "part" in a vehicle (and by "parts" I would even include screws, nuts and washers), when was the last time that you ran across a significant piece of software (an OS, a word processor, a CAD/CAM system, a accounting package) that has less than 100K "parts" (aka, lines of code)? Most significant software, the kind that would require Lemon Law protection, is significantly larger than 100KLOC. But size is just the beginning of the problem.
When was the last time that you were able to model and test a 100KLOC (or 500KLOC or 1MLOC) software system before "manufacturing" it? A new car, or a new airplance, can be almost completely modeled and tested in virtual space before seeing the light of physical space. Not so for software systems, at least not ones that the average business can afford. (not that any software ever really "sees the light of physcial space," anyway ;-)
Additionally, software doesn't function just as the "parts" that are "manufactured." Some "parts" don't exist until the software is executed (i.e., files, objects and other data). So, how do you test something 100% (or at least to a level that makes your lawyers happy) that has 250K (or 500K or 1M) parts, when you can't touch those parts, or even anticipate 100% what all the parts will look like? If manufacturing worked this way (where parts are created and destroyed, modified and manipulated when the product is sent to the field), would Henry Ford ever have been able to create an assembly line. Likely not.
Add to this idea that 100% test coverage of every logical branch, and every permutation of data manipulation, approaches an N-Complete problem. Currently, problems that are N-Complete are considered intractable. As a problem approaches "N-Completeness," it also approaches insoluability (using current technologies). Though 100% testing is not a true N-Complete problem, it is one that is difficult to manage and address -- and doing so ain't cheap.
Finally, stir in the real X-Factor: our users (God bless 'em ;-). Developers: how many times have your users come to you and said "your software is broken," when, in reality, they were using the software for something that you'd never intended for it to do? Once, or twice, I'd guess. When this happened, who was "at fault?"
When software is "driven," unlike SUV's, it isn't constrained by physical laws. There's little risk in "trying something new." On the other hand, SUV drivers understand that "trying something new" (driving off a cliff, taking a corner at high speed, backing into a closed garage door, leaving the windows open during a rain storm, locking your kids inside in the heat of the day) normally has some obvious physical consequence. This is usually not the case for software (unless your software is controlling a CAT scanner... ;-).
With business software, users "try stuff," they get creative, they push the envelope. When you push the envelope in an SUV, it falls over. When a user pushes the envelope in an SUV and it tips over, who's liable? When a user "pushes the envelope" using software and the software "tips over," sending a gigabyte of data to data heaven, who's liable?
Our SUV driver, like every wheeled-vehicle user, is constrained by physical laws. We accept that it may be irresponsible for a driver to "push the envelope" in an SUV. The consequences are obvious and well known to all wheeled-vehicles users. If the consequences are not obvious to the driver at first, they become so quickly. Still, is Ford liable for someone dumping his Excursion into a ditch, even when the user exceeded the design parameters of the system? Not so much.
However, when a software user "pushes the envelope" the consequences are usually not rooted in physics. So, when an intrepid user tries something new, and flushes precious data down the bit-toilet, who's really at fault? The user? Or the "manufacturer" for not anticipating this particular use of the system? A number of a factors would affect assessing blame, and asessing blame could happen in court -- and court ain't cheap.
(Please note that man-rated systems like space shuttle and airliner avionics, nuclear power plant control systems, PET/CAT/MRI control software are, and must be, held to a higher standard than business software and much of what I'm saying doesn't apply there -- but that level of quality ain't cheap, either.)
We could build business software, today, that would better stand the challenges of Lemon Laws, but that would drive up the costs of development. However, would end-users really want to pay $5,000 for M$ Office so that they're assured that it's fully tested -- at least to the point that M$, and it's lawyers, believe they could withstand the assault of a nation full liability attorneys?
What about the Open Source software? Granted, in Open Source bugs are shallower due to the greater number of eyes-balls scrutinizing the code. However, would any of us be willing to spend the time needed to truly test software (peer reviews are effective, but only go so far) to ensure that it'd survive it's day in court? Not likely. Beside, as already noted, since Open Source software is essentially free (without cost), open source developers may not be liable in court.
So, where does this leave us?
To me it seems that Lemon Laws for software are a practical impossibility, at least using today's technologies and practices. Raising the quality bar for software such that it could survive litigation would significantly raise the cost of software itself, likely making software prohibitively expensive. Further, if such laws were enacted, in the current climate (sue! sue! sue!) there would be law-suits. Maybe lots of them. Such activity would sap profits from the industry in the form of legal defense costs. These profits would have to be replaced, further increasing the cost of software to the end-user.
As for Open Source software, as mentioned in other posts in this discussion, Open Source would probably not be liable (or at least "sue-able") under Software Lemon Laws. Hence, there would be little incentive to raise the quality bar sufficiently to protect against litigation.
If it were determined that the Open Source community is liable, since it's primarily a volunteer work force, the work force would dissipate out of fear of litigation. I don't know of too many Open Source developers who would be willing to lose their homes and cars to Software Lemon Law litigation.
The future may change this situation. Who knows? Hopefully, it will. Otoh..... maybe Lemon Laws would force us to get our feces in one sock.....
I was able to scrounge up a couple of references that speak to some of these issues, both for and against:
http://www.kaner.com/coverage.htm
http://www.bullseye.com/coverage.html
http://www.badsoftware.com/qindex.htm
http://www.bostonspin.org/slides/CemKaner.ppt (PowerPoint... sorry)
Some of these issues have been previously discuss here, at our beloved /.:
http://slashdot.org/developers/02/04/21/0058214.sh tml?tid=172
Finally, please note that I don't believe that Software Lemon Laws are inherently bad. What I believe is that they are currently bad for the industry. The industry would not likely withstand the costs associated with protecting itself against Lemon Laws. At least not yet. I remain hopeful that the picture will change in the future.
Thanks for listening... ;-)
P.S.: this is my first post at /., please be gentle... hehehe