Slashdot Mirror
Software Approvals For Consumer Markets?
Odkin asks: "Some friends and I are struggling with a hardware project which is stalled due to costly consumer market approvals (which is alright I guess). But it struck me, why are there only market approvals for hardware and not software? The hardware approvals include functionality tests that ensure that the product works as intended in any way the user would handle it (even unsuitable use). Would such approvals for commercial software improve the quality of the products, including minimizing the risk of data loss and heightening the security? In other words, would it facilitate or inhibit the creation of good software?"
make a better product. However it would make it real hard for small software houses to put out software. Plus isnt the EULA's whole point to get around accountability in a product?
"All I can tell the "lesser of two evils" folks is that if they keep voting for evil, they'll keep getting evil."-Lp.org
Most small video game makers have to run there final versions by the box makers (Sony and Xbox). They run it through a bastion of tests before they will let it out to the consumer market. It doesn't seem to harm the video game makers ability to create good games. Of course, this doesn't include usability testing.
______________________
It is the Project Managers responsibility to get the customers requirements correct, it is also their responsibility to ensure that the requirements are met through automated functional level testing.
cam--
ever read the warranty that comes with yr lawnmower? about how it's only valid if the mower is used "reasonably and correctly"? if you run over rocks or now nine foot wet grass, the warranty won't cover damage. most software is like that.
testing is done for "reasonable" use and the software shop regards "unreasonable" use as being either a) uncovered or b) a violation of the eula.
2 1337 4 u!
Software Testing/Quality Assurance is supposed to perform this function.
The problem is often insufficient tools.
The company I work for as a Software Test Automation Specialist is looking at WorkSoft Certify and we like what we see, except the price-point (approximately triple our current tool: Rational Robot), however, that is currently in negotiations.
This user account is inactive account replaced by the PDA
You have never worked in an ISO9000 shop.
Of course that doesn't mean that processes are any good. It does mean that the processes are documented and we stand by them.
There are some good software shops out there that do a good job of vetting their code of bugs: like the guys who make VMWare. Then there are other shops that don't: like the guy who make MS Windows.
Besides it's too late to require government involvement. The accepted industry practice of putting out buggy crap has already been established with the notable exceptions where NASA(proof that one can't catch every bug) and the FDA(proof that one can wade through immense bureaucratic red tape) are concerned.
Most software goes through a few stages of testing, namely QA (or QC depending on your favorite acronym), alpha then beta testing. At that point it is deemed acceptable and released to the public. The nice part about software vs. hardware is the ease of upgrading, so it doesn't have to be 100% out the door.
Plus, who wants to build a hardware web server, application server, etc? Would it be fast? Yes. Would it be maintainable? No. Could you design it so you could add dynamically to the hardware? Probably. Would that be insanely expensive and take forever? Yes.
You get the idea. Software generally is an application of quick design meant to short-circuit designing things in hardware and give the option of easy upgradeability.
If your real question is, why is most software so crappy... if it's open source, fix it!
While it would of course be nice to have software without errors, the problem then becomes price and time to market. There is a saying in the project management world, "Speed, Price, and Quality. Pick two of the three." I've found this saying to be pretty accurate.
As consumers we tend to want everything now, and cheaply. This would obviously push down the quality of the product. Being an impulse buyer myself I find most products pretty much suck these days because manufacturers (of software or hardware) know that we want everything now and cheap, so they don't focus on quality at all, just time to market. I'm of course exagerating a bit, but it does seem consumerism kills quality.
Was just thinking, wouldn't "safety" in the mind of a software engineer being "failing safely"? i.e. not bringing down the entire system due to one application? I don't mind the single application so much as I mind memory leaks and security holes that tend to decimate the rest of my applications. I would think that an at least somewhat "cookie cutter" approach could be taken with these issues. Granted I wouldn't recommend them for an open source project. My thought is that if you intend to make money off it: it cannot interfere with another project (barring that it is designed to do so i.e. Spybot S&D), it cannot crash the entire system, it must be secure. Of course in this day and age people accept that things aren't going to be secure, not without insane overhead. Well that is what the certification process is about. UL charges $$$ to have things UL certified. They charge even more to come on-site to tell you that you can't call it UL certified until there are 3 additional stickers in place (been there). So create the UL of software for commercial applications. If you want to run it on a PC and make money from it you have to have it XX certified. It costs $3000 to have the software certified, and takes x months. It would slow the release of commercial software but at least stability would be improved. And many crashes are due to third party software, even the interaction between other applications. -Coach
"Never upset a goalie, getting hit with a blocker is an unpleasent experience - facemask or not." -Me
The problem is in defining what exactly consitutes a GOOD approval process for any given piece of software. It's often easier to define this for hardware. You define proper operating ranges and how the thing should respond when used or abused in specific ways, and the result is often a product that will behave as expected in almost all realworld conditions.
In software the failure cases tend to be more open-ended. The set of all possible types of input to the system may involve infinite permutations. You can only test the ones you thought of, and if you thought of them they're probably handled correctly in the first place. If you're developing a commercial app you have to deal with the fact that the hardware and OS your program relies on may in fact be subtly flawed. Also, any set of tests for a piece of software must be custom designed for that piece of software.
Ideally software testing is more of a verification process than a corrective process. Your tests should (but rarely are) be created at the same time as your design and run continuously throughout the project lifecycle.
You can't easily patch hardware. The consumer:
with software once you identify the problem and fix it, the customer might be out of commission a half hour while the download, install, and possibly reboot the machine.
Sorry, I'll bite. That's all good and fine in theory, but in practice that's another story.
I do not regard stuff like a game crashing every half an hour as being caused by "unreasonable" use. Or for example: which of Fallout 2's many script bugs were "unreasonable" use?
I also do not regard stuff like "oops, the user used the back button in the browser" or "oops, the user opened a link in another window" on web sites to be "unreasonable" use. Use of bog-standard browser features, that have been around for more than a decade, _is_ reasonable.
It's the retarded ex-burger flippers who moved into software development during the dot-bomb that are unreasonable there. If Joe Coder can't use the HTTP session right (yes, including supporting multiple windows _and_ the back button) then the only "unreasonable" part is Joe Coder still being employed. Period.
Etc.
Basically I don't know about mandatory government testing, but I would very much like to see some legal responsibility that can't be waved away with an EULA. Some part that says that your responsibility is to the user, not just the current "hey, we only need to take their money. And then who the fsck cares if it works?"
I'd also like to see some legal responsibility for the marketroids, same as in any other industry. If you say that a piece of software does something, then it damn better do that, to the letter. Just like if a steel company's marketroid says "we'll sell you 10 ft long, 1 inch thick beams, with 0.1% carbon content", you can sue the pants off those guys if it's only 9 ft long and with a completely other carbon content.
And yes, I _am_ a software developper. It just makes me sick to see what this industry has turned into. It's the biggest snake oil operation in history. Hundreds of billions of dollars worth of snake oil every year.
And this doesn't come out of nowhere. It's draining the rest of the society to keep a bunch of cheats, liars and leeches in business.
A polar bear is a cartesian bear after a coordinate transform.