Slashdot Mirror
Software Approvals For Consumer Markets?
Odkin asks: "Some friends and I are struggling with a hardware project which is stalled due to costly consumer market approvals (which is alright I guess). But it struck me, why are there only market approvals for hardware and not software? The hardware approvals include functionality tests that ensure that the product works as intended in any way the user would handle it (even unsuitable use). Would such approvals for commercial software improve the quality of the products, including minimizing the risk of data loss and heightening the security? In other words, would it facilitate or inhibit the creation of good software?"
make a better product. However it would make it real hard for small software houses to put out software. Plus isnt the EULA's whole point to get around accountability in a product?
"All I can tell the "lesser of two evils" folks is that if they keep voting for evil, they'll keep getting evil."-Lp.org
Or is it both?
Some software goes through rigorous approval and acceptance testing. I'm looking at the software for the space shuttle. It's like civil engineering - due to the huge liabilities inherent in a failure scenario, an incredible amount of effort is put into ensuring that a failure scenario does not happen.
Some software gets cursory testing. I'm looking at my employer. It's like a burger - who cares if you get one pickle slice or two, as long as you get your burger?
And some software is like an analogy that makes no sense, like bridges and burgers. Mmmm, Chief Justice Warren Burger...
Posting anonymously. Hi, boss!
actually making a software product that functions, even when stressed? now that's just crazy talk.
but in reality this is somethign that every product (hardware or software) should go through. It'll just make a better product.
Pretty much becuase Programmers aren't considered profesionals like "Doctors" and "Lawyers" by the govts, as they can teach themselvfes, and hence are exempt from Malpractice.
Also teh EULA especially in UCITA states shields the software company from damages. Go read just about any EULA when it talks about damages if you don't believe me.
It would certainly help usability. If you extend the analogy of unsuitable use of hardware to software, what if I click the wrong button or enter an illegal command. This should all be handled by good software.
But if that process ever became standard, it might help quite a bit with security. Throw in some bogus data and see if anyone can read it or write to it illegally.
Ultimately, this will never happen unless users demand it, and refuse to buy a product unless it passes such a test. And I don't know if that will happen.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
If you are trying to get liability insurance, that's another thing - you can spend as much money as you have and it may not help.
So... give us a fer instance on what you're trying to do - your box looks pretty innocuous.
Been there, done that, paid for the T-shirt
and didn't get it
Dear Sir,
Because no one trusts a hardware engineer.
Sincerely,
A Software Engineer
Could you clarify exactly what a 'consumer market approval' is? Is it done in house by the company making the product or by a third party institution? Are there generally accepted standards for the process or does each reviewing group have their own procedure?
Could you imagine how much this would slow down the development process? If you had to get *approval* for the release of every new bloody version of a piece of software? (Not to mention patches, auto-updates, etc...) (Also not to mention how much time you'd have to spend simply doing your homework to ensure compliance!)
This has been suggested before, and is a *very bad idea*. It is tolerable for things like drugs and nuclear power where a mistake could injure or kill people. Outside of such high-risk things, this kind of regulation should be avoided like the plague.
Well, first off, the question is misguided -- software development usually does involve consumer testing and feedback at every stage of the process (at least, good software development for a specific user-client; the user never wants what they tell you, nor do you build exactly what they tell you anyway).
More to the point, though, a lot of commercial software would be loads better if it had a more thorough testing process. But this would result in such poor times-to-market that the market would've already been cornered by the piece of crap that was released first and patched in the upgrade.
So yes, this would result in better software, provided you don't mind hamstringing the developers (with tons of new user requests) and the sales staff (when they have a product they can never, ever deliver on time).
Incidentally, sometimes the end-user's ability to use software other than it was exactly intended can be useful, to a sufficiently creative and powerful user... for over-the-counter commercial software anyway...
Freedom isn't free; its price is the well-being of others.
This should probably just be added to the beta testing that most programs go through. Rather than doing it inhouse, simply give it to a bunch of average Joe Computerusers and see what they do with it. A few of them are bound to do something stupid with it, and that's when you fix the bugs there. There's your consumer approval. If an average consumer can use it without breaking it, then it's fit for the market.
Wouldn't that make it harder for open applications to get in industry? Who would pay for the validation?
Most small video game makers have to run there final versions by the box makers (Sony and Xbox). They run it through a bastion of tests before they will let it out to the consumer market. It doesn't seem to harm the video game makers ability to create good games. Of course, this doesn't include usability testing.
______________________
here's what I meant to write
It would certainly help usability. If you extend the analogy of unsuitable use of hardware to software, what if I click the wrong button or enter an illegal command. This should all be handled by good software.
The problem is that software producers (we can all think of one we hate) are in a rush to make more product and to release new versions. And that rush goes against the idea of quality. In a sense, the software has to be just good enough to get a user's money.
But if that process ever became standard, it might help quite a bit with security. Throw in some bogus data and see if anyone can read it or write to it illegally.
Ultimately, this will never happen unless users demand it, and refuse to buy a product unless it passes such a test. And I don't know if that will happen.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
Although, software can destroy 20 years of business data and bring down the whole company anyway, but it's easier to mirror/backup your data than it is to mirror/backup your house/office.
The FCC/CE wants to make sure that your widget doesn't interfere with the other widgets. UL/CSA wants to make sure your widget doesn't burn the house down.
I know that CE has some EMI susceptablilty stuff that isn't exactly safety, but for the most part, the issue is making a safe, non-interfering widget. The widget could fail in 2 days, as long as it fails safely.
You are posing a question that is pretty much unrelated to hardware approvals.
"Eve of Destruction", it's not just for old hippies anymore...
the only way to ensure the creation of better software is to destroy capitalism, thus taking away the profit motive for the speedy creation of software. once there is no money in it, only smart, skilled, and creative people like Dennis Richie or RMS will actually bother to make software. Only the creation of a Socialist Republic a la Seamus Costello and Malachy McAllister, James Connolly, or Karl Marx will create better software, true freedom, and a peaceful world.
Having been on both the hardware and software side of the business, the reason the hardware side goes through so many certifications and steps is purely financial. Building the board the first time is not just the same amount of logic but also checks for interference, electrocution, MTBF and usability. All this takes additional time and investment. If you produce a bunch and stuff goes wrong, fixing it is costly but you can also hurt people. Even liability insurance is more expensive because an inert CD just can't do much damage, but a loose wire can kill.
Every step of hardware is carefully vetted because mistakes (and even success) are so expensive. That, in my opinion, was the huge benefit of computers: they can adapt to your needs by loading cheap software.
{software manufacturer holds fingers in ears}
La La La...I can't hear you...La La La...
*whup* "Get along, little electrons. Heeyah!"
Not so with software. You can't know what hardware the end user will use. You can;t know every little idiosyncracy of every private network on the planet. You can't cover every edge case. Standardized testing like hardware can be put through is far less meaningful in such an environment.
This is not to say that testing, particularly thotough and thoughful testing is not desirable, I just suspect that it takes something other than a cookie cutter approach to test software thoroughly.
"Talk minus action equals nothing" - Joey Shithead, D.O.A.
"Talk minus action equals
This will not make software better, only make it harder for free software developers and small software companies. Will slow the development process and add huge expenses.A way for big corporations to controle the software market.
You need approval for consumer hardware so that you don't kill people.
Bad software may have driven people to suicide
but I don't imagine there is any precedent for
changing the rules for software.
You can't (unless it's software that has a
real 'life-or-death' aspect) compare the
requirements to hardware certification.
Where such certification is required, the
software is produced by companies with big
bucks to invest and customers who are prepared
to pay what is costs to produce certifiably
good software.
Before anyone else jumps in; I know 'certifiably
good software' may be a pipe dream, no matter
what you pay for it, but that does not dismiss
the argument!
It's hard to test software for every possible use of it. Maybe for very specific propriatary software you can test every possible use and senario, but try testing everything you can possibly do with windows. Though you could force a company to test the hell out of specific uses of the software, like security, and possibly get certain aspects improved while other less essential areas just slide on by. However, this will never happen, as it would slow down the release and development of new software. Software companies can dump a lot of money into politicians and their campaigns to make sure they don't pass legislation making this kind of review mandatory for software. Look at cigarrettes. Their lobbyers have kept a deadly product with no redeaming value on the market, and it's less strictly regulated than meat or milk. Money buys everything, and Bill Gates has a lot of it.
Software Testing/Quality Assurance is supposed to perform this function.
The problem is often insufficient tools.
The company I work for as a Software Test Automation Specialist is looking at WorkSoft Certify and we like what we see, except the price-point (approximately triple our current tool: Rational Robot), however, that is currently in negotiations.
This user account is inactive account replaced by the PDA
While it would of course be nice to have software without errors, the problem then becomes price and time to market. There is a saying in the project management world, "Speed, Price, and Quality. Pick two of the three." I've found this saying to be pretty accurate.
As consumers we tend to want everything now, and cheaply. This would obviously push down the quality of the product. Being an impulse buyer myself I find most products pretty much suck these days because manufacturers (of software or hardware) know that we want everything now and cheap, so they don't focus on quality at all, just time to market. I'm of course exagerating a bit, but it does seem consumerism kills quality.
Safety critical software such as aircraft software has to be highly checked and certified. The problem with software which does not happen with hardware is that a a change which is supposed to only fix problem (a) means a recompilation, and there is always the chance that function (b) has been screwed up, so generally we have to restest just about everything, whatever the change. It costs much more to test software to a good standard, than to develop it. We reckon that the minimum change for engine control software will take 6 weeks of 18 hours per day testing to validate. And we DO find unrelated faults, which occur occasionally.
***You learn something Every day. And then you die.***
It depends what you mean by "market approvals." If you mean mandatory, FDA / FCC / FTC style neck-stomping (which is what it sounds like you mean), then consider these actual responses from the Council of Wise Men, circa 2039 (they fell through a small time-warp, one of the many features in Gnome 6.2, which came out the previous year).
- "Your window manager isn't friendly enough to people with one hand and colorblindness. Sorry, you'll have to try again before you can legally release it."
- "Your human interface guidelines vary from ours. Sorry, you'll have to make yours conform to ours, or file a request for an initial variance hearing to take place within 90 days; at that juncture, an administrative board will determine whether a variance will be considered, and may at its option propose alternative remedies."
- "Your word processor saves in a format that's different from the Officially Approved Standard v1.39c (revised), and does not save into one of the other previously approved formats. Since it's new, you can't claim grandfathering protection either. Sure, you claim it's a transparent, XML-based, human-parseable format, but rules is rules. Sorry, you'll have to have your software re-evaluated by The Committee."
- "This game features images we think are offensive. You'll have to revise them before this can be released. Protection of children, dontcha know."
Trying to narrow this question to "commercial software" is a difficult task, too: remember, software can be written by people who don't program for a living, open source / Free software can be sold (and is therefore commercial, though distinct from the current conventional closed-source software business), and software not intended as "commercial" (is perl commercial? Not per se, it isn't) is often used in commercial settings.
Do you really want to limit the field of software to those pieces of software which have passed a bureaucratic approval system? Or to programmers who have buckled under and agreed to some imposed vision of software design? A lot of very nice open source software improves primarily by being tested (read "dragged through the mud") while in its raw state. Some of it might even be very useful in early stages, no matter how ugly it is, and since there's no accounting for taste, I would take umbrage at any particular list of requirements that tried to determine in advance how software should act. (Emacs?)
I like the fact that computers are flexible, and there's a robust, heterogeneous environment with lots of languages, security models, development styles and programmers. No system of centralized control *with the force of law* will do anything but weaken this.
On the other hand, there's plenty of room for voluntary, peacable information sources that do nothing but provide informative ratings, review compliance with currently conventional / acceptable standards, etc. Consumer Reports, Underwriters' Laboratories, Good Housekeeping Seal of Approval. This is also something Insurance companies do, and a reason that there's "hacking insurance" as featured on Slashdot a year or two ago. If a business cares to heed, or to act on, any of these sources' advice, they're free to and it may benefit them in the long run. I certainly don't want products to require the Good Housekeeping Seal, though.
[heart on sleeve]
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
I am a Controls Engineer. One of my duties is to write software for automatated manufacturing equipment. I am going to have to do FDA validation on my software for functionality, mostly for faults and fault recovery.
my $.02
I know what the Internet is, what the hell is this Interweb business?!
Hardware testing is done because there are a number of Government regulations that require it (FCC, CPSC), product liability requires it, and the common law treats hardware just like any other property. After those minimums are met, the free market kicks in, driving toward better reliability/lower cost/pretty design - whatever the market wants at a given moment.
Software is a new animal, and neither the government nor the common law has caught up with it. (Hollerith cards @ 100 y/o vs. the Code of Hammurabi @ 1000's y/o.) As a result, software makers have had free reign in the market, because there is no mechanism that sets minimum standards. Seemingly absurd licensing practices are not challenged because there are few on-point cases in the common law and our legislators simply don't have the mental prowess to see software as a different class of stuff - part real property, part intellectual property, part printed word, part device. Nor do they have any incentive to work the problem out.
One of 2 things must happen to get software on the same footing as hardware:
1) Legislative action addressing the fundamental nature of software and how the law will treat it (I personally favor killing both software copyright AND patents and coming up with a 3rd classification)
2) Bold precedents in the common law to extend existing legal concepts to the current situation - unconscionable clauses applied to EULA's, detrimental reliance, or recongizing tort claims by users against software makers.
Either one will take someone, either judge or legislator, with some real balls. Other than that, don't hold your breath.
"As God is my witness, I thought turkeys could fly." A. Carlson
...unless that software controls the confinement ring in your homemade fusion reactor....
Winged Power Photography
Indeed:
.
Your questions is the kind of question that:
A) Children in their wide-eyed innocence would ask. or
B) Blooming genius would ask out of fortituded and courage and be shunned
I beg you to take the compliment that A) and B) bestow upon you.
To answer you;
your simple question begs a complex answer,...here goes,...
Hardware is a physical commodity whose use is subject to, The laws of thermodynamics, the Law of Gravity , the restrictions of the Laws of Entropy and choatic disorder, laws of motion, etc. in truth, anything that is itself physical matter or electromagnetic energy may interact with this physical object and derange it into a source of physical harm to another physical object/person.
Hardware can touch you and is seen, felt, touched, etc., etc., etc...
Software can -CAUSE- physical harm, through function or malfunction, through placement or misplacement/displacment etc., etc. but is not the thing that , you guessed it, made actual contact.
Accident investigation can be expensive, yes? Yes!
But imagine the expense of PROVING that software was intended, or intentionally recreated or created, copied or reverse engineered to introduce fault that led to harm or catastrophic event. Multiply this expense at the end and beginning,..huh?
At the creation of said device and at its discovery at the scene of a mishap it must then be examined for possible even probable fault against a model that should predict said apparent failure EVEN THOUGH said model cannot completely emulate all possible causes of fault/failure. ( a hackers-black hat- code as opposed to a hackers - white hats- code) Each has their own style and methods to achieve a particular end. Modern software construction is not so strictly controlled/modelled that there is only ONE WAY, one predictable and inescapable way to make a function in cyber reality / software development - see what I mean?!? The ability to check software would require software codes of conduct WAY PAST Posix or any present idea of standards or conformity.
To prove something or test something we must have limits on what that ' something' can do or be even if it can do or be a lot of things,...software does not have that and proprietary software makers will spend megatons of money to make sure such standards NEVER come about until they( one company) owns the entire theater of software deveopment: Examp: Microsoft helped create the POSIX standard but their own software is not POSUX compliant,..sheeesh!
Idealogically speaking, it sounds like a very good plan. My problem with it is more political. Who comes up with these standards and polices them? Microsoft? The government? Its all pretty subjective and I could see alot of smaller projects (small companies, many open source software projets, etc) get crushed because of a large organization's interests. I'm scared to see who gets what control in enforcing Microsoft's "trusted computing." But that's a bit off topic...
The best model we can probably hope for is:
1) Have a well known peer review system where potential users of the software can see the advantages and disadvantages. I'd be happy even with a more centralized repository of professional reviewers.
2) Some maintenance and policing mechanisms so that the review system doesn't become flooded with disinformation
3) A software movement where more software (especially windows-based proprietary software) is designed using already existing and well tested software components and frameworks that can be scrutinized individually.
Nintendo Seal of Quality
Same thing could be applied to software.
Javascript + Nintendo DSi = DSiCade
The problem is in defining what exactly consitutes a GOOD approval process for any given piece of software. It's often easier to define this for hardware. You define proper operating ranges and how the thing should respond when used or abused in specific ways, and the result is often a product that will behave as expected in almost all realworld conditions.
In software the failure cases tend to be more open-ended. The set of all possible types of input to the system may involve infinite permutations. You can only test the ones you thought of, and if you thought of them they're probably handled correctly in the first place. If you're developing a commercial app you have to deal with the fact that the hardware and OS your program relies on may in fact be subtly flawed. Also, any set of tests for a piece of software must be custom designed for that piece of software.
Ideally software testing is more of a verification process than a corrective process. Your tests should (but rarely are) be created at the same time as your design and run continuously throughout the project lifecycle.
You can't easily patch hardware. The consumer:
with software once you identify the problem and fix it, the customer might be out of commission a half hour while the download, install, and possibly reboot the machine.
In the Windows world, there's always Windows Logo certification.
Of course, it doesn't guarantee that the software is especially useful or bug-free, it simply means that it follows certain user-interface standards (whether those "standards" are ideal is another question, but consistancy is important). Of course technically, Office 2000 and Media Player should not have passed the certification, but that's another story...
The point is, as others have pointed out: hardware testing is mostly about safety and interference concerns, none of which really applies in the software world (barring specific examples like medical, aviation, or RF software, which already are required to meet certain criteria).
I don't see any standard hardware certifications that could be applied to software (usability, design, functionality, etc). So I'm not totally sure what the OP is asking about...
Add to that, much of the time these days, hardware comes with software; be it firmware, drivers, or a full-blown OS and hard disk (set-top boxes, etc). And many times a very high-quality piece of hardware comes with a buggy, closed, crash-prone driver that makes the thing more useful as a paper weight. Or maybe I'm just bitter about my Lexmark X125...
If you want to know that a particular hardware device is a good buy, high-quality, easy-to-use... you either test it out yourself, or you find reviews from magazines or other sources you trust. You ask a friend who has one. Or you buy hardware from a company you trust, and avoid companies that you don't.
The above paragraph works equally well if you substitute software in place of hardware. Thus, I don't feel we need any standards-bodies (or much worse, any sort of mandated certification procedure) for software any more than what we already have, in those cases where it's life-critical.
NGWave - Fast Sound Editor for Windows
is strictly regulated by the FDA. Not only is a software company required by law to obtain premarket approval 510(k) from the FDA before marketing certain types of medical software in the US, but it is also required by law to document and follow a very thorough software development and validation process.
Although this kind of software is usually not sold to the general public, it is used every day in hospitals and clinics to do everything from analyzing bacterial infections to robotic surgery to radiation oncology treatment planning.
I have worked for several software companies, developing software that is considered a class II medical device. Not only did we have to completely document everything from requirements to validation testing, but we had to keep the documents themselves under version control! Knowing that your software could mean life or death to someone, really puts the software engineering process into perspective.
Sorry, I'll bite. That's all good and fine in theory, but in practice that's another story.
I do not regard stuff like a game crashing every half an hour as being caused by "unreasonable" use. Or for example: which of Fallout 2's many script bugs were "unreasonable" use?
I also do not regard stuff like "oops, the user used the back button in the browser" or "oops, the user opened a link in another window" on web sites to be "unreasonable" use. Use of bog-standard browser features, that have been around for more than a decade, _is_ reasonable.
It's the retarded ex-burger flippers who moved into software development during the dot-bomb that are unreasonable there. If Joe Coder can't use the HTTP session right (yes, including supporting multiple windows _and_ the back button) then the only "unreasonable" part is Joe Coder still being employed. Period.
Etc.
Basically I don't know about mandatory government testing, but I would very much like to see some legal responsibility that can't be waved away with an EULA. Some part that says that your responsibility is to the user, not just the current "hey, we only need to take their money. And then who the fsck cares if it works?"
I'd also like to see some legal responsibility for the marketroids, same as in any other industry. If you say that a piece of software does something, then it damn better do that, to the letter. Just like if a steel company's marketroid says "we'll sell you 10 ft long, 1 inch thick beams, with 0.1% carbon content", you can sue the pants off those guys if it's only 9 ft long and with a completely other carbon content.
And yes, I _am_ a software developper. It just makes me sick to see what this industry has turned into. It's the biggest snake oil operation in history. Hundreds of billions of dollars worth of snake oil every year.
And this doesn't come out of nowhere. It's draining the rest of the society to keep a bunch of cheats, liars and leeches in business.
A polar bear is a cartesian bear after a coordinate transform.