Slashdot Mirror
Ask Slashdot: Developer Responsibility When Apps Might Risk Lives?
First time accepted submitter bashaw writes "What ethical responsibilities do software developers have in determining the role that mobile devices take in our lives? As performance increases, size decreases, and the only limitation is the software available, mobile devices have expanded into new areas of our lives for which they were not designed. This raises the ethical question of who decides what software is available, and therefore what role these devices should take. I am a software developer at the Canadian Avalanche Centre. We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver. When it is a question of public safety, does the onus fall on the developers, a government agency or the users themselves?"
does the onus fall on the developers, a government agency or the users themselves?
Yes.
On the one hand, we can crack down hard on anyone who tries to even hint at some medical or safety purpose for a particular app. On the other we can be wild and free-booting and allow people into precisely the sort of trap that the poster outlines.
These apps may well be better than nothing (though they are not tested in any meaningful sense, nor are they compliant in any meaningful sense), but to the extent that they give a false sense of security, they are dangerous.
Personally, I lean towards crystal clear disclosure, and, in Canada, and restrictions on marketing. I do not favour an outright ban, since I could see that as having unpleasant consequences.
Look forward ten years. Suppose my smartphone has a ~90% reliable software and sensor package to tell me if I'm suffering from a heart attack. Suppose also that I'm part of a demographic group that by gender, age, fitness, weight, diet is highly unlikely to be suffering one. (There have been cases before where software has successfully diagnosed heart attacks in situations where physicians didn't believe it -- consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack).
It would not make rational sense in that case for me to purchase a $1000 bespoke medical device to monitor me, but a $5 app might make sense even if it wasn't as reliable.
Similarly if I ski only occasionally and in areas highly unlikely to suffer an avalanche, it might make sense for me to not purchase a transceiver. (For those who say they'd spend anything to protect their lives, even on extraordinary low probability, I suspect you may have some irrational optimizations in your life.)
Offering consumers informed choice seems key; if they are marketing their apps as the equivalent of Avalanche transceivers, that clearly is not informed choice.
Similarly, I'd pressure Google and Apple and Blackberry to come up with a common standard for fine grid device location that these apps could use.
The OP raises some interesting points; I still come down somewhat on the libertarian side of things.
The headline reminded me of a story in a book of mine:
When Brunel's Ship the SS Great Britain was launched into the River Thames, it made such a splash that several spectators on the opposite bank were drowned. Nowadays, engineers reduce the force of entry into the water by rope tethers which are designed to break at carefully calculated intervals.
When the first computer came into operation in the Mathematisch Centrum in Amsterdam, one of the first tasks was to calculate the appropriate intervals and breaking strains of these tethers. In order to ensure the correctness of the program which did the calculations, the programmers were invited to watch the launching from the first row of the ceremonial viewing stand set up on the opposite bank. They accepted and they survived.
This post contains no rudeness or derision of any kind. All arguments are friendly. Terms and exclusions may apply.
Well technically speaking, it would be likely the fault of the specific Government Consumer Protection Authority, whose jobs it is to monitor claims about products and if it finds them false, seek fiscal redress for the risk it puts the public too and ensure the public are warned. The end consumer should never really be put in this position because reality is the only find the failure when they try to apply the product and seeking legal redress can be all too late. I find it all too annoying to get product after product that fails to achieve the levels of performance claimed and really like the idea of an agency that puts the breaks on this, by bankrupting deceitful company after deceitful company as well as those companies executive teams.
Chaos - everything, everywhere, everywhen