Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: Developer Responsibility When Apps Might Risk Lives?

Posted by timothy on from the gps-makers-have-been-trying-to-kill-me-for-years dept.

First time accepted submitter bashaw writes "What ethical responsibilities do software developers have in determining the role that mobile devices take in our lives? As performance increases, size decreases, and the only limitation is the software available, mobile devices have expanded into new areas of our lives for which they were not designed. This raises the ethical question of who decides what software is available, and therefore what role these devices should take. I am a software developer at the Canadian Avalanche Centre. We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver. When it is a question of public safety, does the onus fall on the developers, a government agency or the users themselves?"

19 of 100 comments (clear)

  1. Developer or publisher? by gnasher719 · · Score: 3, Interesting

    Unless someone shows otherwise, the apps mentioned seem to do what the software developers who created them made them do. But the publisher of these apps tries to sell them for uses that they are not fit for. That's the publisher's problem, not the developers'.

    1. Re:Developer or publisher? by rtb61 · · Score: 5, Informative

      Well technically speaking, it would be likely the fault of the specific Government Consumer Protection Authority, whose jobs it is to monitor claims about products and if it finds them false, seek fiscal redress for the risk it puts the public too and ensure the public are warned. The end consumer should never really be put in this position because reality is the only find the failure when they try to apply the product and seeking legal redress can be all too late. I find it all too annoying to get product after product that fails to achieve the levels of performance claimed and really like the idea of an agency that puts the breaks on this, by bankrupting deceitful company after deceitful company as well as those companies executive teams.

      --
      Chaos - everything, everywhere, everywhen
    2. Re:Developer or publisher? by fuzzyfuzzyfungus · · Score: 2

      The ones who lied? Honesty isn't a 100% ironclad defense, there are things you can do with full disclosure that are still unethical; but if I program 'tweet_while_U_die' a program that uses your accelerometer to detect a probably-fatal vehicular collision and then tweets about it for you while you bleed out, I'm in poor taste; but hey, it is what I say it is.

      If somebody brands it as 'Personal Safety Notifier Pro' and insinuates that EMS dispatch is somehow going to find you based on this behavior, I'd like to see them charged with negligent homicide each time that fails to happen.

    3. Re:Developer or publisher? by John.Banister · · Score: 3, Interesting

      While I generally agree with you, I think it's worth looking at the example given by electronic chart programs. They all make you click on a "The prudent mariner will have properly updated paper charts" notice on startup. Once they learned about the publisher's advertising, the developers could make a "notice to idiots" one has to acknowledge on startup saying something like "99.9% of phones made in 2013 don't have the hardware to broadcast on the 457kHz avalanche transceiver band, and this app doesn't trigger that radio in any phones that do have that capability." You could make an option to turn the notice off buried in a setting submenu that idiots won't make the effort to find, but then it'd be good to make enabling the notice part of "activation" to circumvent a publisher selling the program with the notice disabled by default.

  2. Users by Anonymous Coward · · Score: 2, Interesting

    App did not warn me about tornado.
    Seriously, people have to take responsibility for their own choices.
    We're too litigious nowadays; we ought to set the standard that grownups are required to think.

    1. Re:Users by CannonballHead · · Score: 2

      You buy car. The car says it has brakes. The car only has brakes that work, though, when you are going less than 20mph.

      Bummer for you, I guess. You should have known better. You should take responsibility for your own choice to buy that car! Why didn't you get under there and check the brakes thoroughly first? What, you want to sue? Everything they said was completely true, the car has brakes...

      Admittedly, this is a silly and exaggerated example, but I personally have no doubt that a lot of advertising does this.

      The problem isn't necessarily that people are stupid now. The problem is that things are more complex. It's not like most of my purchases are ... well, animals or food.

      And even food is hard, now, since we do such complex things to it and with it... heh.

  3. All by ebno-10db · · Score: 4, Informative

    does the onus fall on the developers, a government agency or the users themselves?

    Yes.

    1. Re:All by sumdumass · · Score: 2

      One of the problems is that how do you know those facts? Of course you are not a dummy and looked into it a bit and applied knowledge you probably already possessed, but have you done so before this story came about?

      Well, suppose you have looked into it and when you did, you saw an app for your phone that said it could send help if ever in an avalanche. If you were an average consumer, would you know based on that exchange that the app couldn't be a substitute? Forget what you know about phones and think like a normal person who's geekiest moment might be watching The Big Bang Theory or reruns if Weird Science.

      I know people who are contract lawyers, who build million dollar homes, who see millions of dollars pass through their hands daily as an investment banker, and a lot of other people who would otherwise seem intelligent by most standards but can't figure out the reason their computer isn't turning on is because they kicked the power strip with their feet and unplugged it or switched it off. One thing I learned a long time ago, what is obvious for some will not be for all. People are just different. It is what makes the world an interesting place.

  4. Re: The Onus by EstherGretel · · Score: 2

    The developer of the software has no say in release schedules, etc. You're way off dude...C level execs are the ones responsible...and the ones making all the money.

  5. Certainly an increasing danger. by Holmwood · · Score: 5, Insightful

    On the one hand, we can crack down hard on anyone who tries to even hint at some medical or safety purpose for a particular app. On the other we can be wild and free-booting and allow people into precisely the sort of trap that the poster outlines.

    These apps may well be better than nothing (though they are not tested in any meaningful sense, nor are they compliant in any meaningful sense), but to the extent that they give a false sense of security, they are dangerous.

    Personally, I lean towards crystal clear disclosure, and, in Canada, and restrictions on marketing. I do not favour an outright ban, since I could see that as having unpleasant consequences.

    Look forward ten years. Suppose my smartphone has a ~90% reliable software and sensor package to tell me if I'm suffering from a heart attack. Suppose also that I'm part of a demographic group that by gender, age, fitness, weight, diet is highly unlikely to be suffering one. (There have been cases before where software has successfully diagnosed heart attacks in situations where physicians didn't believe it -- consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack).

    It would not make rational sense in that case for me to purchase a $1000 bespoke medical device to monitor me, but a $5 app might make sense even if it wasn't as reliable.

    Similarly if I ski only occasionally and in areas highly unlikely to suffer an avalanche, it might make sense for me to not purchase a transceiver. (For those who say they'd spend anything to protect their lives, even on extraordinary low probability, I suspect you may have some irrational optimizations in your life.)

    Offering consumers informed choice seems key; if they are marketing their apps as the equivalent of Avalanche transceivers, that clearly is not informed choice.

    Similarly, I'd pressure Google and Apple and Blackberry to come up with a common standard for fine grid device location that these apps could use.

    The OP raises some interesting points; I still come down somewhat on the libertarian side of things.

    1. Re:Certainly an increasing danger. by fuzzyfuzzyfungus · · Score: 2

      I think that there is one additional factor to consider (that does strongly affect this case; but might be weak or nonexistent in others): In the case of an "Avalanche Transceiver", that's not a generic description of any radio beacon designed for avalanches, it's a fairly specific set of standardized minimum capabilities and interoperability characteristics, recognized by both hardware vendors and significant mountaineering organizations.

      Even the insinuation that you are talking about a similar thing edges pretty close to misrepresentation, because of the degree to which the existing device has been codified.

      In a situation where the app's function is novel, and there is either no consensus at all, or a consensus so obscure as to be confined to some very specific subgroup, it would be much easier to make 'almost as good and a lot cheaper' style claims without edging on seriously misrepresenting the situation.

      If the market for avalance transceiver devices was either a total incompatible clusterfuck, or simply didn't exist, my position on these would be 'the more the better', since the implicit promise would be less, and the existing 'standard of care', so to speak, much closer to being equivalent. As it is, though, they would have to market...very...carefully to not be treading on thin ice in terms of implicit misrepresentation.

  6. A short anecdote by arielCo · · Score: 4, Interesting

    The headline reminded me of a story in a book of mine:

    When Brunel's Ship the SS Great Britain was launched into the River Thames, it made such a splash that several spectators on the opposite bank were drowned. Nowadays, engineers reduce the force of entry into the water by rope tethers which are designed to break at carefully calculated intervals.

    When the first computer came into operation in the Mathematisch Centrum in Amsterdam, one of the first tasks was to calculate the appropriate intervals and breaking strains of these tethers. In order to ensure the correctness of the program which did the calculations, the programmers were invited to watch the launching from the first row of the ceremonial viewing stand set up on the opposite bank. They accepted and they survived.

    --
    This post contains no rudeness or derision of any kind. All arguments are friendly. Terms and exclusions may apply.
    1. Re:A short anecdote by tapi0 · · Score: 2

      Not to take anything away from the principle underlying the anecdote, but SS Great Britain was launched at the Bristol Docks with nothing of great note occurring (apart from getting wedged in the lock leading to the Avon)

  7. I write EMS/Police/Fire 911 Software by FrozenFrog · · Score: 3, Interesting

    I have my own company, with 1 business partner. We write software for 911 dispatch systems. Some of our clients require us to carry "Errors and Omissions" liability insurance, which costs us upwards of $15k a year. Along with with the insurance, we have a pretty detailed EULA agreement covering bugs, etc.

    If you're writing any kind of software that could directly affect the safety of others, insurance is a must.

    Frog

    1. Re:I write EMS/Police/Fire 911 Software by bill_mcgonigle · · Score: 2

      At least you're using insurance responsibly - does the company ever ask for any kind of audit?

      I quit a job at the largest medical center in my area because the PHB's were ignoring the advice of the technical staff and insisting on buying inferior database software that was going to cause medication errors. We estimated a rate of seven mix-ups per year (due to single phase commits among disparate systems) and the bosses calculated that it would be cheaper to settle the lawsuits than to do it the right way. Fortunately, that project failed and they outsourced the whole thing five years later (most of the original talent left).

      Going after app vendors is just the low-hanging fruit. Like busting shoplifters while letting the banksters ruin the economy. Oh, wait, then, nevermind.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
  8. avy rescue community already knows about this by david_bonn · · Score: 2

    This is from May of 2012...

    http://avyupdates.blogspot.com/2012/05/iphone-avy-beacon-app-jokes-on-you.html

  9. Professional Liability Insurance by Anonymous Coward · · Score: 2, Insightful

    It falls on you, assuming your license sucks, but just make sure your Professional Liability coverage handles that sort of thing. I make search-and-rescue alerting software for the aerospace industry, my premiums are about 33% more.

  10. The peddlers by jandersen · · Score: 2

    ...does the onus fall on the developers, a government agency or the users themselves...

    If by "the developer" you mean the company that sells the product, then the answer is clear: it is the developer that is responsible. The government can at best issue guidelines for how safe and fit for purpose a product should be, and it is not realistic to expect all users to understand the full ramifications of their choices, when it comes to things that are likely to be well outside their general competencies.

    On the other hand, if by "the developer" you mean the engineer who designed and coded the app, then I would say not so much, but it depends on the circumstances. In general, software is not sold directly by the developers; there is a business organisation between, that decides what to produce and how to sell it. They are far more likely to be the real culprits.

  11. National Society of Professional Engineers (NSPE) by mjwalshe · · Score: 2

    I. Fundamental Canons Engineers, in the fulfillment of their professional duties, shall:

    Hold paramount the safety, health, and welfare of the public.
    Perform services only in areas of their competence.
    Issue public statements only in an objective and truthful manner.
    Act for each employer or client as faithful agents or trustees.
    Avoid deceptive acts.
    Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession. http://www.onlineethics.org/Resources/ethcodes/EnglishCodes/9972.aspx