Slashdot Mirror

← Back to Stories (view on slashdot.org)

We Hold People With Power To Account. Why Not Algorithms? (theguardian.com)

Posted by msmash on from the fair-question dept.

An anonymous reader shares a report: All around us, algorithms provide a kind of convenient source of authority: an easy way to delegate responsibility, a short cut we take without thinking. Who is really going to click through to the second page of Google results every time and think critically about the information that has been served up? Or go to every airline to check if a comparison site is listing the cheapest deals? Or get out a ruler and a road map to confirm that their GPS is offering the shortest route? But already in our hospitals, our schools, our shops, our courtrooms and our police stations, artificial intelligence is silently working behind the scenes, feeding on our data and making decisions on our behalf. Sure, this technology has the capacity for enormous social good -- it can help us diagnose breast cancer, catch serial killers, avoid plane crashes and, as the health secretary, Matt Hancock, has proposed, potentially save lives using NHS data and genomics. Unless we know when to trust our own instincts over the output of a piece of software, however, it also brings the potential for disruption, injustice and unfairness.

If we permit flawed machines to make life-changing decisions on our behalf -- by allowing them to pinpoint a murder suspect, to diagnose a condition or take over the wheel of a car -- we have to think carefully about what happens when things go wrong.

24 of 143 comments (clear)

  1. Ultimately, a human should be held accountable by Anonymous Coward · · Score: 5, Insightful

    If a prosecutor or judge uses an algorithm to set sentencing or determine parole, the individual prosecutor or judge should still be held accountable if he was in error.

    This applies if the algorithm is a "paper and pencil" fill-out-a-worksheet algorithm or if it's a complex computational algorithm that the judge or prosecutor can't understand. In the latter case, if the judge or prosecutor can't understand the tools he is using, perhaps he should use less sophisticated tools that he does understand.

    1. Re:Ultimately, a human should be held accountable by jellomizer · · Score: 4, Insightful

      This is sometime I keep on trying to state at work.
      This program will help you find things easier. However I cannot program judgement and years of experience, even a learning algorithm may not have data because some things are not recorded.
      Customer X has been a good customer for years, however this month he is behind, this customer actually called the company and let them know that. The computer algorithm will see the late payment, and perhaps send it collection, it doesn't care about the long term relationship.
      An algorithm should be allowed to run, however a human is ultimately responsible to but a stop to an action.

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
    2. Re:Ultimately, a human should be held accountable by demonlapin · · Score: 2

      I’ve seen this in action. A locally-owned shopping center was sold to a large retail management company a few years ago. They immediately started pushing out the locally-owned businesses and filling the spaces with national retailers. Why? Big companies don’t ever miss the rent. I used to go there at least once a month; I haven’t been to a single one of the new places. It struck me as a penny-wise, pound-foolish move, but I guess they know their business.

    3. Re:Ultimately, a human should be held accountable by Hasaf · · Score: 2

      The flaw I see in your reasoning is that, frequently, the users of the algorithms are people at the bottom of the chain, not those in power. Certainly a judge should be up to date on sentencing guidelines; but what about the clerk in his office who was told to print out the standard sentence for the case under discussion?

      A better example would be the truck drive in a automated truck. That driver had no say in the the choice of the algorithm. To hold that driver responsible instead of the people who made the decision to use that particular algorithm.

      Your suggestion can easily turn into another case of holding the least powerful person in the chain responsible while insulating those who make the decisions from the results of their decisions.

    4. Re:Ultimately, a human should be held accountable by jellomizer · · Score: 2

      Big companies do more then miss the rent they just don't pay it, and wrap the property owner in idle threats of lawsuits because of whatever issue they may find.
      Don't bother trying to fight back, it will cost your more in legal fees to fight back. And your politician will turn a blind eye to you because this big company brings in jobs Jobs JOBS!!!!!

      --
      If something is so important that you feel the need to post it on the internet... It probably isn't that important.
  2. Since when? by pablo_max · · Score: 5, Insightful

    Seriously, the "powerful" have not been help accountable for fuck-all in the US since after the 50's.

    Maybe you should rather start with that before you starting picking on maths.

    1. Re:Since when? by Luthair · · Score: 5, Insightful

      I think you're optimistic, the only time in human history when people in power are held to account is when they were lynched by a mob.

    2. Re:Since when? by lgw · · Score: 5, Insightful

      They were accountable before the 50s?

      The only thing that ever held those in power accountable was competition from others with power. For most of the medieval period, for example, church and state each limited the excess of the other. Local Baron get to evil with his serfs? Local clergy would call him out on his immorality, even if they were total hypocrites, in order for the church to gain power at the expense of the secular authorities (and vice versa). When the balance of power tilted too far towards the church in the late medieval/early renaissance, we got the Inquisition, as there just wasn't enough secular power to challenge that.

      You can see similar balances of power throughout history, usually between religious and secular authorities.

      Right now we have an entirely fictitious "competition" for power between government and large corporations. But that's all a fraud to deceive voters: they together form the Establishment, all pro-mega-corp all the the time. Voters get a false choice between "more regulation" and "freer market", but that's all bullshit because there are only foxes guarding all the hen-houses.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    3. Re:Since when? by MachineShedFred · · Score: 4, Insightful

      I was going to say the same thing. I don't accept the premise of the headline, due to absolutely nobody of consequence being held accountable for anything of consequence for the financial meltdown / subprime mortgage fiasco. Once we actually start holding people accountable for things, we can then worry about evil bits being set in registers.

      --
      Slashdot still doesnâ(TM)t support Unicode after it was added to the HTML standard in 1997.
  3. We hold powerful people to account?! by Bugler412 · · Score: 4, Insightful

    That's news to me

  4. a "report"? by cascadingstylesheet · · Score: 3, Insightful

    This is an editorial.

  5. Algorithm Jail by goombah99 · · Score: 4, Funny

    Yes I have a folder I keep all the naughty algorithms in. If they escape I erase their stacks. Real death

    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:Algorithm Jail by DamnRogue · · Score: 2

      Sentence the algorithm to death. Train it with new data so the mistake is not repeated, and the offending version never activated again.

    2. Re:Algorithm Jail by alvinrod · · Score: 2

      Are you certain you won't just wind up with an algorithm that is good at lying to you, shifting blame for its mistakes, or other acts of subterfuge? The algorithm doesn't even need to be conscious of its actions to do those things. If you're using genetic algorithms you might want to be careful of what kind of engineering you're really doing.

  6. uninformed point of view by iggymanz · · Score: 3, Interesting

    The software in life critical and safety systems indeed is already held to account. Conviction for a crime requires humans who review evidence and its veracity. Privacy depends on your lawmakers, some countries have a mindset of respecting it, others don't and wont' throwing the buzzword "AI" into a sentence doesn't change the problem that it's about software in general, whether or not a marketer slaps "AI" label on it

    1. Re:uninformed point of view by mujadaddy · · Score: 4, Funny

      uninformed point of view

      Guardian tech story

      Pick two.

      --
      Populus vult decipi, ergo decipiatur...
      "Force shits upon Reason's back." - Poor Richard's Almanac
  7. let's be precise by ooloorie · · Score: 3, Interesting

    We Hold People With Power To Account. Why Not Algorithms?

    We don't generically "hold people with power to account". The law requires that legally competent adults comply with legally binding agreements they have entered (employment contracts, etc.), and the law punishes criminal behavior. Legal competency requires free will and agency, neither of which "algorithms" possess.

    If we permit flawed machines to make life-changing decisions on our behalf

    The ABS braking algoritihm in your car makes "life-changing decisions" you your behalf. It is you (the car's owner) and the manufacturer who are responsible for those decisions, depending on circumstances.

    On the other hand, doctors make life changing decisions all the time, frequently get it wrong, and frequently are not held accountable. Nor should they be: when you make life changing decisions with limited information, you often get it wrong. That's not a flaw, that's life.

  8. Ultimately, humans ARE responsible by argStyopa · · Score: 4, Interesting

    In the OP's posted story, Robert was the dumb fuck that almost drove off a cliff.
    You cannot hold algorithms accountable, they're NOT PEOPLE. They cannot be punished. They don't feel remorse.

    All we can do is to explicitly build a legislative system that follows the trail back to the human that gave the algorithm that power.

    If Bob is driving a car, it's STILL Bob's responsibility to watch to damned road.
    If Bob is sold a self-driving car with the written assurance from the dealer that this car will drive itself in conditions a, b, and c, if Bob gets killed during a, b, or c, ultimately the dealer is liable at LEAST for manslaughter, worse if they knew it wasn't perfected.

    If the dealer was assured by the manufacturer, then the manufacturer is responsible. I would even say all the way to personal liability the person or group of persons who signed-off that this *was* capable.
    Don't like that risk, Mr Auto Executive? Then don't sign off that X is safe until you're willing to take that risk.

    (And I don't know if I'm just excessively cynical, but I don't see a lot of "holding people with power" to account EITHER. Hell, I don't see that holding ANY people to account - even for the logical consequences of their OWN CHOICES - is much of a priority in our society.)

    --
    -Styopa
  9. We DON'T Hold people with power accountable by Archangel+Michael · · Score: 5, Interesting

    We rarely hold people with power accountable. Instead, decisions are made in committee and by a series of processes that obfuscate and remove culpability of decisions from those that are authorizing it.

    Lets say to do Action A, it requires approval by several committees or individuals. We'll keep it simple.

    Committee 1 votes to do Action A (sounds like a good idea)
    Person 1 checks to see if Action A(b) violates some metric (it doesn't)
    Person 2 checks to see if Action A(c) passes certain functional tests. (it does)
    Person 3 verifies results of tests A(b) and A(c) (it does)
    Committee 2 finalizes approval of Action A

    Action A causes massive death due to Action A doing something nobody checked for. There is no person responsible for this, it was just a bad "accident". However, looking back, Action A was a bad idea from the start, but it passed all the tests. Nobody is responsible.

    Hillary can say with clear conscience that her signature on Uranium One deal was only one of 17 required, it isn't her fault. Even though the sale of Uranium to the Russians was stupid idea, no one person can be blamed. No accountability. The buck stops in committee.

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  10. First, we do not by gweihir · · Score: 4, Insightful

    Be powerful enough and you can commit almost any crime and get away with it. Second, you cannot hold an abstract concept accountable.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  11. We do? by Anonymous Coward · · Score: 2, Insightful

    When was the last bankster to go to prison?

  12. Maybe not Re:Algorithms are just an excuse by davidwr · · Score: 2

    A person designed those algorithms.

    More and more, algorithms are designing or at least re-designing algorithms.

    In some systems, we are so far removed from "the person" that no one person could possibly understand the system in any reasonable period of time. By the time he did understand the system, the system would have very likely been decommissioned as obsolete or it would have re-trained itself and no longer be what it once was.

    --
    Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
  13. Premise fail by RonVNX · · Score: 2

    We _do not_ hold people with power to account. So this is a nonsense question.

  14. Algorithms aren't about perfection... by zarmanto · · Score: 2

    People often look to the algorithms of things like a GPS navigator or a news aggregator with the notion that it's always going to spit back results which are somehow "better" than whatever a human could have come up with... and to a certain extent, that might even be true. The thing that we don't always bother to ask is, what specifically did the human programmers of that algorithm decide to define as "better"? In some cases, it's a matter of what's least expensive, because that's what the consumer/end-user wants. In other cases, it's a matter of what's most profitable, because that's what the "real" customer wants. And in a few cases, "better" could easily be nothing more than taking that whole damned decision making process out of the end-user's hands, just so that they don't have to think about it.

    Take GPS as an obvious potential example of this latter scenario: In many cases, there are a myriad of different possible routes which will all get you to the same destination in roughly the same time-frame -- barring obvious slowdowns, like a major accident on one of those routes. If you happen to know several such routes yourself, try testing your GPS: go "off route," and see what happens. I've conducted this test myself a few times, in one instance even going off route multiple times over the course of a drive... and the GPS happily rerouted and recalculated the estimate time of arrival each and every time... and outside of taking an obviously ridiculous route, the GPS's ETA only rarely extends beyond a few minutes different from the very first ETA that it had offered me, at the beginning of my trip. And yes... now and then, I can even manage to beat the GPS's estimate. (Your mileage may vary, and all that good stuff.)

    So it's not always about getting the algorithm to help you find "the best" option... sometimes it's just about making a decision, and running with it. The same paradigm could easily be applied to many other decisions that we make in life. It hasn't been pushed quite this far yet, but consider: "Should I wear my blue shirt with tan slacks today, or the red shirt with black slacks?" "Should I have Moe's for lunch, or Chick-fil-a?" "Should I wear Old Spice or Ax, today?" "Boxers or briefs?" "Straight tie or bow tie?" Ohhhhhhh, the decisions!

    Now, these are of course pretty far outside of the norm... most of us can usually come up with our own answers to these common everyday decisions. But that's just a few minor examples of the direction that things could go, once the machine has been supplied with enough of the right (?!?) data. And mark my words: if you can find a decent way to make the machine do it, there will be an audience willing to pass off even these minor decisions to the "wisdom of the machine."

    And why not? After all, making decisions is, in-and-of-itself, just one more piece of stress in our lives. And who needs unnecessary stress... right?