Slashdot Mirror

← Back to Stories (view on slashdot.org)

Microsoft Developing a Tool To Help Engineers Catch Bias in Algorithms (venturebeat.com)

Posted by msmash on from the a-fight-within dept.

Microsoft is developing a tool that can detect bias in artificial intelligence algorithms with the goal of helping businesses use AI without running the risk of discriminating against certain people. From a report: Rich Caruana, a senior researcher on the bias-detection tool at Microsoft, described it as a "dashboard" that engineers can apply to trained AI models. "Things like transparency, intelligibility, and explanation are new enough to the field that few of us have sufficient experience to know everything we should look for and all the ways that bias might lurk in our models," he told MIT Technology Review. Bias in algorithms is an issue increasingly coming to the fore. At the Re-Work Deep Learning Summit in Boston this week, Gabriele Fariello, a Harvard instructor in machine learning and chief information officer at the University of Rhode Island, said that there are "significant ... problems" in the AI field's treatment of ethics and bias today. "There are real decisions being made in health care, in the judicial system, and elsewhere that affect your life directly," he said.

6 of 239 comments (clear)

  1. Wrong Bias by Anonymous Coward · · Score: 3, Insightful

    Correctly read as: "Microsoft is developing a tool to help developers detect wrong bias in their algorithms."

  2. The bias of reverse bias by Citizen+of+Earth · · Score: 5, Insightful

    The main problem with this endeavor is that the "bias" they are trying to suppress is actually the opposite of bias. They seek to treat people differently on the basis of identity politics instead of on their actual behavior. The AIs will naturally be confused by being disallowed to latch onto the strongest signals in the data.

  3. Except no by bug_hunter · · Score: 2, Insightful

    From the article:

    Northpointe’s Compas software, which uses machine learning to predict whether a defendant will commit future crimes, was found to judge black defendants more harshly than white defendants.

    So that was an existing algorithm that judged somebody on how they were born rather than their individual behavior.

    --
    It's turtles all the way down.
    1. Re:Except no by AmiMoJo · · Score: 1, Insightful

      "Prior convictions" and "future convictions" are too simplistic.

      For example, getting a minor drug possession conviction is rather different to one for murder. And the system is known to be far more likely to give young black men convictions for minor drug offenses than it is to give them to older white guys, even when the crime and circumstances are identical.

      So we have a situation where the algorithm would need to understand the severity of each conviction, the circumstances in which it was given, and the bias that already exists which we have a desire to correct. That's something humans find difficult, let alone a relatively simplistic algorithm.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    2. Re:Except no by Chris+Mattern · · Score: 1, Insightful

      The COMPAS algorithm, while opaque, does not have race as an input. It was found its accuracy could be matched by an algorithm with just two variables: age and prior convictions.

      The joker in that is the "prior convictions." If there was bias in how the subject was convicted in earlier cases, then the algorithm will codify that bias.

  4. Re:Unbiased approach. by Anonymous Coward · · Score: 4, Insightful

    Eliminating Bias from AI means discarding facts and data that violate SJW principals.