Microsoft Developing a Tool To Help Engineers Catch Bias in Algorithms (venturebeat.com)

Microsoft is developing a tool that can detect bias in artificial intelligence algorithms with the goal of helping businesses use AI without running the risk of discriminating against certain people. From a report: Rich Caruana, a senior researcher on the bias-detection tool at Microsoft, described it as a "dashboard" that engineers can apply to trained AI models. "Things like transparency, intelligibility, and explanation are new enough to the field that few of us have sufficient experience to know everything we should look for and all the ways that bias might lurk in our models," he told MIT Technology Review. Bias in algorithms is an issue increasingly coming to the fore. At the Re-Work Deep Learning Summit in Boston this week, Gabriele Fariello, a Harvard instructor in machine learning and chief information officer at the University of Rhode Island, said that there are "significant ... problems" in the AI field's treatment of ethics and bias today. "There are real decisions being made in health care, in the judicial system, and elsewhere that affect your life directly," he said.

Comments

Re:The bias of reverse bias

[frank_adrian314159]

The AIs will naturally be confused by being disallowed to latch onto the strongest signals in the data.

Uh not unless it's a really crappy AI. If you haven't noticed, chances are any human directive will be treated as that by the neural network - another signal that is larger/more salient because it is input by a human. Just the way that the system would be designed to do unless you want it completely independent of human control.

In short, don't project your own human confusion about neural nets onto the technology just because you don't like the implications of human control of machines.

Re:Except no

[bug_hunter]

Here's a more more interesting question:

Do you want a justice system that says:
For the crime of breaking an entering:
White person : 2 years
Black person : 4 years
Asian person : 1 year
etc

Do you imagine the groups on the larger sentencing of that spectrum having faith in the justice system?

What exactly is an algorithm bias?

[misnohmer]

I've been reading stories in removing bias from algorithms but still don't get it. What is an algorithm bias? If the results don't have perfectly flat distribution across sex, race, religion, and other protected groups?

Re: Except no

[phantomfive]

I've found that to be a problem in my attempts to make neural networks: too often a complex network can be simplified to just a few variables that, once found, can be hard coded. In some ways it's really depressing.