Slashdot Mirror
Microsoft Developing a Tool To Help Engineers Catch Bias in Algorithms (venturebeat.com)
Microsoft is developing a tool that can detect bias in artificial intelligence algorithms with the goal of helping businesses use AI without running the risk of discriminating against certain people. From a report: Rich Caruana, a senior researcher on the bias-detection tool at Microsoft, described it as a "dashboard" that engineers can apply to trained AI models. "Things like transparency, intelligibility, and explanation are new enough to the field that few of us have sufficient experience to know everything we should look for and all the ways that bias might lurk in our models," he told MIT Technology Review. Bias in algorithms is an issue increasingly coming to the fore. At the Re-Work Deep Learning Summit in Boston this week, Gabriele Fariello, a Harvard instructor in machine learning and chief information officer at the University of Rhode Island, said that there are "significant ... problems" in the AI field's treatment of ethics and bias today. "There are real decisions being made in health care, in the judicial system, and elsewhere that affect your life directly," he said.
The COMPAS algorithm, while opaque, does not have race as an input. It was found its accuracy could be matched by an algorithm with just two variables: age and prior convictions. Even this simple model shows the same "bias" that COMPAS is accused of. The bias isn't in the algorithm; it's in the real world.
I think we have to be a little more formal with terminology. The summary and most articles these days use "algorithm" and "AI" interchangeably. You can use an algorithm to train a machine learning model, but the model isn't really an algorithm in the classical sense.
The trained model can definitely have bias based on the training data. The classical example is, train a word2vec or glove model on the texts of wikipedia, then find the vector representations of doctor and nurse. You'll find that nurse is considered a female term while doctor is male.
This may be acceptable for trivial things like advertising or movie suggestions, but machine learning is now being used for important things like job application screenings. Many times the model can be very opaque and this bias may not seem obvious. Even worse, it seems every company now wants to have AI in their product, and may have half-rate data scientists that graduated from a data science bootcamp.
The research I've seen on this subject is serious work. In the case of the doctor/nurse vector representation, the goal would be to make the occupation gender neutral. The tricky part is that you'd still want the model to retain certain qualities, like mother being female and father being male.
An algorithm that uses historic data, which was distorted by human bias, to predict future events. These reinforce human bias from the past. For instance, did you know that in 1864, practically no black people in the South ever paid a debt back? If you use that fact (which was, you know, caused by slavery) to figure that black people were higher credit risks, which meant higher rates, which meant more defaults, which meant worse credit, etc, your algorithm is biased.
Your ad here. Ask me how!
Slight tangent: The article cites the ProPublica study on the Northpointe software in which journalists (not statisticians) reported the software as biased. What they left out is that an independent study found this study showing bias to be wrong.
Source: Flores, Bechtel, Lowencamp; Federal Probation Journal, September 2016, "False Positives, False Negatives, and False Analyses: A Rejoinder to “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And it’s Biased Against Blacks.”", URL http://www.uscourts.gov/statis...
In fact the ProPublica analysis was so wrong that the authors wrote: "It is noteworthy that the ProPublica code of ethics advises investigative journalists that "when in doubt, ask" numerous times. We feel that Larson et al.'s (2016) omissions and mistakes could have been avoided had they just asked. Perhaps they might have even asked...a criminologist? We certainly respect the mission of ProPublica, which is to "practice and promote investigative journalism in the public interest." However, we also feel that the journalists at ProPublica strayed from their own code of ethics in that they did not present the facts accurately, their presentation of the existing literature was incomplete, and they failed to "ask." While we aren’t inferring that they had an agenda in writing their story, we believe that they are better equipped to report the research news, rather than attempt to make the research news."
The authors of the ProPublica article are no longer with the organization, but this article shows up in any news article about AI bias. The fake story just doesn't want to die...
With all that said, I have some hopes that algorithms will help make truly race-blind decisions in criminal justice. It's easier to test them for bias than humans, and decisions are made in a consistent, repeatable manner.
SJWs simultaneously complain that black people are being arrested more, and that an algorithm that predicts higher recidivism for blacks is improperly biased and should return the same risk for whites.
Explain this to one, and you'll get a blank stare followed by an accusation that you're a racist.
Comment removed based on user account deletion