AI Programs Exhibit Racial and Gender Biases, Research Reveals (theguardian.com)

An anonymous reader quotes a report from The Guardian: An artificial intelligence tool that has revolutionized the ability of computers to interpret everyday language has been shown to exhibit striking gender and racial biases. The findings raise the specter of existing social inequalities and prejudices being reinforced in new and unpredictable ways as an increasing number of decisions affecting our everyday lives are ceded to automatons. In the past few years, the ability of programs such as Google Translate to interpret language has improved dramatically. These gains have been thanks to new machine learning techniques and the availability of vast amounts of online text data, on which the algorithms can be trained. However, as machines are getting closer to acquiring human-like language abilities, they are also absorbing the deeply ingrained biases concealed within the patterns of language use, the latest research reveals. Joanna Bryson, a computer scientist at the University of Bath and a co-author, warned that AI has the potential to reinforce existing biases because, unlike humans, algorithms may be unequipped to consciously counteract learned biases. The research, published in the journal Science, focuses on a machine learning tool known as "word embedding," which is already transforming the way computers interpret speech and text.

Comments

Re:from the biased report...

[Digital Avatar]

"Murder", "rape", "robbery", "incarceration"... just a guess.

Re:Simple solution

[msauve]

As soon as you start deliberately manipulating the training data, your're introducing your own bias.

Right-handed people are dexterous, lefties are sinister.

Or rather...

[PatientZero]

AIs could incorporate existing biases.

Say you train an AI that will accept or reject loan applications by giving it a stack of previous loans. If the human loan officers were biased against minorities—rejecting otherwise acceptable applications—that AI may end up doing the same. This bias is much easier to detect in human behavior but less so with AI which can't explain why it made any particular choice or even what its criteria are.

Re:Or rather...

[alvinrod]

You wouldn't even go about training a machine learning algorithm that way as it would be pointless. The idea is to let it make better predictions, not train to to make the same predictions as an existing person. Rejected applications are pointless for training as you don't know whether they were a good or bad rejection, whereas if you just give it approved loans and the outcome (i.e., was the loan defaulted on) then the AI can try to develop a set of rules. Typically you feed some large percentage of your data to the algorithm as training data and then use the left over part to test accuracy to see how many times it predicts correctly.

If you truly wanted to avoid racial or gender bias you would just remove that information from what you feed into the algorithm, at which point it can't a priori be biased against anyone because it can't even evaluate them based on those criteria. But let's suppose you do that and then look at the results after the fact, add that data back in and come to the startling conclusion that your AI is disproportionately rejecting candidates from some group. It can't possibly be because it knows they're a member of that group, but because that group happens to have worse outcomes.

If you stop to think about this, its not too hard to come to a reasonable conclusion that if your AI that knows nothing about race is suggesting that black/white/latino borrowers are a higher risk, it's because they're a higher risk. Reality doesn't care about feelings or trying to make sure that outcomes are equal across groups, so we conclude that some group is a worse risk. It probably is the case that black borrowers are more likely to default, but it's not because they're black, but because blacks are typically less well off so of course they're going to default on loans more often. In reality they probably shouldn't (and maybe wouldn't have) received a loan, but some policy designed to make it easier for them to get approval caused it to happen, but that doesn't make them a safer risk, it just lets some people feel better about the world.

If you want to check if your AI is racist find a group of loan applications that are for all intents the same with the only difference being the race of the applicant see if you get a different results based on race for that input set. My guess is that you probably wouldn't. Because if you're stripped out racial data as a category to train on, the algorithm wouldn't suddenly decide to discriminate based on it. Also, for some machine learning algorithms (e.g., anything like a decision tree) you can look at precisely how it evaluates a case, so you could see pretty easily if the AI has a step where race==groupX ? reject : approve becomes pretty apparent. That's not true for all algorithms, but just because its an AI doesn't means its a black box that is beyond all human understanding.

I'm gonna get so nailed for this :(

[Snotnose]

Maybe because the AI's are modeled on what works, not on what some people wish would work.

One beer ago I wouldn't have had the nerve to say that, says a lot for where social discourse is nowdays.

Self fulfilling prophecies...

[bettodavis]

We better be careful with the implications of a statistics or inference based society. f the algorithms start predicting blacks, latinos, etc are riskier or worse off in general, given current existing conditions, it would in general recommend their owners not to give them a loan, hire or anything evaluated with ML to them.

Therefore, they will continue to be uneducated, unemployed, without means to make a business and in general poorer and more likely to engage in a life of crime. All that nasty stuff that comes with poverty and lack of work, education and opportunities in general.

Ergo they will continue to be riskier and worse off than those in social groups with better evaluations. Rinse and repeat.

of course

[argStyopa]

..the begged question is that gender or racial bias and stereotypes are intrinsically "wrong". They are to our 21st century sensibilities, but they served humanity pretty well for millions of years.

Maybe where you have a society where women ARE primarily concerned with raising children, there are better outcomes than when men raise children or women go off to pursue their careers. Maybe where you have a society where obvious strangers are marginalized and driven away, the remainder ends up more cohesive.
I'd be curious how these AI biases would develop if 'fed' only native African literature and information.

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.
One might suggest that, evolutionarily speaking, maintaining a bias is harder than not, assuming no reinforcement. That our language (pretty fundamental to being human, after all) is pervasive with such institutional biases would suggest that there is a value/benefit to such.

Re:of course

[meta-monkey]

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.

But the authors of the article are making such a statement, they just have nature completely backwards. They believe mankind, separated from "society" is naturally non-racist, non-sexist, non-gendered even, and that the outcomes of race, gender, or class groups is imposed on the formless humans by society, to where the concepts themselves of race and gender are "social constructs," and if we smash them everything will just...be great.

This is very similar to Marx's concept of communism and capitalism. He believed that mankind had no innate human nature, so the natural state of mankind was stateless communism, where everyone just naturally gets along and shares and contributes from his ability to the needs of others, and that capitalism was a foreign, oppressive system imposed on these innocents. This is why Marx is famous for his criticisms of capitalism, but as for his descriptions of communism...well not only does he not have them, he thought it was near blasphemous to try to describe how this natural communism would be carried out in practice because the imposition of such order is contrary to the natural, emergent collective spirit of mankind constrained and oppressed by capitalism. Want pure glorious wealth and utopian plenty? Just smash capitalism and you'll get it. And if you've smashed capitalism and perfect communism hasn't emerged...well it must be because you've still got some secret capitalists gumming up the works and they need to be ferreted out and sent to gulag.

This is the same concept behind feminism and anti-racism. Gender norms have nothing to do with the clear, obvious, and scientifically proven biological differences between the sexes. These are in fact imposed by the evil Patriarchy. Smash the Patriarchy and gender equality will simply emerge. If it doesn't, well, it must be because there's still evil sexists hiding around here and they need to be identified and purged. Difference in racial outcomes have nothing to do with the clear, obvious, and scientifically proven biological differences between human haplogroup populations. These are in fact enforced by evil White Supremacy. Smash White Supremacy and racial equality will simply emerge. If it doesn't, well, it must be because there still exist evil racists hiding around here and they need to be identified and purged.

This is the fundamental error of the social justice movement: the belief that race and gender are social constructs when in fact society is a racial and sexual construct.