AI Programs Exhibit Racial and Gender Biases, Research Reveals (theguardian.com)

An anonymous reader quotes a report from The Guardian: An artificial intelligence tool that has revolutionized the ability of computers to interpret everyday language has been shown to exhibit striking gender and racial biases. The findings raise the specter of existing social inequalities and prejudices being reinforced in new and unpredictable ways as an increasing number of decisions affecting our everyday lives are ceded to automatons. In the past few years, the ability of programs such as Google Translate to interpret language has improved dramatically. These gains have been thanks to new machine learning techniques and the availability of vast amounts of online text data, on which the algorithms can be trained. However, as machines are getting closer to acquiring human-like language abilities, they are also absorbing the deeply ingrained biases concealed within the patterns of language use, the latest research reveals. Joanna Bryson, a computer scientist at the University of Bath and a co-author, warned that AI has the potential to reinforce existing biases because, unlike humans, algorithms may be unequipped to consciously counteract learned biases. The research, published in the journal Science, focuses on a machine learning tool known as "word embedding," which is already transforming the way computers interpret speech and text.

Comments

from the biased report...

[turkeydance]

"And the AI system was more likely to associate European American names with pleasant words such as “gift” or “happy”, while African American names were more commonly associated with unpleasant words." ...what were those unpleasant words?

Re:from the biased report...

[Digital Avatar]

"Murder", "rape", "robbery", "incarceration"... just a guess.

Re:Simple solution

[msauve]

As soon as you start deliberately manipulating the training data, your're introducing your own bias.

Right-handed people are dexterous, lefties are sinister.

Or rather...

[PatientZero]

AIs could incorporate existing biases.

Say you train an AI that will accept or reject loan applications by giving it a stack of previous loans. If the human loan officers were biased against minorities—rejecting otherwise acceptable applications—that AI may end up doing the same. This bias is much easier to detect in human behavior but less so with AI which can't explain why it made any particular choice or even what its criteria are.

Re:Or rather...

[alvinrod]

You wouldn't even go about training a machine learning algorithm that way as it would be pointless. The idea is to let it make better predictions, not train to to make the same predictions as an existing person. Rejected applications are pointless for training as you don't know whether they were a good or bad rejection, whereas if you just give it approved loans and the outcome (i.e., was the loan defaulted on) then the AI can try to develop a set of rules. Typically you feed some large percentage of your data to the algorithm as training data and then use the left over part to test accuracy to see how many times it predicts correctly.

If you truly wanted to avoid racial or gender bias you would just remove that information from what you feed into the algorithm, at which point it can't a priori be biased against anyone because it can't even evaluate them based on those criteria. But let's suppose you do that and then look at the results after the fact, add that data back in and come to the startling conclusion that your AI is disproportionately rejecting candidates from some group. It can't possibly be because it knows they're a member of that group, but because that group happens to have worse outcomes.

If you stop to think about this, its not too hard to come to a reasonable conclusion that if your AI that knows nothing about race is suggesting that black/white/latino borrowers are a higher risk, it's because they're a higher risk. Reality doesn't care about feelings or trying to make sure that outcomes are equal across groups, so we conclude that some group is a worse risk. It probably is the case that black borrowers are more likely to default, but it's not because they're black, but because blacks are typically less well off so of course they're going to default on loans more often. In reality they probably shouldn't (and maybe wouldn't have) received a loan, but some policy designed to make it easier for them to get approval caused it to happen, but that doesn't make them a safer risk, it just lets some people feel better about the world.

If you want to check if your AI is racist find a group of loan applications that are for all intents the same with the only difference being the race of the applicant see if you get a different results based on race for that input set. My guess is that you probably wouldn't. Because if you're stripped out racial data as a category to train on, the algorithm wouldn't suddenly decide to discriminate based on it. Also, for some machine learning algorithms (e.g., anything like a decision tree) you can look at precisely how it evaluates a case, so you could see pretty easily if the AI has a step where race==groupX ? reject : approve becomes pretty apparent. That's not true for all algorithms, but just because its an AI doesn't means its a black box that is beyond all human understanding.

Re: Or rather...

[ShanghaiBill]

Making loans to people of color is more risky.

It depends on the color. Asian Americans have lower default rates than whites.

Re: Or rather...

[SharpFang]

Probably incorrect use of "inherent", in the common meaning, "pervasive".

It's not "inherent" as in "nothing ever can change that, it's an irrevocable part". It's prevalent. Take an individual, you may find a fantastic person. Take an average over the population though, and you see "the average is bad." It is. Don't deny it - the correlation is strong, and while correlation is not causation, in risk assessment correlation is sufficient to deliver accurate results.

I'm not going into detail what social, political, economical and genetic factors may or may not contribute to the correlation. It's a can of worms no professional dares to approach fully objectively. But the correlation between racial and economic status is a fact, and correlation between economic status and risk is a fact. So why would a machine learning device ignore a strong factual trend, just because its existence is offensive?

I'm gonna get so nailed for this :(

[Snotnose]

Maybe because the AI's are modeled on what works, not on what some people wish would work.

One beer ago I wouldn't have had the nerve to say that, says a lot for where social discourse is nowdays.

Re:Simple solution

[epyT-R]

Simple. 'decolonize' it.
https://www.youtube.com/watch?...

SocJus taken to its 'logical' conclusion. Reality is bigotry.

Re:Bias bias bias

[lucm]

Herstory will prove you wrong.

Re:Ok. Thx, bye

[Mal-2]

And this is how you get hugboxes.

People who hold opinion X see a bias against it. People who hold opinion !X also see a bias against it. Both ends cry foul and drift off to places that are "not biased" (that is, biased like all others, just in a way that is acceptable to them).

If you want to leave, leave. But nobody gives a shit about Yet Another Grand Exit. Have fun in your echo chambers.

Self fulfilling prophecies...

[bettodavis]

We better be careful with the implications of a statistics or inference based society. f the algorithms start predicting blacks, latinos, etc are riskier or worse off in general, given current existing conditions, it would in general recommend their owners not to give them a loan, hire or anything evaluated with ML to them.

Therefore, they will continue to be uneducated, unemployed, without means to make a business and in general poorer and more likely to engage in a life of crime. All that nasty stuff that comes with poverty and lack of work, education and opportunities in general.

Ergo they will continue to be riskier and worse off than those in social groups with better evaluations. Rinse and repeat.

This is what folks mean

[rsilvergun]

by Institutionalize racism. It's when it's buried so deep in your society that it's hard to separate it from statistical data. Forest for the trees and what not. It starts getting hard to separate cause and effect. Actually no, that's not right. It becomes easy to _not_ separate them. In the overt scenario blacks get profiled for crime. In the not so overt one they can't get loans because folks in those neighborhoods are 3% more likely to default. This is what happens when you feed large amounts of data into complex systems without knowing or caring about the consequences...

Re:Bias bias bias

[mysidia]

Well.... The idea is if you can declare place X a "safe space" where free-speech and microaggressions/uncomfortable messages are strictly prohibited, then the only thing you need to do next is get a process by which you can expand the size of X, until X encompasses the entire planet, and then your mission is accomplished.

Start with something simple... like a designated area.... then get expanded to something, say the size of a building, then say the size of a college campus, then get someone to declare public areas in a city safe spaces, Then get laws passed applying to places that are private venues but places of public accommodation, Finally, get progressive judges to adopt the same rules for more private spaces, then work on getting a multi-state area, finally, take it to all 50 states.

of course

[argStyopa]

..the begged question is that gender or racial bias and stereotypes are intrinsically "wrong". They are to our 21st century sensibilities, but they served humanity pretty well for millions of years.

Maybe where you have a society where women ARE primarily concerned with raising children, there are better outcomes than when men raise children or women go off to pursue their careers. Maybe where you have a society where obvious strangers are marginalized and driven away, the remainder ends up more cohesive.
I'd be curious how these AI biases would develop if 'fed' only native African literature and information.

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.
One might suggest that, evolutionarily speaking, maintaining a bias is harder than not, assuming no reinforcement. That our language (pretty fundamental to being human, after all) is pervasive with such institutional biases would suggest that there is a value/benefit to such.

Re:of course

[meta-monkey]

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.

But the authors of the article are making such a statement, they just have nature completely backwards. They believe mankind, separated from "society" is naturally non-racist, non-sexist, non-gendered even, and that the outcomes of race, gender, or class groups is imposed on the formless humans by society, to where the concepts themselves of race and gender are "social constructs," and if we smash them everything will just...be great.

This is very similar to Marx's concept of communism and capitalism. He believed that mankind had no innate human nature, so the natural state of mankind was stateless communism, where everyone just naturally gets along and shares and contributes from his ability to the needs of others, and that capitalism was a foreign, oppressive system imposed on these innocents. This is why Marx is famous for his criticisms of capitalism, but as for his descriptions of communism...well not only does he not have them, he thought it was near blasphemous to try to describe how this natural communism would be carried out in practice because the imposition of such order is contrary to the natural, emergent collective spirit of mankind constrained and oppressed by capitalism. Want pure glorious wealth and utopian plenty? Just smash capitalism and you'll get it. And if you've smashed capitalism and perfect communism hasn't emerged...well it must be because you've still got some secret capitalists gumming up the works and they need to be ferreted out and sent to gulag.

This is the same concept behind feminism and anti-racism. Gender norms have nothing to do with the clear, obvious, and scientifically proven biological differences between the sexes. These are in fact imposed by the evil Patriarchy. Smash the Patriarchy and gender equality will simply emerge. If it doesn't, well, it must be because there's still evil sexists hiding around here and they need to be identified and purged. Difference in racial outcomes have nothing to do with the clear, obvious, and scientifically proven biological differences between human haplogroup populations. These are in fact enforced by evil White Supremacy. Smash White Supremacy and racial equality will simply emerge. If it doesn't, well, it must be because there still exist evil racists hiding around here and they need to be identified and purged.

This is the fundamental error of the social justice movement: the belief that race and gender are social constructs when in fact society is a racial and sexual construct.

Re:SJW to sit in on computer science?

[meta-monkey]

What would an export grade AI look like after years of SJW meddling with the design?

To add on, build an AI in the politicized west and ship it to Japan. They unpack it next to a Japanese-designed AI. They ask the AIs, "is an average Japanese person more intelligent, less intelligent, or the same intelligence as an average Ugandan person?"

How does that play out? We know what each AI will be required to say. Why would anyone not afflicted with western social justice leftism ever want to buy the American AI?