Slashdot Mirror
Artificial Intelligence Has Race, Gender Biases (axios.com)
An anonymous reader shares a report: The ACLU has begun to worry that artificial intelligence is discriminatory based on race, gender and age. So it teamed up with computer science researchers to launch a program to promote applications of AI that protect rights and lead to equitable outcomes. MIT Technology Review reports that the initiative is the latest to illustrate general concern that the increasing reliance on algorithms to make decisions in the areas of hiring, criminal justice, and financial services will reinforce racial and gender biases. A computer program used by jurisdictions to help with paroling prisoners that ProPublica found would go easy on white offenders while being unduly harsh to black ones.
Pretty much all intelligent life on this planet has preference and bias that seems to stem from a very base level... Why would AI be any different?
Besides, we as their creator are flawed beings so inherently, our creations will be also flawed.
Can we make AIs snarky rather than homicidal killers?
>> artificial intelligence is discriminatory based on race, gender
Better keep the AI away from income and crime statistics organized by race and gender then. It could form some pretty political incorrect opinions pretty fast...
It's not that the AI or algorithm has a bias, but that it's trained or given inputs that have that bias. For example, in the parole system, the software was given inputs that included not just details of the crime and sentence, but subjective ratings by guards who may well be racist. As usual, garbage in leads to garbage out.
Make the AI ignore it or feed it a subset that gives it the 'right' experience?
After political correctness has subjugated humanity, it sets its sights on the machines! I take some small comfort in knowing that it can never actually change reality itself. Even if no one is allowed to notice, the world will continue following the laws of physics.
The AI is only as smart as the data its fed. If the statistics are biased (as in, mathematically, not subjectively), then the AI will be as well. The only way to "fix" this will be to either cook the input, or add political correctness to the algorithms.
I get that the ACLU and others are afraid that this will cause a feedback loop to reinforce stereotypes, but altering the AI is the wrong way to go about it. This is a societal problem that needs to be fixed at the societal level.
sig: sauer
A woman who is good at navigating should not be denied a driving job because most women are bad at it.
We want to be a Just society, so we need a means of ensuring that we do not unfairly punish or limit people because of facts that are true of OTHER people who happen to be similar to them.
So the real story in their cherry picked example is two fold:
-It's wildly inaccurate, and Northpointe's product should be put out to pasture and never used, period.
-A system is being used to influence punishment that is not open to auditing because 'proprietary'.
Note that the systems explicitly did not have knowledge of race. So we have two possibilities:
-Some criteria that correlates to race is triggering it
-The system is perpetuating existing bias in perception and reality. For example:
-"Was one of your parents ever sent to jail or prison?" could easily cause the ghosts of prejudice that caused unjust incarceration to recur today.
-"How often do you get in fights at school?" Again, if one is subjected to racial tension, they may unfairly be a party to fights they didn't ask for.
XML is like violence. If it doesn't solve the problem, use more.
....we just need to develop a SJW AI to harangue the other AIs about their biases, real or perceived.
We can then offload all political nonsense to the AIs, who will be too busy fighting with one another to go full Skynet on the rest of us.
People build a tool that has no concept of bias.
The tool shows results that some people don't want to admit.
The tool has to be racist and sexist.
Now people will BUILD IN race and sex rules to counteract unbiased decisions.
So now the tool is racist and sexist.
People are stupid.
AI learns from our own biases. Those who claim that reality is biased and not humans tend not to think that many biases are self fulfilling prophecies. Black people are not naturally more violent, but poor people are, for many complex social and psychological reasons. Don't forget that black people started as slaves in North America and that it most often takes many many generations for poor people to get out of poverty, which is getting even harder now with income inequalities. So, are black people more violent or are many of them born with more chances of getting violent? Causality is the word here. In conclusion, I don't think humans are a good start for AI. We are flawed in god know how many ways, and it's not only a matter of data processing capabilities, it's a matter of how our primal emotions, like love, fear and anger guides pour professional and political decisions.
Or, rather, adopt the mindset that an AI is somewhat like a child. A child that grows up in a (racist/sexist/whatever)-ist household is statistically more likely to turn out fairly similar, as is a child whose school curriculum holds such biases. The people implementing/training these things are going to (hopefully subconciously) impart their own biases upon them, or at least the biases present in the training datasets. If you train a parole-bot with all of our (US, but probably most places) historical parole data, of course it's going to be quite racist! I don't know what the 'proper' solution is, but I feel like attempting to manually adjust the AI after the fact is a terrible idea; to me, it makes more sense to manipulate the training data set until you get a reasonable result.
There is no XUL, only WebExtensions...
Um, wrong. Blacks aren't more violent. Current popular black culture is violent, which is teaching black youth exposed to it to be violent. Asians aren't "good at math". Most Asian cultures put more of an emphasis on math at an earlier age than western societies. Non Asian students studying overseas from an early age are also "good at math". And children with an Asian ethnicity but born and raised in western cultures are just average at math.
The problem is making policy targeted at individuals based on statistical correlation of a group. We have this individualistic notion in the US at least that every person can forge their own path in life.
That narrative doesn't work when there are systemic barriers put in place pre-emptively due to statistical analysis.
Very few people deny the hard numbers that black people (in the US) commit more crimes. Or that chinese/japanese/korean (in the US, not all "asians") 1st and perhaps 2nd generation people are more academic. I haven't looked up the women and navigation statistics.
The problem comes when you take that general statistic and start making policy that target individuals. E.g. "Looking for a data analyst? Hire that asian-looking guy!"
Even worse when it comes to measures that perpetuate said statistic. E.g. "he's black, so let's assume he's guilty of a crime until proven otherwise".
99% of all the AI news lately is actually ML (Machine Learning). But I guess AI sounds more sexy. But the point is with ML if there is bias in the data sets, you also naturally get bias in the result.
AI has a transparency problem. A massive, huge one. This'll be made worse as people learn to trust the computer, and to regard it as their friend.
Your quest for a solution in this context is misguided, and your implication that ShanghaiBill wants blacks to be mistreated is vile.
It is in nobody's best interest to deny reality.
Contribute to civilization: ari.aynrand.org/donate
Sure, and that's totally fair. The issue comes when, say, 60% of JobsRequiringNavigatingSkills are men and 40% are women, and people say "this is unfair".
To be honest, though, it depends on the job. Men have, typically, much more upper body strength than women, so are more suited to being things like garbage men. Yet nobody's clamoring for equal numbers of women to be garbage *people*.
Yet they are for firefighters, even though firefighting is basically a job where you turn upper body strength into saved lives, simply because they want to be seen as "equal".
People are different and have different things they're good at and bad at. Most HR people are women even though that's a comfortable, high paid, safe job. And I'm okay with that.
Check out my sci-fi book "Lacuna" at http://goo.gl/MVxX8
You're jumping to the end too quickly.
Blacks are convicted of crimes more often, certainly. Does that mean they're more violent, or that they get caught more? Or that they live in worse situations than whites? Are Asians particularly good at math, or do Asian parents favour certain qualities that lead to more favourable math outcomes? Are they in more stable communities so their kids have a better opportunity to study math? Is it cultural or innate? Are women actually bad at navigating, or is it that we're less likely to take little girls out to go camping and get experience at navigating? Is that your own bias, since I've always heard that women are better at navigating?
We actually have statistics that white people just aren't convicted as often for drug offences despite having similar or higher rates of use and dealing. Based on conviction data, a machine learning system would internalise the bias that blacks are more likely to have an involvement with drugs, despite that not being true. Garbage in, garbage out, right?
http://www.dailymail.co.uk/new...
http://www.huffingtonpost.ca/e...
https://www.washingtonpost.com...
http://www.cnn.com/2009/CRIME/...
(Notice that those articles are from 2009, 2011, 2013 and 2014—this is not new data.)
So generalities are not necessarily based in reality. Indeed, your claim that 'Asians are good at math' is particularly bad since Asia is HUGE and there's no way everyone from that area of the world is good at math. And as a half-Chinese guy that's okay at math but much worse than my white partner, and who knows plenty of Chinese people that have no affinity for math at all, I feel like a lot of these generalities are based on folklore and a few selective tests that aren't really representative of ability.
The USA and Canada are not the bastions of equal opportunity that they purport to be, not for everyone. First Nations people in Canada and black people in the USA are consistently disadvantaged through broad government policy.
So all this to say that getting good, clean data for machine learning systems that remove human bias is incredibly difficult, since most humans are unwilling to admit their biases don't necessarily have a basis in reality, or are the wrong conclusions drawn from incomplete knowledge of data.
I work at a company that scores job candidates with an AI system, so I have some experience with this. One thing to keep in mind is that most AI systems these days are deep learning algorithms that depend on a reliable training set. If gender or racial biases exist in the training set (whether justified or not), a good deep learning system will learn these biases and propagate them. My company makes an active effort to prevent these types of biases from creeping into our system.
Blacks are convicted of crimes more often, certainly. Does that mean they're more violent, or that they get caught more? Or that they live in worse situations than whites?
It means that the first 10 times Johnny White gets caught stealing gum, he gets a warning by the shopkeeper, the next 5 times the shopkeeper calls the cops and he's taken home by the cops, then the 16th time, he's formally warned, having that be the first time there's any formal record of his misdeeds. Tyrone Brown gets charged the first time, and gets 10 years "to make an example of him".
That's why the conviction rate isn't a good statistic, the data shows that the entire system has biases.
Learn to love Alaska
It's interesting how you redirected the discussion from "violence" to "drug offenses", which are entirely different things. According to the FBI stats in 2013, there were 2,698 murders committed by blacks, and and 2,755 committed by whites. When you consider that blacks only comprise 12.2% of the population, yet committed nearly as many murders as whites which are 63.7% of the population, there is a significant tendency towards violence. Additionally, 83% of the people murdered by blacks were also black, so majority of those murders were not racially motivated either.
In order for your theory about blacks being found guilty more often to also hold true for murder, whites would have to be found guilty of murder roughly 1/5th of the amount that blacks are to account for the huge discrepancy in the murder rates we see.
Better known as 318230.
The problem is not that the data set reflects the reality. The problem is not that the AI makes mistakes, but that the particular mistakes the AI makes reflect the bias of the society that programmed it.
The link in the summary is to an article which is itself a summary. From the original (here: Machine Bias There’s software used across the country to predict future criminals. And it’s biased against blacks.), the software attempted to predict the probability of future offenses of criminals on probation. It did not, of course, always get it right. But when the actual percentage of re-offenses was compared to the predictions, the AI got it wrong differently for blacks than for whites. Here's what the article said.
We also turned up significant racial disparities, just as Holder feared. In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.
The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants. White defendants were mislabeled as low risk more often than black defendants.
Blacks are vastly more violent per capita than Whites, as shown by the DOJ random surveys asking about crimes one has been a victim of in the past year, then asking particulars about who did it. Blacks are vastly over-represented in assaults and robberies in the US, though all felonies are also committed more often by Blacks per capita. Particularly interracial crime is overwhelmingly Black-on-White rather than the reverse, over a 25-to-1 ratio per capita. For rapes it's 95% certain to be a ratio of hundreds to one. (No W on B rapes reported in the issues of the survey I've been able to find, which is extrapolated to "less than 10", while the DOJ extrapolated tens of thousands for each year for B on W rapes.) This is from lengthy, over 20-page, victim surveys sent to several thousand members of the general population each year, with strong follow-up to get all surveys filled and returned. It isn't cherry-picked or biased by cops and prosecutor's decisions, it's first-hand reports from people who were victimized.
A large law-review published study I read of sentencing in federal criminal courts, which compared similar situations (charges, prior records) statistically show only a very slight bias against Black men compared to White men, a somewhat larger bias against White women compared to Black women (possibly due to Black women being more likely to have dependent children), and a huge bias against men of either race compared to women of either race.
"... your claim that 'Asians are good at math' is particularly bad since..."
Go look at standardized math test scores, for instance the math GRE. The average Asian man is at the 98th percentile compared to Black women, and Black women are at the 2nd percentile compared to Asian men. If we broke out just the Han Chinese and Korean ethnicities the gap would be even bigger, other Asian ethnicities don't do as well, but so what? It's another bit of prior information to take into account when figuring likelihood of being good at math in the absence of more reliable information. It still makes sense to prefer the Korean guy to the extremely rare Black woman with the same score on a math test when hiring for math-heavy job, since there is a much higher chance that the Black woman's high score was in error since it is much further from her population's average (reversion to the mean).
Equitable outcomes ought to mean that people get what they deserve based on rational and objective criteria. Properly trained AI systems make rational decisions; they don't "discriminate" in any meaningful sense. When such a system produces unequal outcomes by race, gender, and/or age, it's because those unequal outcomes are justified by statistical differences between those populations. It is not "equitable" to give someone, say, a higher salary than they would rationally command by taking their gender or race into account. In different words, the ACLU is actually promoting discrimination based on race, gender and age, not "equitable outcomes".
"Further, the fact that more people of a particular race are prosecuted is not a reflection of bias in the data, rather a bias in the prosecution."
In this case, "persecuted" was more accurate.
Data is Data. It cannot exhibit a bias.
I can only surmise that you're not an experimental scientist. Data has bias all the time.
In physics (my field) the bias usually has no social consequence-- astronomical statistics, for example, are biased toward bright stars (since they're much easier to see than faint ones, and hence overrepresented in the data set). In social "sciences," however, the bias very often does have social consequences. SAT scores from children whose parents spend tens of thousands of dollars on SAT Prep courses, for example-- surprise!-- score better on SAT exams than ones who don't. The data shows a correlation of SAT score with parental income. Is this real? Better correct for the SAT-prep course effect before making a conclusion.
Data is biased. All the time. Be ready for it.
...Plus, being from the Guardian, I am skeptical that they didn't twist the data some to obtain their desired outcome, which ironically touches on the subject of this story.
Huh? MIT Tecnology Review and Propublica were the source. The link in the summary was this: https://www.axios.com/algorith... which linked here: https://www.propublica.org/art... and here MIT Technology Review
But..in the article, it said they were NOT using race as an AI training factor....so, it wasn't racial bias being programmed in.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
You can't have AI that learns on its own and have AI that isn't racially biased unless you artificially code blocks to it reaching certain logical conclusions. Then of course you've just made a dumb AI. The entire point of big data is to ferret out patterns in the noise.
That is complete nonsense. That is so far skewed from reality that I do not know where to begin... Are you seriously claiming that a group comprising 6% of the population committing 50% of the murders is less violent because of some sort of mysterious systemic racism? White people are capable of great acts of violence just like any other group but statistically, we're living in pre-immigrant Scandinavia as far as crime rates go if you remove black perpetrated crime from the stats.
It's easy to provide AI with data. It's hard to make it understand the limitations and biases of that data. For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Indeed, I would consider racial bias to be a subset of "faulty programming."
Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory. It would have to be skewed in such a way that it disproportionately benefited specific populations based on their race in the interest of "not being biased".
A simple example to illustrate the point, using something that's not as polarizing as criminality:
Suppose we wanted to estimate cancer risk for individuals. As is often the case in statistics, the goal is to estimate the values of unknown attributes using known attributes.
In this hypothetical scenario, white people have double the cancer risk of black people. We've also decided that for reasons of policy that it's immoral to judge people on the basis of their skin color, whether or not that actually correlates with risk.
If we looked at basketball players (for example), we might see that white people tended to play basketball individually, and focused on activities that could be done by themselves (shooting longer distances), while black individuals tended to grow up in urban environments with busier courts, and that they would focus on shorter shot distances, and skills which would contribute better to 5 on 5 games.
If we train a model using that data, we could easily find ourselves in a situation where the average shot distance ends up correlating with one's risk of cancer, because cancer correlates with race, and race correlates with shot data. This is normal, and expected, because the underlying data itself reflects this reality.
Since blacks have higher criminality rates, and higher recidivism rates, any just risk assessment algorithm is going to end up biased against black individuals. This is true whether their increased crime rates are due to poverty, intelligence, broken families, economic inequality, bad education, increased use of welfare, take your pick.
At the end of the day, the correlation won't tell you why - just that it's there. If the risk is higher for black individuals, and it doesn't assign (on average) a higher risk for black individuals, then the algorithm is a bad algorithm, because it's been weighted in such a way that it will disproportionately favour black individuals. It's social engineering that sends people of other races to prison more often in the interest of political correctness.
For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.
... which is itself based on the observation that black people are more likely to carry illegal items.
This is a problem that customs deals with all the time. They discriminate in their searches because it's significantly more effective. In Canada, for example, Americans going to Whistler have their electronics searched because there is a high amount of illegal work. Americans going to Alaska are searched for guns (because they found so many).
They have non-profiling days where all selection is random, and they have mandatory times when everyone gets searched. They do this to validate their discrimination models, and waste a lot of time finding very little.
Evidence-based policing is going to end up racist, because reality is racist.
... which is itself based on the observation that black people are more likely to carry illegal items.
That's a circular argument. We stop more black people so we find them carrying illegal items more often, which must mean they carry more often so we should stop them more often.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Are they? His argument is is the reason for blacks getting caught more often is they can checked more often. It's not that they are committing more crimes, it's that they get caught committing more crimes. Very different. It's a self-fulfilling prophecy. You think they commit more crimes, therefore you non-randomly check them more often to see if they are committing crimes and you find some fraction of the time that they are and use this as justification. Maybe the exact same thing would happen if you did the same thing to all races and backgrounds.
Then we feed these results into an AI and get false positives for blacks and false negatives for whites.
So true, we were on a road trip when I was a teenager (3 countries blah blah, I read a LOT) and my dad realized if he was wearing his sunglasses at foot and mouth disease checkpoints the car always got stopped and searched (which was a pain because it was packed to the brim). If he took them off before we got to a checkpoint we were waved through without a search.
FYI in foot and mouth disease outbreaks they routinely put up roadblocks in strategic areas and any meat is not allowed through, it's kinda like a quarantine, but not really effective if you ask me. If they searched EVERY car then I would say it would help, but only searching cars with dodgy looking people in them is pointless.
There are three kinds of falsehood: the first is a 'fib,' the second is a downright lie, and the third is statistics.
It was determined, that the program gave too much weight to the sheer number of factors counting against the person instead looking how bad some of the factors were. It would rather give a white guy with repeated offenses against other's sexuality a good score (because for him, only one factor looked bad, all others were ok, like steady income, no drug use etc.pp.) than a black charged with theft, because he might have been a homeless school dropout, with no known siblings or caring parents.
mistakes
This seems unlikely. I figure it's far more likely that the AI is simply solving the wrong problem.
No, the problem is that the input data it used had invisible bias. There is an old saying in the computer industry "Garbage in, garbage out.". If the input is biased, the output will be biased.
If the AI's job is to assess the odds of recidivism, taking into account all available data, then it's neither going to go out of its way to be racist, nor go out of its way not to be racist.
What the heck is wrong with computer engineers? You guys think "oh, the problem can't be bad programming, the computer is never wrong. It has to be the user. Somehow."
No, of course it didn't "go out of its way" to be racist. It just happened that the results were racist. One easy explanation for this is that the racism was inherent in the input data.
If it's showing a bias against black convicts, presumably that's because black convicts really do have worse recidivism rates for whatever reason.
So, what you're saying here is that you didn't read the article. (here)
The point of the article was that data showed that the predictions of the AI did not match the data, and the way in which they did not match the data was that they overpredicted recidivism for blacks, and underpredicted recidivism for whites.
(Of course, that's 'recidivism rate' according to the data. It doesn't disprove, say, the existence of a racist police force with a racially-biased arrest pattern.)
True. That's another, different reason that an AI might give output results that are racist.
I'd be willing to bet that if you did a backtesting study, pitting the AI against human judges, the AI would beat them.
The data showed that the AI was slightly better than a coin flip.
Slightly.
It might well also be far more racist, as the judges are likely to want to discount race on moral grounds. They don't just want an AI that predicts recidivism rates, they want one that does so whilst incorporating our senses of morality and fairness. They're obviously not the same thing.
So, basically you're repeating here that you didn't read the article.
Indeed, I would consider racial bias to be a subset of "faulty programming."
Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory.
Stop right there. We're talking about different things.
You are talking about "racial bias reflected in reality", but the article I am referring to is talking about racial bias that is in the output of the AI but is not reflected in reality. The article talks about the comparison of the AI output with actual results that show that the AI overpredicts blacks will commit crimes, and underpredicts that whites will commit crimes. The AI is not "reflecting reality".
The whole point is that the AI is inserting racial bias that does not exist in the actual world. (Or, more to the point, that the data being input to the AI already has racial bias in it, and the AI output is reflecting that bias.)
So we acknowledge that black offenders are statistically more likely to reoffend than white offenders.
But why is that? I know a lot of people assume that this is “just how black people are.” But the image media paints of “black” is far more socioeconomic than anything else. Do poor blacks commit more crimes than poor whites? What about in the middle class? Upper class? If poor whites and poor blacks have differences in recidivism, is this due to a cultural or genetic difference in how these people handle the stresses and challenges in their lives? And if so does this difference conver advantages in other circumstances?
Something we need to be mindful of is that people often conform to the roles that others assume for them. If you’re black and everyone assumes you’re going to be a criminal, and one day you get an immoral impulse (like ALL humans do), the negative self-image that was handed to you will be a strong influence over how you decide to give in to that impulse or not.
My dad always had this attitude that women were less intelligent than men. He would never admit to that, but there are assumptions he made that had an effect. My sister had dyslexia and she’s female, so there was always this belief that she wasn’t more than “average” intelligence. And once people develop a belief, it is common for them to only notice the things that confirm that belief, while things that contradict it get automatically filtered out. It turns out that she is extremely bright, just not in areas that my father recognized. Long story short, I’m betting that if she had been recognized for her intelligence, she could have channeled that positively. Instead, she turns into a manipulative sociopath.
Other people’s beliefs about you can fuck you up.
The biggest impediment for blacks to get out from under this higher recidivism trend is what people assume to be the cause of the trend. It’s chalked up to something inherent about being “black.” Commonly, when a white male makes mistakes, people are apt to blame it on stress or other external factors, and they’re working hard, and they mean well, and they’re doing the best they can. Only after someone has evidence of nefarious intentions do we change our opinion. If we were to treat everyone else the same way, it would make a world of difference.