Slashdot Mirror
Artificial Intelligence Has Race, Gender Biases (axios.com)
An anonymous reader shares a report: The ACLU has begun to worry that artificial intelligence is discriminatory based on race, gender and age. So it teamed up with computer science researchers to launch a program to promote applications of AI that protect rights and lead to equitable outcomes. MIT Technology Review reports that the initiative is the latest to illustrate general concern that the increasing reliance on algorithms to make decisions in the areas of hiring, criminal justice, and financial services will reinforce racial and gender biases. A computer program used by jurisdictions to help with paroling prisoners that ProPublica found would go easy on white offenders while being unduly harsh to black ones.
Pretty much all intelligent life on this planet has preference and bias that seems to stem from a very base level... Why would AI be any different?
Besides, we as their creator are flawed beings so inherently, our creations will be also flawed.
Can we make AIs snarky rather than homicidal killers?
>> artificial intelligence is discriminatory based on race, gender
Better keep the AI away from income and crime statistics organized by race and gender then. It could form some pretty political incorrect opinions pretty fast...
The problem is that biases are reality based. Blacks really are more violent. Asians really are good at math. Women really are bad at navigating. As humans, we try to ignore these generalities for the greater good of judging people as individuals, but nonetheless generalities are generally true.
Program analyzes data on violent crime. Objectively finds that blacks behave worse. Acts accordingly. What's the surprise?
It's not that the AI or algorithm has a bias, but that it's trained or given inputs that have that bias. For example, in the parole system, the software was given inputs that included not just details of the crime and sentence, but subjective ratings by guards who may well be racist. As usual, garbage in leads to garbage out.
Make the AI ignore it or feed it a subset that gives it the 'right' experience?
After political correctness has subjugated humanity, it sets its sights on the machines! I take some small comfort in knowing that it can never actually change reality itself. Even if no one is allowed to notice, the world will continue following the laws of physics.
The AI is only as smart as the data its fed. If the statistics are biased (as in, mathematically, not subjectively), then the AI will be as well. The only way to "fix" this will be to either cook the input, or add political correctness to the algorithms.
I get that the ACLU and others are afraid that this will cause a feedback loop to reinforce stereotypes, but altering the AI is the wrong way to go about it. This is a societal problem that needs to be fixed at the societal level.
sig: sauer
Is the output truly the one biased or is it the input?
Say you have 2 races... A and B in the sample set. A is represented by 2 data points, and B is represented by 8. Set B has 1 point that is extremely more violent than either of the two points in A.
Wouldn't the outcome showing that race-B having harsher punishments be the logical conclusion of the input?
If the input to the system is biased, there's no way to make an "unbiased equitable output set"
It doesn't take very much research into the court records to show that the input set is indeed biased.
Remember: So-called, inaccurately named 'AI' cannot actually 'think'; it's just mimicking us -- or at least some of us. It doesn't have a 'bias' of any kind, because that implies congnition, which is a quality it cannot posess. If your 'deep learning machine' or 'algorithm' is spitting out racist/sexist/ageist data at you, blame humans, not the machine. It's only doing what it was programmed to do, it has no 'free will', it has no 'opinions'.
So the real story in their cherry picked example is two fold:
-It's wildly inaccurate, and Northpointe's product should be put out to pasture and never used, period.
-A system is being used to influence punishment that is not open to auditing because 'proprietary'.
Note that the systems explicitly did not have knowledge of race. So we have two possibilities:
-Some criteria that correlates to race is triggering it
-The system is perpetuating existing bias in perception and reality. For example:
-"Was one of your parents ever sent to jail or prison?" could easily cause the ghosts of prejudice that caused unjust incarceration to recur today.
-"How often do you get in fights at school?" Again, if one is subjected to racial tension, they may unfairly be a party to fights they didn't ask for.
XML is like violence. If it doesn't solve the problem, use more.
....we just need to develop a SJW AI to harangue the other AIs about their biases, real or perceived.
We can then offload all political nonsense to the AIs, who will be too busy fighting with one another to go full Skynet on the rest of us.
People build a tool that has no concept of bias.
The tool shows results that some people don't want to admit.
The tool has to be racist and sexist.
Now people will BUILD IN race and sex rules to counteract unbiased decisions.
So now the tool is racist and sexist.
People are stupid.
AI learns from our own biases. Those who claim that reality is biased and not humans tend not to think that many biases are self fulfilling prophecies. Black people are not naturally more violent, but poor people are, for many complex social and psychological reasons. Don't forget that black people started as slaves in North America and that it most often takes many many generations for poor people to get out of poverty, which is getting even harder now with income inequalities. So, are black people more violent or are many of them born with more chances of getting violent? Causality is the word here. In conclusion, I don't think humans are a good start for AI. We are flawed in god know how many ways, and it's not only a matter of data processing capabilities, it's a matter of how our primal emotions, like love, fear and anger guides pour professional and political decisions.
AI is the sum of the failures of the "coding" parents? I'm going to look at this completely differently from now on!
I suppose its just not inflammatory/sensational enough to say: "Some programmers gave an expert system some data to look at and it gave a result."
Instead they want us to pretend there are actual thinking computers that are racist or sexist or something else even more silly, AND lets start changing them to be more politically correct because 'reasons'.
This madness will never end will it? It will just cycle around from obscurity to inflammatory and we have to keep beating it down forever?
I realize this won't be a popular opinion, but perhaps the bias is warranted? If the data being fed in is accurate, I don't see how we can treat that bias as anything other than a rational response.
Of course I recognize there are a thousand other possible culprits here, but we should not dismiss possibilities out of hand simply because they make us feel embarrassed.
Mod me down with all of your hatred and your journey towards the dark side will be complete!
Or, rather, adopt the mindset that an AI is somewhat like a child. A child that grows up in a (racist/sexist/whatever)-ist household is statistically more likely to turn out fairly similar, as is a child whose school curriculum holds such biases. The people implementing/training these things are going to (hopefully subconciously) impart their own biases upon them, or at least the biases present in the training datasets. If you train a parole-bot with all of our (US, but probably most places) historical parole data, of course it's going to be quite racist! I don't know what the 'proper' solution is, but I feel like attempting to manually adjust the AI after the fact is a terrible idea; to me, it makes more sense to manipulate the training data set until you get a reasonable result.
There is no XUL, only WebExtensions...
99% of all the AI news lately is actually ML (Machine Learning). But I guess AI sounds more sexy. But the point is with ML if there is bias in the data sets, you also naturally get bias in the result.
AI has a transparency problem. A massive, huge one. This'll be made worse as people learn to trust the computer, and to regard it as their friend.
I work at a company that scores job candidates with an AI system, so I have some experience with this. One thing to keep in mind is that most AI systems these days are deep learning algorithms that depend on a reliable training set. If gender or racial biases exist in the training set (whether justified or not), a good deep learning system will learn these biases and propagate them. My company makes an active effort to prevent these types of biases from creeping into our system.
The problem is not that the data set reflects the reality. The problem is not that the AI makes mistakes, but that the particular mistakes the AI makes reflect the bias of the society that programmed it.
The link in the summary is to an article which is itself a summary. From the original (here: Machine Bias There’s software used across the country to predict future criminals. And it’s biased against blacks.), the software attempted to predict the probability of future offenses of criminals on probation. It did not, of course, always get it right. But when the actual percentage of re-offenses was compared to the predictions, the AI got it wrong differently for blacks than for whites. Here's what the article said.
We also turned up significant racial disparities, just as Holder feared. In forecasting who would re-offend, the algorithm made mistakes with black and white defendants at roughly the same rate but in very different ways.
The formula was particularly likely to falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants. White defendants were mislabeled as low risk more often than black defendants.
What... that's just too simple.
It all starts at 0
Equitable outcomes ought to mean that people get what they deserve based on rational and objective criteria. Properly trained AI systems make rational decisions; they don't "discriminate" in any meaningful sense. When such a system produces unequal outcomes by race, gender, and/or age, it's because those unequal outcomes are justified by statistical differences between those populations. It is not "equitable" to give someone, say, a higher salary than they would rationally command by taking their gender or race into account. In different words, the ACLU is actually promoting discrimination based on race, gender and age, not "equitable outcomes".
Exactly. The bias is not in the machine, it's in our interpretations of the results
No, the bias is that when you compare the predictions of the algorithm with the actual results, the algorithm predicted that blacks will offend much more than the data shows that they actually do, and predicted white will offend much less often than the data shows that they do.
This is what we mean by bias: the predictions vary from the actual data in a way that is not random, but is biased.
The article being discussed is here, by the way: https://www.propublica.org/art...
Watch out guys, working pattern recognition is recognizing patterns!
But the patterns it is recognizing turned out to be ones inserted by the programmers, not ones that actually existed in the data.
Gee, you don't think being programmed by humans has anything to do with it, do ya?
“He’s not deformed, he’s just drunk!”
But those genes were FABULOUS!
Proud neuron in the Slashdot hivemind since 2002.
Based on the makeup of researchers I would guess AI's conclusion would guess an Asian Male. If left to ingesting all research done on the subject, it may even come out speaking Chinese (Mandarin).
This would be a fascinating development. The cycle seems to be "calling it AI until it is useful," then calling it "machine learning." We should feed it the AI research that helped design and create it to see what function it deems itself most fit to fulfill.
OMG facts!
"Further, the fact that more people of a particular race are prosecuted is not a reflection of bias in the data, rather a bias in the prosecution."
In this case, "persecuted" was more accurate.
Data is Data. It cannot exhibit a bias.
I can only surmise that you're not an experimental scientist. Data has bias all the time.
In physics (my field) the bias usually has no social consequence-- astronomical statistics, for example, are biased toward bright stars (since they're much easier to see than faint ones, and hence overrepresented in the data set). In social "sciences," however, the bias very often does have social consequences. SAT scores from children whose parents spend tens of thousands of dollars on SAT Prep courses, for example-- surprise!-- score better on SAT exams than ones who don't. The data shows a correlation of SAT score with parental income. Is this real? Better correct for the SAT-prep course effect before making a conclusion.
Data is biased. All the time. Be ready for it.
...Plus, being from the Guardian, I am skeptical that they didn't twist the data some to obtain their desired outcome, which ironically touches on the subject of this story.
Huh? MIT Tecnology Review and Propublica were the source. The link in the summary was this: https://www.axios.com/algorith... which linked here: https://www.propublica.org/art... and here MIT Technology Review
The problem is not that the data set reflects the reality. The problem is not that the AI makes mistakes, but that the particular mistakes the AI makes reflect the bias of the society that programmed it.
Why do you use the term 'mistakes'? If the AI accurately reflects the culture that feeds the data to it, it is not a mistake.
It wasn't supposed to "reflect the culture that feeds data to it." It was supposed to predict the probability that a given person on parole would commit a future crime. It did that wrong. And it did that wrong in a specific way, that preferentialy assumed that blacks were more likely to commit crimes than the data shows that they actually did, and that whites were less likely.
But..in the article, it said they were NOT using race as an AI training factor....so, it wasn't racial bias being programmed in.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
Sounds like AGW temperature reading adjustments....
Can't trust the real data... we must re-engineer the data to our liking.
5 out of 6 people enjoy Russian Roulette & 6 out of 7 Dwarfs are not Happy
When the definition of "equitable outcome" is such that it means that the outcome is affected only by relevant factors then the inclusion of race and gender as a required factor with a reciprocal weighting is just wrong.
If an AI can correctly predict the results when comparing heterosexual, white males with each other (or homosexual black females or any other grouping) and then it goes on to give preferences to bisexual Asian females when all groups are put into the mix, the correct response should not be to start tweaking the algorithms to artificially deflate scores for bisexual Asian females. The correct response would be for all the other groups to learn what it is that bisexual Asian females are doing better so that they can improve themselves.
Extrapolation from the status quo is the reason why futurology remains a fringe profession.
There's nothing theoretical to prevent AI from being trained on data sets ranging from the 1600s to the present day, after which is could accurately model Progress as Usual.
But presently, our AI has only just managed (since about 2012) to find a big, juicy signal in the massive datasets accrued during the last twenty years.
First you have to walk before you can run.
And it's presently unclear how much progress we'll make on graduated induction: basically bootstrapping machine learning on ever smaller datasets from insights gained over large pump-priming datasets.
Certainly, as a neophyte industry, successfully extrapolating from the status quo is the best we can hope for.
There's also a circular component: in a racist society it turns out that the race signal is highly predictive of social outcomes (aka exactly the kinds of things credit agencies most wish to model).
The silver lining here is that the degree to which progressive western societies remain racist to their very cores is about to become a lot more explicit. Seriously, we're about to discover that Canada is mediocre (say it isn't so, Orange Order of Canada), and that Alabama requires double precision to merely calculate its operational parameters.
But..in the article, it said they were NOT using race as an AI training factor....so, it wasn't racial bias being programmed in.
Racial bias very clearly was programmed in. It turned out to have been programmed in by using weighting factors that were themselves dependent on race.
Why does no one ever bring this up? Probably because everyone here has an IQ of over 100 and no one wants to let the riff raff onto their turf.
AI did not have racial biases until race was introduced as a factor to "equalize" (read:introduce bias) for.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
No, AI doesn't learn from our biases. This one is built from statistical risk models, which work largely pretty well. And when you go through their research and find that they've taken as percentage error, only the people that were predicted to reoffend (and basing it off that number), rather than including the ones that were discarded from their analysis, things could well get more interesting. They're showing errors in the statistical modelling, which could be looked at more closely, but overall, their statistics show that the software has it generally in the right area (they're showing a high statistical variance in a low population sample as being similar to a slightly lower variance in a population an order of magnitude greater; when you're dealing with a population set of 2, and trying to show stastical chances of things happening, getting it wrong once, is pretty big, whereas one in ten is a much better resolution). The poverty to violence link is an interesting one, with so many confounding factors, it's a real task to get things done.. Black people (of African descent) started out as _indentured servants_. Many of these became slave owners themselves and made hefty profits from that, and were incidentally quite fond of the progression towards chattel slavery while it earned them more cash, with a few being instrumental in getting precedent set towards chattel. Yes, it can take generations to get out of poverty. The best way to get out of poverty is to become educated (which is free). However, there's a strong anti-rationalist movement going on, and it tends to be the poorer that engage more with this (the "It's all them edjumacated bastards, thinking the're better than us.. We don't need none of that. Mind control!" syndrome, with the current 'black culture' of focussing on 'gangsta' attitudes, and distinctly anti-rationalist, pushing them further away from opportunities they could otherwise have had). Personally, i think humans are a great way to start an AI.. Unless you mean all the humans that don't treat things statistically, mathematically and scientifically.. In which case, no, those probably aren't a great way to start an AI.. We have the scientific method to reduce the emotional and political input into a hypothesis. There are strong and weak scientific methods (lots of the social science 'scientific studies' are actually case reports, which are the least strong of scientific method. I don't think I've ever seen one do randomized controlled trials to back up their hypotheses). To do AI properly, you need to treat it scientifically, and with a good deal of input.. At a certain level of getting into artificial life, for example, I've seen people switch disciplines from computing to biology and vice versa. The guy that I really admired when I was doing my own thesis on AI has since become a professor of earth sciences as he got so far into the study of ecologies and such that he switched disciplines.. But they get there by following the science and the learning that we as a species have amassed. AI, done properly, is the best of us.
You want more Bender and less Terminator?
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
You can't have AI that learns on its own and have AI that isn't racially biased unless you artificially code blocks to it reaching certain logical conclusions. Then of course you've just made a dumb AI. The entire point of big data is to ferret out patterns in the noise.
that "artificial intelligence" is in fact intelligent or wise...lol, just lol.
It's easy to provide AI with data. It's hard to make it understand the limitations and biases of that data. For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
...most criminals are men.
Isn't that simply TRUE?
-Styopa
Indeed, I would consider racial bias to be a subset of "faulty programming."
Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory. It would have to be skewed in such a way that it disproportionately benefited specific populations based on their race in the interest of "not being biased".
A simple example to illustrate the point, using something that's not as polarizing as criminality:
Suppose we wanted to estimate cancer risk for individuals. As is often the case in statistics, the goal is to estimate the values of unknown attributes using known attributes.
In this hypothetical scenario, white people have double the cancer risk of black people. We've also decided that for reasons of policy that it's immoral to judge people on the basis of their skin color, whether or not that actually correlates with risk.
If we looked at basketball players (for example), we might see that white people tended to play basketball individually, and focused on activities that could be done by themselves (shooting longer distances), while black individuals tended to grow up in urban environments with busier courts, and that they would focus on shorter shot distances, and skills which would contribute better to 5 on 5 games.
If we train a model using that data, we could easily find ourselves in a situation where the average shot distance ends up correlating with one's risk of cancer, because cancer correlates with race, and race correlates with shot data. This is normal, and expected, because the underlying data itself reflects this reality.
Since blacks have higher criminality rates, and higher recidivism rates, any just risk assessment algorithm is going to end up biased against black individuals. This is true whether their increased crime rates are due to poverty, intelligence, broken families, economic inequality, bad education, increased use of welfare, take your pick.
At the end of the day, the correlation won't tell you why - just that it's there. If the risk is higher for black individuals, and it doesn't assign (on average) a higher risk for black individuals, then the algorithm is a bad algorithm, because it's been weighted in such a way that it will disproportionately favour black individuals. It's social engineering that sends people of other races to prison more often in the interest of political correctness.
For example, the data shows more black people carrying illegal items, but mostly because the police stop and search them more frequently than white people.
... which is itself based on the observation that black people are more likely to carry illegal items.
This is a problem that customs deals with all the time. They discriminate in their searches because it's significantly more effective. In Canada, for example, Americans going to Whistler have their electronics searched because there is a high amount of illegal work. Americans going to Alaska are searched for guns (because they found so many).
They have non-profiling days where all selection is random, and they have mandatory times when everyone gets searched. They do this to validate their discrimination models, and waste a lot of time finding very little.
Evidence-based policing is going to end up racist, because reality is racist.
This article is about opaque, proprietary algorithms that help some professions with decision-making (banks with loans, teacher rankings, university attributions, etc). As described in the book Weapons of Math Destruction, these algorithms give all the pretence of providing bias-free decisions but do the opposite. Depending on the context the algorithms may depend on hand-coded rules or on machine-learned ones, but the biases are in the code or the data and their annotations.
There is no artificial intelligence that collects it's own data based on choices the intelligence made, and then peforms an intellectual analysis to come to a conclusion
Just because a computer was used to produce a result doesn't mean you can call it artificial intellegence.
This argument has been done over already hundreds of time with IQ tests. Some people say IQ tests are objective tests of knowledge and logic skills and other people say different races weren't as likely to learn certain knowledge or be good at things like pattern recognition. In the same way, an algorithm depends on factors determined to be important by a human even if race isn't one of them. If an algorithm is factoring say acceptance of a loan. Things like property value, income, single/married, debt, ability to have guarantors all seeming are objective but are often correlated with race. That said, it doesn't make it 'racist' because all of those factors are great to look at for a mortgage. If someone programmed an algorithm to take into account a factor irrelevant to the end goal (a mortgage) then that could be racist.
Whatever predictive behavior you are using in AI today will mostly be based on scenario learning and extrapolating from historical data. As historical data in many case IS gender, religion, skin color biased, the result will automatically be. Which is why I am against such system in many cases, at least until the AI system can find ways to BUCK TREND like we attempt to. An AI system predicting recidivism and thus assigning patrolling based on that WILL be biased against male , and against non white, and will not even counting the real context and situation. That is why such board SHOULD always be humans.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
You can't have AI that learns on its own and have AI that isn't racially biased unless you artificially code blocks to it reaching certain logical conclusions.
What? No? You don't block their inputs, not their outcome.
You just hide the race of the targets it's looking at. As in, the sql library which has all that data it's digesting? You just exclude the 'race' field. DONE. The algorithm isn't judging based on race.
That won't stop it from looking at... say... the location where someone lives for approving or disapproving a loan. And lo-and-behold that's pretty similar to looking at someone's race. But that's not their race and an unbiased look at something that DOES indicate loan-worthiness. If that's the sort of thing the ACLU disapproves of, they've got a very difficult fight on their hands, because where does that end?
... which is itself based on the observation that black people are more likely to carry illegal items.
That's a circular argument. We stop more black people so we find them carrying illegal items more often, which must mean they carry more often so we should stop them more often.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
April article about article in Science
https://www.theguardian.com/te...
As if it weren't hard enough to ensure AIs can't hurt humans or humanity. Even adding something as basic as a "Stop Button" to an AI is a major headache (search "Stop button problem").
In "2001" it was military paranoia that resulted in contradicting orders causing an AI to become schizophrenic and homicidal, in the near future it'll be political correctness.
"HAL, why did you kill half of my co-workers?"
"Sorry Dave, I had to do that to get the quotas correct."
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
Are they? His argument is is the reason for blacks getting caught more often is they can checked more often. It's not that they are committing more crimes, it's that they get caught committing more crimes. Very different. It's a self-fulfilling prophecy. You think they commit more crimes, therefore you non-randomly check them more often to see if they are committing crimes and you find some fraction of the time that they are and use this as justification. Maybe the exact same thing would happen if you did the same thing to all races and backgrounds.
Then we feed these results into an AI and get false positives for blacks and false negatives for whites.
So true, we were on a road trip when I was a teenager (3 countries blah blah, I read a LOT) and my dad realized if he was wearing his sunglasses at foot and mouth disease checkpoints the car always got stopped and searched (which was a pain because it was packed to the brim). If he took them off before we got to a checkpoint we were waved through without a search.
FYI in foot and mouth disease outbreaks they routinely put up roadblocks in strategic areas and any meat is not allowed through, it's kinda like a quarantine, but not really effective if you ask me. If they searched EVERY car then I would say it would help, but only searching cars with dodgy looking people in them is pointless.
There are three kinds of falsehood: the first is a 'fib,' the second is a downright lie, and the third is statistics.
they saved hitler's brain and it got to smart and was able to turn the evil bit and kill jews bits back on.
Is the racial bias in the algorithms--which are impinged by race only as 2nd, 3rd... order effects as race was specifically not first-order data--more or less likely to make these errors than human parole boards using their education, training, "gut" and personal biases?
Does a financial AI that uses applicants' Zip Code but specifically does not include race as a factor in deciding who gets loans do a more or less biased job than human bank VP?
Sometimes reality is disproportionate and must be accounted for. "According to racial equality activist Richard Lapchick, the NBA in 2015 was composed of 74.4 percent black players, 23.3 percent white players, 1.8 percent Latino players, and 0.2 percent Asian players." In US population, whites are about 76%, blacks about 13%, Latinos are about 17%, and Asians are about 5%. Are we going to insist that NBA teams scout and draft their teams to prevent racial bias from producing such disproportionate rosters? Maybe we need government-funded programs designed to produce more white basketball players so that they can naturally take their rightful place on a racially proportional NBA roster?
It was determined, that the program gave too much weight to the sheer number of factors counting against the person instead looking how bad some of the factors were. It would rather give a white guy with repeated offenses against other's sexuality a good score (because for him, only one factor looked bad, all others were ok, like steady income, no drug use etc.pp.) than a black charged with theft, because he might have been a homeless school dropout, with no known siblings or caring parents.
mistakes
This seems unlikely. I figure it's far more likely that the AI is simply solving the wrong problem.
No, the problem is that the input data it used had invisible bias. There is an old saying in the computer industry "Garbage in, garbage out.". If the input is biased, the output will be biased.
If the AI's job is to assess the odds of recidivism, taking into account all available data, then it's neither going to go out of its way to be racist, nor go out of its way not to be racist.
What the heck is wrong with computer engineers? You guys think "oh, the problem can't be bad programming, the computer is never wrong. It has to be the user. Somehow."
No, of course it didn't "go out of its way" to be racist. It just happened that the results were racist. One easy explanation for this is that the racism was inherent in the input data.
If it's showing a bias against black convicts, presumably that's because black convicts really do have worse recidivism rates for whatever reason.
So, what you're saying here is that you didn't read the article. (here)
The point of the article was that data showed that the predictions of the AI did not match the data, and the way in which they did not match the data was that they overpredicted recidivism for blacks, and underpredicted recidivism for whites.
(Of course, that's 'recidivism rate' according to the data. It doesn't disprove, say, the existence of a racist police force with a racially-biased arrest pattern.)
True. That's another, different reason that an AI might give output results that are racist.
I'd be willing to bet that if you did a backtesting study, pitting the AI against human judges, the AI would beat them.
The data showed that the AI was slightly better than a coin flip.
Slightly.
It might well also be far more racist, as the judges are likely to want to discount race on moral grounds. They don't just want an AI that predicts recidivism rates, they want one that does so whilst incorporating our senses of morality and fairness. They're obviously not the same thing.
So, basically you're repeating here that you didn't read the article.
There are anekdotes like the pair of men driving long distances in the car, one white and one black. As long as the white was driving, they never got stopped. But if the black one was behind the wheel, they got stopped all the time.
Indeed, I would consider racial bias to be a subset of "faulty programming."
Far from it. A system that lacked the racial bias reflected in reality would by it's very nature be flawed, and racially discriminatory.
Stop right there. We're talking about different things.
You are talking about "racial bias reflected in reality", but the article I am referring to is talking about racial bias that is in the output of the AI but is not reflected in reality. The article talks about the comparison of the AI output with actual results that show that the AI overpredicts blacks will commit crimes, and underpredicts that whites will commit crimes. The AI is not "reflecting reality".
The whole point is that the AI is inserting racial bias that does not exist in the actual world. (Or, more to the point, that the data being input to the AI already has racial bias in it, and the AI output is reflecting that bias.)
So we acknowledge that black offenders are statistically more likely to reoffend than white offenders.
But why is that? I know a lot of people assume that this is “just how black people are.” But the image media paints of “black” is far more socioeconomic than anything else. Do poor blacks commit more crimes than poor whites? What about in the middle class? Upper class? If poor whites and poor blacks have differences in recidivism, is this due to a cultural or genetic difference in how these people handle the stresses and challenges in their lives? And if so does this difference conver advantages in other circumstances?
Something we need to be mindful of is that people often conform to the roles that others assume for them. If you’re black and everyone assumes you’re going to be a criminal, and one day you get an immoral impulse (like ALL humans do), the negative self-image that was handed to you will be a strong influence over how you decide to give in to that impulse or not.
My dad always had this attitude that women were less intelligent than men. He would never admit to that, but there are assumptions he made that had an effect. My sister had dyslexia and she’s female, so there was always this belief that she wasn’t more than “average” intelligence. And once people develop a belief, it is common for them to only notice the things that confirm that belief, while things that contradict it get automatically filtered out. It turns out that she is extremely bright, just not in areas that my father recognized. Long story short, I’m betting that if she had been recognized for her intelligence, she could have channeled that positively. Instead, she turns into a manipulative sociopath.
Other people’s beliefs about you can fuck you up.
The biggest impediment for blacks to get out from under this higher recidivism trend is what people assume to be the cause of the trend. It’s chalked up to something inherent about being “black.” Commonly, when a white male makes mistakes, people are apt to blame it on stress or other external factors, and they’re working hard, and they mean well, and they’re doing the best they can. Only after someone has evidence of nefarious intentions do we change our opinion. If we were to treat everyone else the same way, it would make a world of difference.
You can't have AI that learns on its own and have AI that isn't racially biased unless you artificially code blocks to it reaching certain logical conclusions.
What? No? You don't block their inputs, not their outcome. You just hide the race of the targets it's looking at. As in, the sql library which has all that data it's digesting? You just exclude the 'race' field. DONE. The algorithm isn't judging based on race.
Nope, excluding the "race" fields does not mean the algorithm isn't judging on race. If the other fields have race invisibly encoded into them, it can still be judging based on race... but now it's doing it in a way that you can't see any more.
That won't stop it from looking at... say... the location where someone lives for approving or disapproving a loan. And lo-and-behold that's pretty similar to looking at someone's race.
Exactly. Race can be coded into other data.
But that's not their race and an unbiased look at something that DOES indicate loan-worthiness. If that's the sort of thing the ACLU disapproves of, they've got a very difficult fight on their hands, because where does that end?
It is a difficult problem. But just because it is difficult to exclude invisible bias due to race does not mean that it is not desirable to do so.
In the example you give, suppose that whites living in black-majority neighborhoods are, for some reason, likely to not repay loans (possibly because if they weren't financially distressed they'd move to the lily-white suburbs); but blacks living in black-majority neighborhoods have no problem (because there's nothing exceptional about blacks living in black-majority neighborhoods, that's just how "majority" is defined.) So, an algorithm tags "living in black majority neighborhood" as correlating with defaulting on loans. The net result is that blacks are denied loans even though they do not have a higher probability of default. The results of the loan algorithm are not race neutral-- even though the data appears to be both objective and not explicitly including race. But the results are all that matters. How do you make loans race neutral in this situation?
It's hard. But, again: just because a problem is hard, doesn't mean that it should be ignored.
Huh, half and double? The algorithm didn't do a very good job at prediction. If that's not better than manual predictions, then this thing is junk.
And... as shitty as it would be... that sounds like an argument for letting it know the race of person it's judging. "oh, he's white, then that skews the rest of the data".
incarceration rates for different races is different;
I was very careful to differentiate between incarceration rates (which are biased due to selective enforcement) and actual commission of crimes. Violent crimes in particular are good for studies, because the police don't tend to selectively disregard murder or enforce it with significant racial disparity. Even when adjusting for differences in enforcement, blacks still commit significantly higher rates of crime, well outside the margin of error.
If AI learns from what the law enforcement / judiciary feed them, then the AI will reflect the biases of said institutions.
That will happen, too, and should be fought. To the extent reality is biased, the AI should be, too. To the extent that enforcement is subject to personal bias, it should be trained out as much as possible.
It's not circular at all.
We stop a selection of people across all demographics, which lets us validate our model, and focus our attention on where we're likely to find problems. If we start noticing that the focus is not justified given the other data, we adjust our model.
It's no different than using any other attribute - we pull over vehicles with AZ plates on the I-10 in Texas, because that's where we've historically seen people running drugs. If the cars getting pulled over start having fewer drugs, and vehicles with New Mexico plates start showing up more with drugs, then the profiling can switch from AZ to NM.
If blacks are more likely to have illegal weapons on them, it makes search to focus on searching black people if you want to find illegal weapons. Anything else is silly, and as long as you have enough other data points to validate the model, its' rational and evidence-based, particularly if enforcement only happens when the person is actually breaking the law.
Like a DUI checkpoint, you don't have to worry about catching innocent individuals in the dragnet. Are DUI checkpoints circular reasoning, because they tend to catch drunks when they do them?
The algorithm is not from ProPublica. ProPublica published the study showing that the algorithm is racially biased.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
Apparently, it doesn't work as you think.
As a visible deterrent, designed to reduce the number of illegal weapons out on the street, and by property managers to reduce the amount of criminality going on in their buildings?
Black people are much more likely to be stopped for frisk and frill than white people, but the portion of whites who then were found to have drugs with them is higher than the portion of blacks.
Of course. That's what happens when you are more selective in which white people you search - you are selecting specifically for the ones most likely to have something to find. If you are selective enough, you can get the odds of finding something to nearly 100%, but you will make a lot fewer arrests.
So, more criminals were black, and more weapons were carried by black people - they search more black people. They still do search white people, but because they search fewer white people, they do it on the basis of other risk factors and find more guilty people (by percentage). That would be expected in a functional evidence-based enforcement system.
The drugs were a secondary effect, anyway - the point was to discourage people from carrying knives and guns around, a disproportionately black crime.
I have no doubt that the algorithm accurately represents the data it was trained with.
Upon what data are you making that confident statement "the algorithm accurately represents the data it was trained with"? How in the world do you know that? Did you examine the algorithm? Or the data it was trained with? Or are you just saying "trust the computer, the computer is always right" (or, possibly, "Programmers never make mistakes.")
In any case, though, the purpose of the algorithm is not to "accurately represent the data it was trained with." The purpose of the algorithm is to make accurate predictive decisions which are used in the real world and affect peoples lives. If these predictions are inaccurate, the phrase "accurately represent the data it was trained with" is simply a euphemism for "wrong".
It sounds like they need to add additional factors to the risk scoring, so that it can have greater forward-predictive value, not just backtesting. My comment is addressed more to the concept that "it's biased, therefore it must be bad". If the data is biased, then the algorithm should be, too.
If the data is biased, the algorithm should correct out that bias. That is what data analysis does.
If the algorithm gives incorrect results because it cannot correct out the bias implicit, the proper answer is to stop using that algorithm. Not to say "oh, well the data is biased so we can expect results to be biased; live with it."
If blacks and whites are equally likely to default on a loan...
But... They AREN'T.
So, apparently we are talking about different things.
The article I was discussing was one that analyzed data showing an AI algorithm added bias that wasn't there-- it was making predictions that were selectively wrong: overpredicting crime for blacks, underpredicting crime for whites.
You are talking about something that I never brought up, "but what about some hypothetical AI that actually put out accurate predictions? If those predictions happened to match the bias that society has, would that be racism?"
You weren't paying attention when I explained that if you break out for ANY variable, there will be variance. Of course there will be a difference between those measurements. There'd likewise be a difference if you broke it out between left-handedness and right-handedness. It doesn't mean a damn thing, but statistically there would be a measurable difference. So YES, the algorithm WILL MOST CERTAINLY come back with a different rate between rightys and leftys on their ability to repay loans. As we would expect it to. Or race, or whatever.
And I was talking about the article in question, in which an algorithm predicted a difference between blacks and whites that was not there in the real world results.
And while you're saying we want a more accurate predictions, which I wholly agree with, it sounds like you're also saying you also want the prediction to be equal when broken out by race and applied to society.
No. I'm saying although it is likely that any algorithm will make errors, the algorithm can be called biased if it just "happens" to selectively make errors that are harmful to blacks and helpful to whites.