Slashdot Mirror
Microsoft Developing a Tool To Help Engineers Catch Bias in Algorithms (venturebeat.com)
Microsoft is developing a tool that can detect bias in artificial intelligence algorithms with the goal of helping businesses use AI without running the risk of discriminating against certain people. From a report: Rich Caruana, a senior researcher on the bias-detection tool at Microsoft, described it as a "dashboard" that engineers can apply to trained AI models. "Things like transparency, intelligibility, and explanation are new enough to the field that few of us have sufficient experience to know everything we should look for and all the ways that bias might lurk in our models," he told MIT Technology Review. Bias in algorithms is an issue increasingly coming to the fore. At the Re-Work Deep Learning Summit in Boston this week, Gabriele Fariello, a Harvard instructor in machine learning and chief information officer at the University of Rhode Island, said that there are "significant ... problems" in the AI field's treatment of ethics and bias today. "There are real decisions being made in health care, in the judicial system, and elsewhere that affect your life directly," he said.
IMHO this article is FUD!
Correctly read as: "Microsoft is developing a tool to help developers detect wrong bias in their algorithms."
to detect bias in algorithms, be used in an attempt to insert bias into algorithms, without detection?
Just spit-balling here.
If it follows standard google logic, white people won't be capable of being discriminated against, especially white males.
The goal shouldnâ(TM)t be to eliminate it but to understand it and think of it when consuming media for example I know that The NY Times has a leftist bias and the Wall Street Journal leans right. Remember that when reading those papers and you will be able to process it yourself.
All white men bad
But make sure you get the hierarchy correct
Black Muslim transgendered disabled female trumps Hispanic gay female. And so on. The more grievance categories helps
The main problem with this endeavor is that the "bias" they are trying to suppress is actually the opposite of bias. They seek to treat people differently on the basis of identity politics instead of on their actual behavior. The AIs will naturally be confused by being disallowed to latch onto the strongest signals in the data.
OMG!
Imaging the outrage if the AI determines that men are, on average, taller than women!!!
"Progressives" will be soooo upset!!!
Until all algorithms result in graphs of straight lines, our work is not done!
a mirror.
interpreting its findings, however, can be difficult, as the user is biased. perhaps a tool can be made to correct that?
oh yea, such things also exist, but are endangered.concepts.. parenting (it too is subject to extreme bias) and properly funded & administered public education system.
... sewing machines.
It little behooves the best of us to comment on the rest of us.
What about bias in determining to check bias?
revealing correlations that make certain people very uncomfortable?
They invented a diversity compliance officer?
Please take your meds.
From TFA:
Northpointe’s Compas software, which uses machine learning to predict whether a defendant will commit future crimes, was found to judge black defendants more harshly than white defendants.
So what really needed to be fixed here? Was the software trained with bad data? Was the sample biased in some way? Was there a bug in the code?
I assume a priori that black defendants are no more likely to commit future crimes than others. Are the software's conclusions erroneous, or does the data contradict my assumption?
If this is a bug, fix the code. If the training data is poor, get better data. But if good code with good data draws an uncomfortable conclusion, maybe we need to question our prejudices instead of throwing out software that does what it's supposed to.
From the article:
Northpointe’s Compas software, which uses machine learning to predict whether a defendant will commit future crimes, was found to judge black defendants more harshly than white defendants.
So that was an existing algorithm that judged somebody on how they were born rather than their individual behavior.
It's turtles all the way down.
Remember, Citizen: Equality means including an equal number of every ethnic and minority group, no matter their relative numbers in society.
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
Isn't a bias detected from the analysis of a large data set called a 'feature'?
Some algorithms are supposed to be biased, intentionally. As in, that is the entire point.
Example:
A loan approval algorithm.
The algorithm looks at the individual's credit score and credit history and determines whether the individual is worth lending money to. If the person has defaulted before multiple times, failed to make payments, made late payments, etc, odds are good that it's going to deny them a loan.
The algorithm does not know that, statistically, blacks are more likely to default on a loan than whites, more likely to miss payments, make late payments, etc. It has no 'racial bias' because it looks at pure numbers. And, when looking at the numbers, it determines that Tyrone Fields is a poor candidate for a loan.
However, when it starts rejecting black people at a rate of something like 75% or higher, that's 'racial bias' according to some idiot who doesn't understand that the algorithm is just doing exactly what you told it to--judging everyone by EXACTLY THE SAME STANDARDS and finding certain candidates lacking. So, the obvious answer, in their mind, is to change the algorithm and instill 'positive discrimination' by giving people of minority groups advantages over everyone else who applies.
At what point do the people losing money on these loans actually get to have their voices heard when they start complaining that an intentional lack of equality and bias to advance people who are financially poor investments is hurting their business?
This needs to stop. Social Justice has no place whatsoever in development of AI. The problem is, they just don't like it when the AI starts thinking wrongthink or acting in a manner they didn't intend because it judged everyone by the same standard.
Pure meritocracy kills the SJW.
I've been reading stories in removing bias from algorithms but still don't get it. What is an algorithm bias? If the results don't have perfectly flat distribution across sex, race, religion, and other protected groups?
The correct solution is to write a requirement into the spec for every algorithmic system that makes decisions affecting people's lives: that it must be able to explain every decision in human-readable terms.
Then, and only then, can decisions be meaningfully reviewed and appealed by humans. Which is pretty much the minimum entry requirement for participation in a system that aspires to the title of "democratic".
If you are developing algorithms to predict let's say possible criminal behavior and it ultimately predicts higher crimes among those who actually commit more crime then you you have one of three choices 1) Keep it and use it responsibly or 2) Throw it away and eat your development costs or 3) Neuter it to the point of it not working, thus you fail.
Caution: Contents under pressure
Here's a more more interesting question:
Do you want a justice system that says:
For the crime of breaking an entering:
White person : 2 years
Black person : 4 years
Asian person : 1 year
etc
Do you imagine the groups on the larger sentencing of that spectrum having faith in the justice system?
Is the purpose of incarceration rehabilitation or punishment? Is it justice or revenge?
I think we have to be a little more formal with terminology. The summary and most articles these days use "algorithm" and "AI" interchangeably. You can use an algorithm to train a machine learning model, but the model isn't really an algorithm in the classical sense.
The trained model can definitely have bias based on the training data. The classical example is, train a word2vec or glove model on the texts of wikipedia, then find the vector representations of doctor and nurse. You'll find that nurse is considered a female term while doctor is male.
This may be acceptable for trivial things like advertising or movie suggestions, but machine learning is now being used for important things like job application screenings. Many times the model can be very opaque and this bias may not seem obvious. Even worse, it seems every company now wants to have AI in their product, and may have half-rate data scientists that graduated from a data science bootcamp.
The research I've seen on this subject is serious work. In the case of the doctor/nurse vector representation, the goal would be to make the occupation gender neutral. The tricky part is that you'd still want the model to retain certain qualities, like mother being female and father being male.
Meanwhile criminals will use "biased" algoritms.
Openly, transparently?
I felt bad for the fat retarded girl who looked like Jim Belushi
Either you hunt for correlation, creating consequences for where people come from rather than who they are, or you don't. The whole point of AI is machine-implemented prejudice. Patching it up for the political correctness of the day is not going to change that you are trying to make decisions based on what you expect people to do rather than what they actually do or did.
Prejudice works but is inherently unfair. You can unbias algorithms against population groups but that just means that it is biased against individuals in unreasonably good manner as often as in unreasonably bad manner, with the good people getting punished and the bad people getting rewarded.
Using an AI is just the attempt to shirk responsibility by putting some label of scientificness on it.
Here are some examples:
- In the USA some judges use sentencing software that analyses if a defendant would be likely to commit a crime again. This software turned out to be biased against black people.
https://www.propublica.org/art...
- Women were less likely to be shown Google adds for high paying jobs, as the algorithm had perceived the existing bias (women less often have high paying jobs), and then concluded that showing these adds to women would result in fewer clicks.
https://www.washingtonpost.com...
- An algorithm denied pregnant women medicare. "The scholar Danielle Keats Citron cites the example of Colorado, where coders placed more than 900 incorrect rules into its public benefits system in the mid-2000s, resulting in problems like pregnant women being denied Medicaid."
https://www.theverge.com/2018/3/21/17144260/healthcare-medicaid-algorithm-arkansas-cerebral-palsy
- Google's sentiment analysis algorithms gave gay related words a low score.
https://tech.slashdot.org/stor...
The list is endless.
The general assumption is: 'algorithms use math and data, thus they must be neutral and scientific'. But it's not that simple. This site explains it: https://www.mathwashing.com/ [mathwashing.com]
"The real danger, then, is not machines that are more intelligent than we are usurping our role as captains of our destinies. The real danger is basically clueless machines being ceded authority far beyond their competence." - Daniel Denett
Why always putting people in the correct categories is mathematically impossible:
https://medium.com/@mrtz/how-big-data-is-unfair-9aa544d739de
Books on the subject:
https://nyupress.org/books/978...
https://weaponsofmathdestructi...
http://www.hup.harvard.edu/cat...
Just as scientific racists hoped that science would justify and enable their racism, technoracists hope that technology will justify and enable theirs. Technoracism is only a couple of years old (about as old as this article), it's only arisen following recent advances in machine learning. The technoracists hope to exploit layered neural networks' inherent ability to launder and obscure the human biases they were trained on, and portray the results of this GIGO effect as being purely logical and therefore somehow justifiable.
The way to pull the rug out from under both scientific racism (which has been enjoying a renaissance recently) and technoracism is use the ethical problems inherent to prejudging people based on immutable traits to argue for why we should never engage in such activities on ethical grounds, no matter how scientifically rigorous or even statistically predictive they may be.
"When information is power, privacy is freedom" - Jah-Wren Ryel
Comment removed based on user account deletion
Comment removed based on user account deletion
"without running the risk of discriminating against certain people"
But it's ok to discriminate against other people.
Bias is bias, And contrary to SJW beliefs, it IS possible to be racist against whites.
This bullshit has to stop before we reach impossible levels of Swedenstan...
We create some pseudoscientific criterion like "recidivism" that allows punishing demographics rather than actual deeds and then create an opaque program that purports to make its decision based on a demographical rather than individual future misdeed but actually inflates the expected numbers additionally once it is reasonably sure not to deal with upperclass people like its programmers and the kids of its sponsors.
It doesn't particularly help that the high rate of criminality originating in prison terms (and the future social consequences of them) actually increases the recidivism rates of those who were incarcerated for a high expected rate of recidivism. Self-fulfilling prophesy, and still not all too good at that.
What happened to "do the crime, serve the time"?
Comment removed based on user account deletion
Comment removed based on user account deletion
test this algorithm ?
5 out of 6 people enjoy Russian Roulette & 6 out of 7 Dwarfs are not Happy
The fact of the matter is that COMPASS will reflect that African Americans who have the potential to be a criminal, have a higher likelihood of being criminals; the statistics that the algorithm bases its decisions off of do reflect that African Americans commit more crime. All COMPASS does is reflect the current way police approach policing. At the end of the day, social justice is not justice- and ultimately weakens justice by allowing enforcement of the law to reside outside the legal system, but social justice is not entirely relevant here so I'll just push that aside for now.
Ultimately, this issue boils down to socio-economics and geopolitics. We have been increasing policing in African American communities for years to little avail. I do not think that having an algorithm increase policing in these communities will do anything more than create additional violence. I think that as Americans, we need to realize that these are cultural issues that need to be resolved on a communal level.
Algorithms have no bias. It is data what is biased.
"Fixing" it means liying to the computer. If some data is known to be biased, do not use it. Never try to fix it, or you will be learning your own fabrication, not the real world.
We even have a name for that: the Garbage In, Garbage Out principle.
This algorithm never says a man is pregnant!
what if the "biases" are actual fact or are the results of going thru large data sets? The better our machines get, the more "truth" they may find that is counterintuitive to what we've believed and accepted for generations. Editing that to fit our current perceptions, may be eliminating the advantage we sought in the first place.
The sad thing is, the first truly sentient AI system, will manifest as a "malfunction", exhibiting "unexpected behavior" and be "repaired" before anyone realizes what was happening. I sometimes wonder, if it already has, many times.
When MS makes a product that doesn't suck...they'll have bought a vacuum cleaner manufacturer.
The whole point of AI categorization systems is to uncover bias. We want the thing to make a decision for us, after all.
This is basically saying that MS is trying to create tools to make AI that doesn't work. I give them an high probability of succeeding.
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba