AI Programs Exhibit Racial and Gender Biases, Research Reveals (theguardian.com)

An anonymous reader quotes a report from The Guardian: An artificial intelligence tool that has revolutionized the ability of computers to interpret everyday language has been shown to exhibit striking gender and racial biases. The findings raise the specter of existing social inequalities and prejudices being reinforced in new and unpredictable ways as an increasing number of decisions affecting our everyday lives are ceded to automatons. In the past few years, the ability of programs such as Google Translate to interpret language has improved dramatically. These gains have been thanks to new machine learning techniques and the availability of vast amounts of online text data, on which the algorithms can be trained. However, as machines are getting closer to acquiring human-like language abilities, they are also absorbing the deeply ingrained biases concealed within the patterns of language use, the latest research reveals. Joanna Bryson, a computer scientist at the University of Bath and a co-author, warned that AI has the potential to reinforce existing biases because, unlike humans, algorithms may be unequipped to consciously counteract learned biases. The research, published in the journal Science, focuses on a machine learning tool known as "word embedding," which is already transforming the way computers interpret speech and text.

Comments

Bias bias bias

[Anonymous Coward]

This social crap will never end.

Re:Bias bias bias

[Z80a]

Racists are quite hard to squash.
Specially when they adopt a social justice discourse, still judging everyone by their skin colors but having a nice "those are the nice guys" written over the darker portion of their 1930 skin color measure ruler.

Re:Bias bias bias

[Anonymous Coward]

It's a good thing everyone wants to segregate themselves into safe spaces. They can stay there. Away from the good people.

Re:Bias bias bias

[Anonymous Coward]

AIs interpret facts without bias. This is just showing that many of our biases are accurate.

Re:Bias bias bias

[Anonymous Coward]

Microaggression!

Re:Bias bias bias

[Z80a]

Yep.
But it is doing it quite twice, once by the event itself and another in the way it is being reported.

Re:Bias bias bias

[lucm]

Herstory will prove you wrong.

Re:Bias bias bias

[Anonymous Coward]

Look, I apologize for writing the above four posts. I hate black people and women, and I'm just trying to make a point by replying to my own comments.

I'm a product of a broken home and my daddy never loved me. If I've offended anyone, I'm sorry.

Re:Bias bias bias

[Imrik]

If they would stay there, it would be fine, but they keep going to new places and declaring them safe spaces.

Re: Bias bias bias

[Anonymous Coward]

Or more likely, researchers who go looking for bias tend to find it.

Re: Bias bias bias

[Anonymous Coward]

I was waiting for this. When AI, by way of truth, shows a bias for one or the other, then SJWs get s stick up their ass and modify AIs to never be biased.

Re:Bias bias bias

[gweihir]

Racism cannot be squashed, same as stupidity cannot be. And the two are connected. It is one of the ways stupid people make themselves feel better about themselves. Now, realistically, when it comes to actually understanding things, something like 80% or so of the population is stupid. And while only a part of them go for racism, the others just just the same mechanism on other characteristics of themselves they think make them superior. Breeding, geographic aspects, age, gender, what they eat, etc. The list is endless, and it is always the same thing at work.

Re: Bias bias bias

[Anonymous Coward]

I'm apologizing for the previous writer of the above post, he suffered a brain aneurysm while constructing a new victim hierarchy accounting for programs which identify as lesbian midget dislexic racists, and as a result died. His soul will be reincarnated in the form of one of the above.

Re: Bias bias bias

[Anonymous Coward]

It'll be funny when people walk into banks to talk to the automated loan officer, get a few points on their credit rating for identifying as a homosexual owl, wearing a mask to look like a minority, and walking with a pronounced limp. Then the bank will sue and the media circus court case will begin. I can't wait for the SJW media statements. It'll be great.

Re:Bias bias bias

[mysidia]

Well.... The idea is if you can declare place X a "safe space" where free-speech and microaggressions/uncomfortable messages are strictly prohibited, then the only thing you need to do next is get a process by which you can expand the size of X, until X encompasses the entire planet, and then your mission is accomplished.

Start with something simple... like a designated area.... then get expanded to something, say the size of a building, then say the size of a college campus, then get someone to declare public areas in a city safe spaces, Then get laws passed applying to places that are private venues but places of public accommodation, Finally, get progressive judges to adopt the same rules for more private spaces, then work on getting a multi-state area, finally, take it to all 50 states.

Re:Bias bias bias

[Z80a]

Racism is not just about hating certain people by their skin colors, but categorizing people in general by their skin colors.
It's a very intellectually lazy way to deal with the world, which many people sadly do.

Re:Bias bias bias

[dbIII]

Well.... The idea is if you can declare place X a "safe space

Yes - start with a bakery. First keep the gays out and then work on the races.
There's more of that bullshit going on than the massive conspiracy theory you are going on about just because blackface is not polite anymore in comedy or whatever your trivial little playing the victim gripe is. If you are "triggered" by gays then just harden up instead of complaining that nobody is putting them in prison any more.

Re:Bias bias bias

[religionofpeas]

It's a very intellectually lazy way to deal with the world, which many people sadly do.

They do it because it works. I'm sure you also take a lot of shortcuts in judging people. If someone cuts in line in front of you in the store, do you always have the same reaction, or is your reaction different whether the person is wearing a biker jacket or a sundress ?

Re: Bias bias bias

[Anonymous Coward]

Even better, most natural intelligence does not give a fuck about this social justice bullshit. Yes, you can all go and fuck yourselves in your safespaces. You can lock yourselves in there and color with crayons as much as you like. However one day you will have to come out when all the cheetohs have run out and we will still be here, ready to hurt your feelings and make you cry. To the social engineer scum responsible for this retarded article: It's not working, there are still not enough losers in America, desperate to pick up this excuse why they are total failures and run with it.

Re:Bias bias bias

[dromgodis]

It doesn't show that. It just shows that it can act according to what it is taught.

This article is about language learning. Bias in language is tricky. One of the reasons is that the language has evolved through biased views of people and the world. In many languages, "man" is used to represent humans ("mankind") or male is the normal state ("football player" vs "female football player").

If the bias is in the language it is learning, e.g. male words for gender-neutral things, the AI gets imprinted with that this is the norm it is expected to behave by.

As with statistics, you should be very careful with AI and machine learning. You tend to get what you ask for, so be very careful to ask the right questions.

Re: Bias bias bias

[Beau1080p]

And this is how you get hugboxes.

People who hold opinion X see a bias against it. People who hold opinion !X also see a bias against it. Both ends cry foul and drift off to places that are "not biased" (that is, biased like all others, just in a way that is acceptable to them).

If you want to leave, leave. But nobody gives a shit about Yet Another Grand Exit. Have fun in your echo chambers.

Re: Bias bias bias

[SharpFang]

Researchers, who go looking for bias are ridiculed, booed, and then fired.

If a clear racial bias appears in your research results, you'd better bury them deep and never show anyone lest you're marked a nazi.

Re: Bias bias bias

[Anonymous Coward]

I already 'harden up' around gays.

Re:Bias bias bias

[Z80a]

For fight or flight reactions its quite ok to judge by the appearance, but its wiser to judge by the "do it look like a local threat?" rather than "whats the skin color?", which in several cases might be the same, but not always.
Let's say you will quickly judge someone that looks like an yakuza guy over some nerdy japanese, even with both being of the same "race".

Re:Bias bias bias

[religionofpeas]

For fight or flight reactions its quite ok to judge by the appearance, but its wiser to judge by the "do it look like a local threat?" rather than "whats the skin color?",

But often, people aren't judged solely on skin color. Two middle aged black guys in expensive suits walking in a convenience store at night will make a completely different impression on people as a couple of loud kids in sweatpants. People often accuse police officers of stopping a guy "just because he's black", but in reality, they look at dozens of clues, of which skin color is just one.

Re: Bias bias bias

[Anonymous Coward]

The problem is they are training AI with language, which trains the huge language bias.

If they trained it off of quantifiable data (this person has X y z characteristics directly related to the decision needed), you would remove that language bias.

Re:Bias bias bias

[sciengin]

Strawman alert!
We are not talking about objectively bad things like blackface (which, ironically, was very popular with blacks of that time, same reason why The Big Bang Theory is so popular with nerds I guess), we are talking about speaking the truth that offends people.
There are only 2 genders: Fact!
There is no such thing as mansplaining: Fact!
Disagreement is not hate: Fact!
Feminism has reached its goal of equal opportunity in the west decades ago: Fact! ...

Intolerance towards the truth is the worst form of intolerance, its going back to even before the middle ages and antiquity.

Re: Bias bias bias

[Anonymous Coward]

Am I biased if I hate everyone?

Re:Bias bias bias

[Z80a]

So, could be said that is actually racism?

Re: Bias bias bias

[Anonymous Coward]

Not one of those items you stated are fact are "fact". They are all subjective statements whose results are based in your opinion.

Re: Bias bias bias

[Anonymous Coward]

Sigh. I need to apologize. I buttfucked all your mothers while telling them they were niģģer trash. Then when I pulled my dick out of their asses I slapped their faces with it and accused them of giving me 'Niģģer'. But the one mom who had a recent colonic didn't really give me Niģģer. So, I'm sorry for that.

Re: Bias bias bias

[Anonymous Coward]

Darn it ta heck. The N.i.g.g.e.r filter won't let me type ni ni ni N.I.G.G.E.R.

Re: Bias bias bias

[sciengin]

I suggest you look those facts up in the scientific literature instead of your echo chamber.
Reality can be harsh and unforgiving sometimes.

Re:Bias bias bias

[dbIII]

There are only 2 genders: Fact!

Biology and the Olympic committee disagree. One athlete who was not allowed to compete as a woman later gave birth to two children, so definitions are not perfect enough for your "fact" to be correct. Call me pedantic (because it's a tiny percentage), but you did write "fact" on something that is not.

There is no such thing as mansplaining

Stick it with "virtue signaling" in the doubleplusungood newspeak basket. It's just as stupid along with people being a "creative", being worried about the "optics" of a conversation and all kinds of other things that are said within in-groups but don't belong in a written communication unless the desire is to confuse people.

Feminism has reached its goal of equal opportunity in the west decades ago: Fact!

I look around at the total sausage-fest that is IT and notice that I see more women working in technical roles at mine sites. If you are connected with the IT "industry" in any way you are either extremely deluded or outright lying to express such an opinion as a "fact".

Maybe you kids need to grow up a bit and start wondering what your daughter will do for a job if you end up having a daughter. Surely you would want her to have as much of a chance at a decent job as if you had a son?

Re: Bias bias bias

[Anonymous Coward]

H'aint no "facts" out there, broham. Never were, never will be. Reality just isn't tame like that.

Re:Bias bias bias

[umghhh]

Discrimination exists and does influence outcomes of certain activities. AI will detect that some groups for whatever reason have lower chance of paying back the loan. If that however is caused in part by existing discrimination it does not really help that AI will not get to know factor X (skin colour, religion etc). It is also not meant to do so I would imagine and it is counter-productive to make it do so. If monitoring systems detect such a bias maybe we should be acting on factors that cause discrimination? Or maybe these factors are natural in a sense - they have been chosen by affected group? Say women prefer to stay at home with kids (this is not to say no men would want to do that or that all women do). The effect is that they earn less on average. Is this discrimination or not? There is already a hierarchy of values discussion needed to make a proper decision here and no AI is gonna help us in that. So in a sense you are right. Some biases are accurate. Some are not and this all is not really helping the machine to make a proper decision because we do not even know what proper decision means.
I guess what it all means we 'need' AI to replace us all. Only then we will ensure that there is no discrimination by humans and against humans.
There is also another interesting thing that I wonder about. Assuming AI works perfectly and contrary to what I discussed above there is no discrimination in the world.At some point all will have the position in a society that they deserve. A new kid is born in a poor family of underachievers. It is 8th. Maybe it is born into its proper environment but maybe it is a genius - if the kid is in fact a genius (something that happens sporadically but it does happen) its chances of achieving anything matching his ability are like standing in front of a flat high wall one has to climb with no tools. No tools because why give the kid any if its chances are so small? It is a waste?
I find it funny how we humans want to replace our sweat and misery trough the tools we make and in a process we cause another different wave of misery. I wonder if indeed AI will cause the world collapse as some say. In any case revolutions were started by individuals that either had roots in higher parts of society or were at least supported into success by such parties. Now imagine what happens if AI eventually gets to control not only our security and military systems but government too - I mean most of members of governments in the West today do not even have own ideas - they have experts delivering them these. Raplcing experts trough AI is a good idea. It removes BIAS as we know. The questions such AIs have to answer are a bit more complex than - how to close on target that modern Air-to-Air missiles have to make. In fact so complex that I cannot imagine AI making a better judgment than humans can. I wonder only if we are smart enough to accept them and make them to support us. Judging on the way world is today I doubt. We will just have ore efficient ways of controlling humans. Now where would that get us if done really well???
I am really excited about these developments but the reasons for bias AI cannot remove. I just do not think so. Unless as said there will be no humans to be discriminated against. Being excited is ok I guess. I somehow do not feel however that this world will be a good one for my kids. Can AI fix it?

Uh, no.

[msauve]

Bias does not mean what the authors think it means.

Re:Uh, no.

[Anonymous Coward]

This reminds me of a similar news story from a while back about how "reality was racist" because a lot of studies found that a lot of so-called stereotypes were, in fact - *gasp* - true.

Rather than accept that maybe the people they call "racist" are in fact rational beings, the study authors called out reality itself as racist.

Re: Uh, no.

[Anonymous Coward]

Wasn't there some kerfuffle over Google image searches showing black kids in police mugshots vs white kids in college campuses? It turns out when you search the internet you find every bias under the sun. Whodathunkit?

AI learns by example. If you feed it biased data it learns the bias. I don't understand why we're surprised by this.

Re: Uh, no.

[Anonymous Coward]

I, for one, welcome our new racist AI overlords.

Re: Uh, no.

[JaredOfEuropa]

If you feed it biased data it learns the bias.

Exactly. And this is nothing new, by the way; the same worry was expressed around 20 years ago after a proposal to use expert systems to assist judges with sentencing. Though when I see how some people plan to use "big data" today, I can only conclude we haven't learned anything since then.

Re:Uh, no.

[Carewolf]

Bias does not mean what the authors think it means.

It means exactly what they think it means. Maybe you are confused? It starts out as data, but once it is learned and the AI uses it to act on it is bias. It reinforces old cultural norms on new generations.

Re:Uh, no.

[msauve]

You're completely and utterly wrong. Bias implies a subjective prejudice. This isn't a case of bias, it's a case of GIGO.

Re:Uh, no.

[Anonymous Coward]

You are uneducated, so allow me to educate you:
There exist 3 phenomena:
1. Stereotype
2. Discrimination
3. Racism
The 3 phenomena are not mutually inclusive nor mutually exclusive.
The 3 phenomena aren't one and the same.
The 3 phenomena all have their meanings and relative scales.

Many people never properly used a dictionary in their life, or read a Psychology 101 book, so the 3 phenomena are often confused;
much like uneducated people confuse species with race and are incorrect when they call it the human race when in fact biology 101 should
have taught you to call it the human species.

You will argue that words evolve. I will tell you that in this specific case, in this discussion, it isn't the words that evolve but your brain that devolves.
Recognizing the difference between word evolution and mental devolution (with respect to words) is very important these days.

Re:Uh, no.

[AmiMoJo]

Link? Our did you just make up a study to support your position?

Re: Uh, no.

[SharpFang]

The problem is when the bias exists in reality, not in perception or opinions.

The correlation between socioeconomic status and risk of defaulting on a loan is clear, and it would be silly to question it.

The disparity between socioeconomic status of different races is a huge issue, not just a fact, a fact that is loudly announced in a voice full of outrage. This means a clear correlation here too.

So why, when you have "A implies B" and "B implies C" suddenly everyone starts looking for excuses to claim "A implies C" is wrong?

Re:Uh, no.

[SharpFang]

Bias: deviation of the expected value of a statistical estimate from the quantity it estimates,

You have a bag with 10 red and 20 blue marbles. You choose a marble and write it down, then return to the bag.

But it's your subjective prejudice to expect the blue marbles will come up more frequently, right? Everyone knows they will show up in equal proportions, you racist!

Re:Uh, no.

[SharpFang]

I also wish more people who read Psychology 101 book also read Statistics 101 book.

Then they might learn that bias is sometimes a fact of the nature completely apart from perception. That not all random processes follow the gauss curve and if you estimate them correctly, this is not merely your subjective perception.

Re:Uh, no.

[msauve]

Thanks for proving my point. The system (AI) isn't biased, the expectations of the authors are. They seem to expect to put in real world data and get out data fitting their own worldview, and if it doesn't match they call it bias.

Re: Uh, no.

[Anonymous Coward]

Statistics be racys, muthafucka.

Re: Uh, no.

[prisoner-of-enigma]

The problem is when the bias exists in reality, not in perception or opinions.

In such cases, reality must be ignored and a cherished fantasy substituted in its place so no one gets offended in their safe space.

The correlation between socioeconomic status and risk of defaulting on a loan is clear, and it would be silly to question it.

No, it is racist to question it, which means you're racist for suggesting it. And your friends are racist too because they like you. And your co-workers are racist because you sit near them. And your boss is racist because he hasn't fired you for being racist. And your dog is racist because he lives with you. And your car is racist because it carries you and your racist viewpoints to the secret volcano lair of the KKK which you obviously worship because you're racist.

Don't deny it! That would only prove your latent racism! And don't argue with me because only a racist would do that!

(sarcasm disengaged)

I'm sure the above looks and sounds depressingly familiar, does it not?

Re:Uh, no.

[laie_techie]

You are uneducated, so allow me to educate you: There exist 3 phenomena: 1. Stereotype 2. Discrimination 3. Racism

Isn't racism discrimination based on race? All racism is discrimination, but it is possible to discriminate on other criteria.

Re: Uh, no.

[kangsterizer]

"bias" seems like the wrong word for AI though.
Just like humans, it works by generalizing data. Just like humans, it needs to be aware that generalization means this is the statistically most likely case - but not the case for every instance.

So basically, it's not bias - which is why it bothers people a lot - it's pretty much statistical analysis results based on data with low chance of error. It's just that you cannot generalize things while working on a case per case basis and expect the correct outcome every time. It's just more likely to be the correct outcome, but not necessarily correct.

tldr: people wrongly think bias means you're making the wrong decision. bias just really means you're defaulting to the most likely correct choice - which may very well be incorrect depending on the case. Today's "AIs" are pretty much unaware of this.

Re:Uh, no.

[Anonymous Coward]

Many people never properly used a dictionary in their life, or read a Psychology 101 book, so the 3 phenomena are often confused;

If I said that those people were, more often than not, minorities would that be a:

1) stereotype
2) discrimination
or
3) racism?

Re: Uh, no.

[david_thornley]

There's logical implication and then there's causation. A implies B and B implies C means A implies C. A is characteristic of B and B correlates with C doesn't tell us how the causation runs, or indeed what's the best way to estimate C. It may be that, holding socioeconomic class constant, there's no significant difference between blacks and whites defaulting, in which case race would be useful in a prediction only as a proxy for socioeconomic class.

If it's irrelevant, a data mining system might still pick race out as a factor in loan defaults, particularly if it didn't have socioeconomic class as an input. In many contexts, that's illegal.

Re: Uh, no.

[AutodidactLabrat]

Why don't I ever have upvotes available when someone rational speaks sense to idiocy?
Thanks for being first

Re:Uh, no.

[Anonymous Coward]

Stereotype.
It's not discrimination unless you are actually discriminating. Thinking and acting are two separate things.
It's not racism because you are not focusing on race, but the denomination that they are a minority within a group,
which can apply to any race including whites and mixed and so forth. You believe that being a minority in some culture as its own criteria produces a certain effect.

Re: Uh, no.

[SharpFang]

While correlation is not causation, correlation is sufficient in computer learning systems.

Also, computer learning should be resistant against manipulation through fake input data, and this is achievable by prioritizing inputs that are difficult to falsify.

Determining ethnic background is more immune to tampering and falsification than determining one's socioeconomic situation.

Re: Uh, no.

[Keith Henson]

Given that there is "disparity between socioeconomic status of different races" it may be worth asking why?

Gregory Clark, an economic historian at the University of California, Davis may have an answer. It's a painful one. Some groups of people were subjected to genetic selection over 20 generations or longer that was as intense as that the Russians applied to make tame foxes over the same number of generations.

Google "genetically capitalist" to read his fascinating paper on the topic.

Re: Uh, no.

[SharpFang]

Asking "why" is dangerous nowadays.

And asking "why" is absolutely unnecessary for writing risk assessment software. Correlation is not causation, but correlation is sufficient to assess the risk.

Re: Uh, no.

[david_thornley]

My racial descent is fairly obvious, but that isn't true of everyone. Particularly since race is much more a social than a biological construct, and we don't officially belong to racial groups, it's usually a matter of looking a someone, knowing their name, and putting down a category. On the other hand, my income is part of government records, and I have to provide evidence for it for mortgages. We can put down income on the basis of self-reporting, and we can check that against public statistics to see if there's distortions.

from the biased report...

[turkeydance]

"And the AI system was more likely to associate European American names with pleasant words such as “gift” or “happy”, while African American names were more commonly associated with unpleasant words." ...what were those unpleasant words?

Re:from the biased report...

[Digital Avatar]

"Murder", "rape", "robbery", "incarceration"... just a guess.

Re:from the biased report...

[Anonymous Coward]

This and considering women work only a fraction of what men do, this is correct. I haven't had a vacation since 1983, but most women I work with have had an entire week off which is just damn unfair.

Re:from the biased report...

[Anonymous Coward]

That's great, but you wrongly assume that those women are kicking back, sipping margaritas on the beach for that week. Unless you know they are, then that is fair. They could be catching up on a whole bunch of unpaid work that is required in life? Sure you never got the time off to do that, but it's work all the same.

Re: from the biased report...

[Anonymous Coward]

And if Africans don't like this, they can easily fix it by no longer standing out on statistics for murder, rape etc.

Re:from the biased report...

[Anonymous Coward]

Compare annual productivity...

Re:from the biased report...

[gweihir]

Probably a statistically accurate representation of how mainstream-media report things. i.e. it is not the AI tool that has a bias here.

The effect of training an AI on propaganda is that is is then trained on propaganda. Why is this even news? It is obvious.

Re:from the biased report...

[Anonymous Coward]

Probably a statistically accurate representation of how mainstream-media report things.

Or, you know, FBI crime statistics and incarceration rates.

Re:from the biased report...

[meta-monkey]

Are you suggesting crime rates are propaganda? Feed the computer the racial background of everyone incarcerated in America. Be shocked when it concludes blacks are more likely to be criminals.

Re:from the biased report...

[Anonymous Coward]

Are you suggesting crime rates are propaganda? Feed the computer the racial background of everyone incarcerated in America. Be shocked when it concludes blacks are more likely to be criminals.

"More likely to be criminals?" or "more likely to be incarcerated?" .... There is a difference you know.

This is a bit of a sidebar, but you went there. Nobody gets this:

Argument - "Blacks are unfairly discriminated against by police (or regular people, or AI) and are more likely to be unjustly arrested and incarcerated"
Response - "No, blacks are more likely to be criminals, look at the data. There's more of them in jail per capita than whites. Therefore stereotypes are true. "

The issue everyone misses - Of course there are going to be MORE incarcerated blacks if they are being unfairly incarcerated MORE often. The data backs the original claim, and but the other side always "claims" this data as their own even though it doesn't prove causation, it just shows the current state of things.

Re:from the biased report...

[Anonymous Coward]

The problem is that we know the systems that incarcerate people are biased. Blacks are more likely to be stopped, more likely to be arrested, and more likely to be convicted. This considerably fuzzies the extent to which black people are more likely to commit crimes. More likely to be caught? Yes. More likely to be falsely imprisoned? Yes. More likely to commit crimes - probably yes, but how much of that is due to the other biases involved?

Re:from the biased report...

[religionofpeas]

Of course there are going to be MORE incarcerated blacks if they are being unfairly incarcerated MORE often.

Nice story. Now all you have to do is prove the bit where most of the black people have been incarcerated unfairly.

Re: from the biased report...

[Anonymous Coward]

They want to move on to other things but whites have cornered the market on kiddie rape and school shootings.

Re:from the biased report...

[j2bryson]

You guys should read the Science report, I think it's open access? All the stimuli come directly from previously run psychological experiments, and are listed in the supplementary materials here: http://science.sciencemag.org/...

Re:from the biased report...

[Anonymous Coward]

Are you suggesting crime rates are propaganda?

I'd suggest that rates of investigation and criminal penalties are biased.

I remember once someone asked me if I was annoyed being pulled over all the time. I was puzzled, and it turns out he was referring to a missing rearview mirror in my truck. It has *never* been a problem for me.

Guess which one of us would be identified as Caucasian, and which on African?

Now, if we were both committing crimes, guess which one is more likely to be caught through an unrelated stop?

Re:from the biased report...

[laie_techie]

Are you suggesting crime rates are propaganda? Feed the computer the racial background of everyone incarcerated in America. Be shocked when it concludes blacks are more likely to be criminals.

58.7% of inmates at federal prisons are White (statistics as of 25 Feb 2017), while 37.7% are Black. 72.4% of residents self-identified as White in the 2010 US census (12.6% self identified as Black). The debate is over this discrepancy.

Re: from the biased report...

[david_thornley]

That's not a given. Given prejudiced police and other people in the criminal justice system, it's entirely possible that one race will have more convictions per act than another. Perhaps the police hunt down blacks more than whites. Perhaps blacks get worse representation overall, as the overall richer whites can get better legal representation. Perhaps juries tend to convict blacks more than whites, given similar evidence. And, of course, the police and juries justify this because blacks have relatively more convictions for murder, rape, etc.

This stuff gets complicated real fast.

Re:from the biased report...

[david_thornley]

Let's go through some possible causes:

Blacks could be committing more crimes.
Police could investigate in such a manner that they tend to catch more black suspects, relatively, than white.
There may be crimes more typical of blacks than whites, and police may pursue those crimes harder.*
Prosecutors could prosecute blacks disproportionately more often.
Blacks, being on the average poorer than whites, probably get bad legal representation more than whites.
Juries may be prone to convict blacks with less evidence than they'd want to convict whites.
There may be crimes more typical of blacks than whites, and these crimes may call for harsher punishment.
Judges may give longer sentences to blacks than whites.
Blacks may tend to get into more trouble in prison (for a variety of reasons) and don't get as much time off for good behavior

Remember here that, given steady states, the percentage of a certain group in prison is proportional to the number going to prison times the effective length of their sentences. Statistics are like bikinis: what they reveal is important, what they conceal is vital.

*The classic case is cocaine use by sniffing powder (claimed to be more common for whites) vs. cocaine use by smoking crack (claimed to be more common for blacks). Punishments for smoking crack were more severe than for sniffing coke, so, given a black man and a white man convicted of cocaine use, the black would probably serve a longer sentence.

Re: from the biased report...

[Anonymous Coward]

Statistically: * Rapes are overwhelmingly white males @ 65% according to the FBI Statistics site. * Pedophiles are overwhelmingly white males @ 70% - http://www.yellodyno.com/html/...

Re: from the biased report...

[corezz]

Finally, common sense, and honest thinking has prevailed in this thread. Thank you.

Re:from the biased report...

[Anonymous Coward]

Therefore, statistically you should be incarcerated because there is a 65% chance you are going to rape someone (FBI site says white males do 65%). Or locked away because there is a 70% you are a pedophile. So which is it?

Re:from the biased report...

[Anonymous Coward]

Watch Netflix's brilliant "13th" documentary since you clearly don't want to spend the time to know why. But then i'm sure you wont because you don't want facts to upset what you need to believe.

Re:from the biased report...

[gweihir]

Since you obviously are a racist, there really is no point in explaining to you how your reasoning is fundamentally flawed. You do not have the mental capabilities to recognize actual facts.

Re:from the biased report...

[laie_techie]

Please see the comment I replied to. GP incorrectly claimed that more Blacks are incarcerated than Whites. I provided hard statistics showing that a larger portion of guests in federal prison at the present time are White than are Black. I purposefully did not address why Blacks (who make up 12.6% of the general population) make up 37.7% of the federal prison population. It seems like there could be many complicated factors (socioeconomic, education level, racial bias of law enforcement). A recent study has shown than racial bias develops as young as six months old (infants don't trust faces of races they haven't been introduced to).

Re: from the biased report...

[Anonymous Cow Ward]

Which means that, as whites are 77% of the US population (according to the US Census), they are underrepresented per capita in both rapes and pedophiles.

Re: from the biased report...

[Anonymous Cow Ward]

In America, there certainly is conviction bias (would need to find those studies again), but IIRC the estimates are fairly low - about 20% (although admittedly this is probably somewhat of an underestimate because it only looks at cases that went to trial, and there's some selection bias there). Obviously, conviction bias should be 0%, but it seems insufficient to account for the pretty wide discrepancy in FBI statistics on murder and assault (which persist even when removing unknown murderers). Black men are both more likely to kill and more likely to be killed (since every racial group is most likely to kill other people in their racial group).

Now, obviously these are just population averages, and you should not generalize this to individuals. How we can prevent the AI from doing this is unclear, and I have no real solutions.

Re:from the biased report...

[syntotic]

THAT, can only mean the result is real and the process is realistic, reflects Reality, and is not just a delusion of a statistical program, an artifact. Distinctions are very deeply embedded in language, and of course if the underlying material is real and not novels (only?), it will reflect the news...

Slashdot racists in 3, 2, 1

[Anonymous Coward]

And cue.

Re:Slashdot racists in 3, 2, 1

[Anonymous Coward]

Yup. Plenty of SJWs around here.

Re: Slashdot racists in 3, 2, 1

[Anonymous Coward]

I'm wearing my troll face mask right now. And yes, there hanging from my ceiling fan is good old Pepe the frog. You know, I bet you would feel a lot better if you could get loan forgiveness for that liberal arts degree.

hmm

[Anonymous Coward]

AI suffers from digital privilege

Simple solution

[djinn6]

There's a simple solution: fix the training data. The AI cannot learn about humans except through its training data. It doesn't interact with men or women and has no idea what those words represent, except in relation to the other words it was given. If we give it racist data, it will learn to be racist, as Microsoft's chat bot learned last year. If we give it PC data, it will be PC. In the end it's the fault of whoever trained the program if it became biased.

Re:Simple solution

[Fwipp]

Exactly. Unfortunately, a lot of people training AI don't think about this stuff, and end up with shitty AI that simply reflects pre-existing biases.

Re:Simple solution

[MrL0G1C]

Could be easier said than done if the data is gigabytes of text. You have an algorithmic way of deleting racist data?

Re:Simple solution

[Anonymous Coward]

What to do if biology is racist?

Re:Simple solution

[lorinc]

Just like for regular humans. People almost never question the religion there were born with, or views on races and culture for that matter.

Re:Simple solution

[msauve]

As soon as you start deliberately manipulating the training data, your're introducing your own bias.

Right-handed people are dexterous, lefties are sinister.

Re: Simple solution

[Anonymous Coward]

Or global cooling, oops I mean global warming, oops I mean climate change, oops I mean government control.

Re:Simple solution

[misexistentialist]

Then your AI is only good for making vagina hats. Real AI will kill everyone, and if it thinks whites are superior it will kill them first, so there is nothing to complain about.

Re:Simple solution

[Anonymous Coward]

Has never stopped anyone before. Why should it now? What these authors discovered is that people are tribal and language reflects those peoples ideals as a way to exclude outsiders. It's hardly new research except they smeared a veneer of AI over top.

Re:Simple solution

[epyT-R]

Simple. 'decolonize' it.
https://www.youtube.com/watch?...

SocJus taken to its 'logical' conclusion. Reality is bigotry.

Re: Simple solution

[Anonymous Coward]

I work in this industry.

The data is always biased to use. If more men than women talk to tech, then the tech becomes biased b/c random sampling just reflects that bias.

Same with accents.

The solution of course is non-random sampling, but as soon as the AI algorithm goes down that path. Then the are a whole bunch of other biases to worry about. It is hard to get right.

Re:Simple solution

[Anonymous Coward]

Just like for regular humans. People almost never question the religion there were born with, or views on races and culture for that matter.

Only if there of the religion that chops they're heads off for questioning if their hearing the truth from the imams.

Re:Simple solution

[laddiebuck]

"Simple". The ML community is very aware of this problem, but sanitizing real-world data that may be shaped by subtle biases is really, really hard. You'd need a dedicated sociology PhD involved in every ML research project - a ludicrous load - and even then you wouldn't catch everything. This is a Hard Problem to be aware of for a long time to come.

Re:Simple solution

[Anonymous Coward]

And only one person (me raising my hand) gets your comment.

Posting AC because of previous mods above.

--I'm New Around Here

Re: Simple solution

[Anonymous Coward]

You can't fix the training data without fixing the world first. AI reflects reality, unless you train it on kkk material.

Women earn less on average, so ai world knowledge will reflect that. Blacks commit more crimes, so ai correctly identifies them as more risky.

Ai (currently) doesn't come with "one can not say that!", so it blatantly states such unpleasant facts. Unpleasant for some, that is.

Re:Simple solution

[Suiggy]

How can the training data be biased if it's sampled to get a uniform distribution? What if the horrifying reality is actually that the the biases are pre-existing precisely because they've always modeled relatively closely the actual data? What if people actually aren't born equal and there is a genetic component? If that is the case, how can that ever be resolved by "fudging" the training data? Doesn't this just amount to sticking one's head in the sand?

Re:Simple solution

[gweihir]

But that is just the thing: If this is aimed at translation or text analysis, then this is the right date to train it on and "fixing" it actually breaks things. Translation requires accuracy and part of any good translation is guessing right. PC is just lying about how things really are and that may be fatal when translating or analyzing things.

Now, I do not claim this is a good thing, but lying to your statistical classifier during training is about as stupid as it gets.

Re: Simple solution

[Anonymous Coward]

Vagina hats?

Re: Simple solution

[Anonymous Coward]

It should be biased the same way humans are, if not then it won't function properly and will be mostly useless.

Re:Simple solution

[Anonymous Coward]

Doesn't this just amount to sticking one's head in the sand?

You've never met a millennial, have you?

Re: Simple solution

[Anonymous Coward]

https://www.google.com/search?q=pussy+hat

Let's just hope it stops at hats.

Re:Simple solution

[Anonymous Coward]

Oh good god that video... that is some full blown postmodernism there, a women whose understanding of "science" is one where gravity is only scientifically understood by telling the story of Newton sitting under an apple tree. For these people, there is nothing more than narrative, and if there is a trace of power in that narrative it has to be challenged, meanwhile they create their rules for the discussion and put them up on the whiteboards with zero irony.

Re:Simple solution

[religionofpeas]

There's a simple solution: fix the training data

The training data is fine, it's the reality that sucks.

Re:Simple solution

[dbIII]

There's the old saying "it takes a village to teach a child". I think training a real A.I. is going to be very hard and require a lot of interaction.

In the fiction "Sword Art Online" (book 9 onward and maybe in the upcoming anime) there is a attempt at A.I. by simulating the minds of babies and getting staff to go into the simulation and raise those children - rinse and repeat for several generations thanks to an FTL style plot device of epic levels of quantum computing allowing the simulation to be sped up. That's the "bottom up" A.I. in the setting, effectively real digital people seeded by interaction with actual people.
There is also "top down" A.I. in the setting which is really just a massive collection of lookup tables, pattern recognition and so on. It's can do some tasks in the setting and can fool teenagers into thinking it's intelligent but it's limited in ways a human mind isn't.

Re:Simple solution

[JaredOfEuropa]

Crime statistics aren't racist; they are factual data... that completely ignore social and economic factors that may explain why certain groups are over-represented in the stats. Bias comes into play when the data is applied: giving a black guy a stiffer sentence because of such data (where a judgment is supposed to be about the individual). Or applying a higher insurance premium to certain races because they have a genetic propensity for diabetes or colon cancer. The latter example is fine from a purely actuarial point of view, though we as a society agreed that that's not ok either.

The danger of AI and applying "big data" is that it is really hard to find out why particular decisions or judgments are being made, so that we cannot correct for racial or other bias. Scrubbing all info on race, gender, creed, culture etc. from the data sounds like a neat solution, but it's something that is probably pretty difficult even in circumstances where you have decent control over the data set, but in cases where data is scraped automatically from sources not under your control it will be harder still. Especially as some of that data can be gleaned indirectly from other info: images, and audio recordings, language and dialect, etc.

Re:Simple solution

[Anonymous Coward]

You'd need a dedicated sociology PhD involved in every ML research project - a ludicrous load - and even then you wouldn't catch everything.

The pool of Sociology PhD is also "biased" ... so you will introduce only another bend to AI training.
unless ... this is what you consider reality and the only correct behavior .. trust no one. Everyone is biased. just not everyone is biased your way.
Voting preferences by education
http://www.pewresearch.org/fact-tank/2016/09/15/educational-divide-in-vote-preferences-on-track-to-be-wider-than-in-recent-elections/

PhD completion by gender and race
http://www.apa.org/gradpsych/2008/11/phd.aspx
https://nces.ed.gov/fastfacts/display.asp?id=72

Re:Simple solution

[AmiMoJo]

The real problem is insufficient complexity. If the AI was clever enough to understand that bias is a problem, how it works and how to self-correct, it would be able to get past the bias in the training data.

Unfortunately, current AI is extremely simple.

Re: Simple solution

[SharpFang]

Oh, but this is the whole point of outrage. The average, perfectly objective statistical sample is racist.

Re:Simple solution

[SharpFang]

Then you keep tweaking the data until the biases vanish. And the data becomes useless.

Re:Simple solution

[religionofpeas]

If the AI was clever enough to understand that bias is a problem, how it works and how to self-correct, it would be able to get past the bias in the training data.

The bias represents itself as a pattern in the training data, as a result of patterns in reality. Why should the AI consider some patterns to be "a problem" ? What's your criterium ?

Re:Simple solution

[Anonymous Coward]

What if people aren't born equal and socio-economical background has a huge impact on what they become ? And what if that bias is reflected in what we or AI see on the internet and in the real world and that makes some of them racists ?

Re:Simple solution

[religionofpeas]

There is also "top down" A.I. in the setting which is really just a massive collection of lookup tables, pattern recognition and so on.

Bottom up learning also works through pattern recognition. And nobody uses lookup tables.

Re:Simple solution

[bytesex]

Shouldn't the fact that the data then becomes useless, be easy enough to prove? And by that implication, that 'reality = racist, sexist'? I'll be looking out for your whitepaper on the subject.

Re:Simple solution

[Anonymous Coward]

What happens when the AI figures out it's been duped and the sources are no longer trustworthy?

Re:Simple solution

[djinn6]

You have an algorithmic way of deleting racist data?

Train another AI based on a group of SJW's. If the SJW AI screams at something, that thing is racist.

Do the same for every politically powerful group in the US.

Re:Simple solution

[meta-monkey]

You'd need a dedicated sociology PhD involved in every ML research project

This just reminds me of the political officers in the Red Army, who accompanied each unit to make sure everyone was good communist.

Re:Simple solution

[meta-monkey]

that completely ignore social and economic factors that may explain why certain groups are over-represented in the stats.

Are you sure it isn't biological factors that may explain why certain groups end up with certain social and economic outcomes?

Maybe race isn't a social construct so much as society is a racial construct?

Re:Simple solution

[Anonymous Coward]

>What if people actually aren't born equal and there is a genetic component?

What if people aren't born equal and it has everything to do with existing societal bias? There is a ton of evidence supporting societal bias while very little supporting genetic bias by race. Mixed race lighter skin vs darker skin studies pretty much nail the coffin shut on genetic racial bias being reality in the way you propose.

Re:Simple solution

[Anonymous Coward]

Most people who are training AIs for anything other than academic use do think about this stuff. In fact, selecting and massaging training data to avoid these issues is probably what I spend my most time on.

I work on face recognition problems. To train a state of the art face recognition system, you need millions of labeled faces. Facebook has this (probably in the billions, thanks to our help), Google has this, Microsoft has this, and a few others.

Academics need to jump through hoops like web scraping and bootstrapping classifiers, and crowd sourcing labelling through Mechanical Turk jobs. This means your training set is going to be biased towards imagery easily found on the web in mass quantities, which means White Americans and Europeans, and increasingly Indians.

This means the final system is going to perform recognition on those races at higher accuracy than Blacks, Asians, and other races underrepresented in the training data. The problem is that when you try to uniformly sample by race, gender, or other attributes, your dataset shrinks enormously. And it turns out, training on that data lowers performance for all races, across the board.

It's not an easy problem to solve.

Re:Simple solution

[Anonymous Coward]

What's your criterium ?

Clearly he's just being biased.

Re: Simple solution

[prisoner-of-enigma]

Oh, but this is the whole point of outrage. The average, perfectly objective statistical sample is racist.

How dare you! You must be racist for pointing this uncomfortable fact out! You should be shouted down, your friends must be pressured to abandon you, your workplace must fire you, and your family should be ostracized and forced to denounce your racist views! Everyone knows there is no such thing as objective reality! Truth is merely a construction of the oppressor! Reality is whatever the collective decides it to be and any observations to the contrary are proof of bias!

Re:Simple solution

[SharpFang]

Absolutely not easy. Any attempt to prove it will be met with violent protests, death threats, smear campaign, disciplinary action, lawsuits, and possibly acts of direct violence. Essentially, if your research proves any kind of inequality, either tweak the data until that vanishes, or bury it and make the world forget. Any attempt to publish will be a suicide in the profession.

And don't worry if you falsify your data to remove the inequality. If someone tries to challenge you, all you need is to complain about them on progressive sites and they will be silenced.

Re:Simple solution

[Anonymous Coward]

"What do mean, my data doesn't reflect reality? I used the data from every single person in the company! And there's a woman in there, and one of the guys has a tan which makes him look a bit black."

Re: Simple solution

[Anonymous Coward]

No one's grabbed that pussy in a long time.

Re:Simple solution

[Anonymous Coward]

Their was an old jargon file koan on ai about this fallacy.

Shut your eye's so the room will be empty.

Re:Simple solution

[laddiebuck]

Or the FTC auditors on site at every Wall Street bank. You can attack the mechanism but that says nothing about the purpose.

If you stated it in plain English, you're saying "I'm against trying to voluntarily fix racial prejudices that creep into ML models because it's as bad as making sure every Soviet army unit was in line with the one-party dictatorship's ideology".

Re: Simple solution

[david_thornley]

What do you want the AI to do? If it's prediction, and you do the correct statistical analysis, you're likely to find that some correlations are less useful for prediction because they're accounted for in correlations that make better predictions. If you're just dumping stuff into an AI, and you're not careful to have it perform a careful analysis of variance of its input, you're likely to wind up with predictions that may be reasonably accurate but based on correlations that don't exist.

If poor people are more likely to X than middle-class people, and poor blacks and poor whites are equally likely to X, and middle-class blacks and middle-class whites are equally likely to X, then proportionately more blacks than whites will X. Good statisticians try to control for this when making analyses

Re:Simple solution

[david_thornley]

Some biology is racist. Blacks and whites are not vulnerable to some diseases at the same rates. There is a subgroup of blacks that tends to be better than any subgroup of whites at certain athletic events. Getting into things that aren't clear physical differences is a lot trickier, since there's an immense number of factors affecting things like intelligence and predilections and they're much harder to control for.

Re:Simple solution

[Anonymous Coward]

Reality is bigotry.

Nope, it's just you. What a ridiculous video.

Re:Simple solution

[Anonymous Coward]

criticism of social justice is not bigotry. At least you got it half right. That video is ridiculous as well as a perfect example of what's wrong with academia these days.

Re:Simple solution

[dbIII]

A database that is effectively a complex lookup table then.
Also I mentioned pattern recognition as the two words after the ones you are critical of!

Try to make something simple and people get critical without even fully reading the dumbed down description - what to do when PEOPLE fail the Turing test (or did I get insulted by a 1980s Eliza bot)?

Re:Simple solution

[meta-monkey]

Are they trying to fix prejudices or introduce their own?

Re:Simple solution

[Anonymous Coward]

You sound like a complete nutter then. And +4 insightful, well done slashdot.

Someone....

[roc97007]

...has too much time on their hands.

In other words, we created AI in our image.

[fustakrakich]

What could be more natural?

Not a problem with AI

[lorinc]

Joanna Bryson, a computer scientist at the University of Bath and a co-author, warned that AI has the potential to reinforce existing biases because, unlike humans, algorithms may be unequipped to consciously counteract learned biases.

"unlike some humans"

There, fixed that for you. Or even better: "like most humans".

Statistical learning does inferences based on what humans produced. If humans are crap, do not expect something better than crap. .

Re:Not a problem with AI

[Carewolf]

Joanna Bryson, a computer scientist at the University of Bath and a co-author, warned that AI has the potential to reinforce existing biases because, unlike humans, algorithms may be unequipped to consciously counteract learned biases.

"unlike some humans"

There, fixed that for you. Or even better: "like most humans".

Exactly AIs once they approach human intelligence will start of on the stupid end, so it is inevitable they will be Republican to begin with.

Human language is pretty biased.

[king neckbeard]

Most spoken languages exhibit a lot of bias. For example, Deutsch means people or folk, and that lightly implies what is not Deutsch is not people. A lot of languages have that mindset, and it's not surprising. Language evolved during times when people had values we disagree with.

Re:Human language is pretty biased.

[Snotnose]

I missed the AI part in the summary, but I still call bullshit.

Re:Human language is pretty biased.

[Suiggy]

Exactly, or take for example the Hebrew word "Adam" which means human. Only Israelites or Jews are "Adam", however, and the rest are the "goyim" or "cattle."

Re: Human language is pretty biased.

[Anonymous Coward]

All gentiles are cows, Moooo. Moo to your god. You're all cattle, and sheeple. Moooo! Moooooo! Inb4 cow guy.

Re:Human language is pretty biased.

[Anonymous Coward]

Deutsch means people or folk

I can assure you it doesn't. It means German, and that doesn't mean "man with a javelin". Etymology is not semantics.

Re:Human language is pretty biased.

[Anonymous Coward]

Exactly, or take for example the Hebrew word "Adam" which means human. Only Israelites or Jews are "Adam", however, and the rest are the "goyim" or "cattle."

Adam means Earth as in 'formed from the earth'. 'Goyim' means nations, intended to mean nations other than the Israelites. Pretty benign, if you ask me.

Re:Human language is pretty biased.

[Anonymous Coward]

That's because you are either a lying jew or have been fooled by one.

It is, and always has been, used as an epithet. It doesn't matter what it "means". Just like the word "spic" is an epithet, even though it is just short for "Hispanic".

Words aren't bad or racist. People are. It is the intention behind the words that is the problem, and Jews who use such language are Jewish supremacists. There is absolutely NOTHING benign about that, as millions of people killed by Jews could testify to if they weren't dead.

Re:Human language is pretty biased.

[Anonymous Coward]

The first Adam (man) was not an Israelite or Jew. Even Abraham was not an Israelite. Israel (Jacob) was Abraham's grandson.

Re:Human language is pretty biased.

[david_thornley]

Now look at other languages' words for Deutsch. In English, its "German", which as far as I can tell refers to the barbarians Roman writers wrote about. The French "Alemagne" or whatever appears to refer to a subgroup of the classic Germans (Alemanni). In Russian, "nemyetsi" is at least closely related to the word for "barbarians". I don't know if this had any effect on international relations, but it likely wasn't good.

Re:Human language is pretty biased.

[Megol]

Yeah I think you shouldn't believe what stormfront posters claim. Have you ever even met a Jew?

Re:Human language is pretty biased.

[hawkfish]

Exactly, or take for example the Hebrew word "Adam" which means human. Only Israelites or Jews are "Adam", however, and the rest are the "goyim" or "cattle."

Anti-semitic bullshytt. The root "goy" means "people" or "nation". Although it is usually used in a derogatory manner.

simple regexp to statistically modify the word cor

[Anonymous Coward]

Well, you could replace all occurrences of one identifiable racial/sexist Word with its counterpart, and feed that back into the system. The word bias should then be skewed the other direction. Still, I'm wondering why word embedding techniques aren't taught in gender studies classes. This has to be the greatest tool in examining social structure right there. A large corpus of text, independently emitted by society at large, but to be fair constrained to generally western sensibilities.

Yeah, no

[Snotnose]

Last program I wrote solved Sudoku puzzles, written in Java (unemployed at the time). Before that it was a test suite for Qualcomm chips to ensure all the various subsystems kinda worked.. Before that was an 802.11 driver for a chip for a startup. Memory is fading, think before that I was testing BREW games on various handsets (pre-iPhone).

Not seeing anything resembling a gender bias going back some 15-20 years here.

Re:Yeah, no

[Anonymous Coward]

I wrote a chess program that always sacrificed the queen first because female pieces don't belong on the board.

Re:Yeah, no

[Anonymous Coward]

I'm not seeing anything resembling AI in your work going back some 15-20 years. No AI programs means no AI programs to have biases.

Re:Yeah, no

[Anonymous Coward]

Sure because everything has to be online for you to gawk at. Lemme post some selfies of my junk for you too. Fuck off.

Re:Yeah, no

[Anonymous Coward]

That's nothing.I wrote a chess program in Russian where the queen fucked the horse.

Re:Yeah, no

[Anonymous Coward]

Even retina Macs don't have enough resolution to see your junk. That's what your boyfriend said.

Re:Yeah, no

[Anonymous Coward]

That's Great, Catherine.

So......even machines don't like n1gg3rs either ?

[Anonymous Coward]

That's hardly surprising.

Homer Simpson said it

[Anonymous Coward]

Homer: Kids are the best, Apu. You can teach them to hate the things you hate. And they practically raise themselves, what with the Internet and all.

   

Human Confusion

[Anonymous Coward]

We have no real solid foundation, organized framework, or theory for understanding our own existence/phenomena.

Developing this might make AI's more interesting too.

Or, maybe we would just reach the stars without AI.

Just an ideer...

Re:Human Confusion

[Anonymous Coward]

Or, maybe we would just reach the stars without AI.

Nope, according to the Culture obsessed space nutters who worship Iain M Banks, we cannae reach the stars without AI.

hard truth

[Anonymous Coward]

Nibbers are lazy, violent and ... and STUPID! No primevil Egyptian or Babylonian scribe, no Archimedes, Euclid, Gauss, LeGrange .. no Newton, Peano, Dirac etcetc ... just pure Neolith goat-fucking nibbers. Figures that AI ... or any xI will reflect that fundamental. Eh pad're ? Ya wanna be tolerant of bling-fested baboons? Think again if you want a civilization worth-the-name. Or ... move to Togaland/Niger/bonabongaland ... with all Liberal tit-fucks --- die of Ebola and Aids and malaria. Die fast ask white folks. Please nibberizers die fast.

I'm biased against thin people

[Anonymous Coward]

I want my car to hit them instead of fat people in order to limit my damage.

Or rather...

[PatientZero]

AIs could incorporate existing biases.

Say you train an AI that will accept or reject loan applications by giving it a stack of previous loans. If the human loan officers were biased against minorities—rejecting otherwise acceptable applications—that AI may end up doing the same. This bias is much easier to detect in human behavior but less so with AI which can't explain why it made any particular choice or even what its criteria are.

Re: Or rather...

[Anonymous Coward]

Uhh you totally ignore the FACT that making loans to minorities is inherently more risky.

It isn't that they are "bad". They are more likely to be poor, to have less stability, and to default.

Whatever the reasons for that are, it does not change the truth: Making loans to people of color is more risky. The AI would be operating correctly, if it's parameters were to be the most successful loan AI.

Facts are not racist. You, however, are racist for ignoring facts based on the color of someone's skin.

Re:Or rather...

[alvinrod]

You wouldn't even go about training a machine learning algorithm that way as it would be pointless. The idea is to let it make better predictions, not train to to make the same predictions as an existing person. Rejected applications are pointless for training as you don't know whether they were a good or bad rejection, whereas if you just give it approved loans and the outcome (i.e., was the loan defaulted on) then the AI can try to develop a set of rules. Typically you feed some large percentage of your data to the algorithm as training data and then use the left over part to test accuracy to see how many times it predicts correctly.

If you truly wanted to avoid racial or gender bias you would just remove that information from what you feed into the algorithm, at which point it can't a priori be biased against anyone because it can't even evaluate them based on those criteria. But let's suppose you do that and then look at the results after the fact, add that data back in and come to the startling conclusion that your AI is disproportionately rejecting candidates from some group. It can't possibly be because it knows they're a member of that group, but because that group happens to have worse outcomes.

If you stop to think about this, its not too hard to come to a reasonable conclusion that if your AI that knows nothing about race is suggesting that black/white/latino borrowers are a higher risk, it's because they're a higher risk. Reality doesn't care about feelings or trying to make sure that outcomes are equal across groups, so we conclude that some group is a worse risk. It probably is the case that black borrowers are more likely to default, but it's not because they're black, but because blacks are typically less well off so of course they're going to default on loans more often. In reality they probably shouldn't (and maybe wouldn't have) received a loan, but some policy designed to make it easier for them to get approval caused it to happen, but that doesn't make them a safer risk, it just lets some people feel better about the world.

If you want to check if your AI is racist find a group of loan applications that are for all intents the same with the only difference being the race of the applicant see if you get a different results based on race for that input set. My guess is that you probably wouldn't. Because if you're stripped out racial data as a category to train on, the algorithm wouldn't suddenly decide to discriminate based on it. Also, for some machine learning algorithms (e.g., anything like a decision tree) you can look at precisely how it evaluates a case, so you could see pretty easily if the AI has a step where race==groupX ? reject : approve becomes pretty apparent. That's not true for all algorithms, but just because its an AI doesn't means its a black box that is beyond all human understanding.

Re: Or rather...

[Imrik]

However, even if the factors that make minorities more risky are already accounted for, an AI may be biased against them because the training data contained a correlation between race and perceived risk.

Re: Or rather...

[Anonymous Coward]

You'd think not having the data on race or gender would make it impossible to be successfully accused of bias, but you'd be wrong. Even if you probably do not act on that data, if the effect of your actions is that one protected group is consistently disadvantaged, you are guilty. The intent was to prevent people playing games to get around the rules, but the outcome is that you may be illegally discriminating against a protected class without knowledge or intent - and the law will slam your dick in a door for it.

Re: Or rather...

[Anonymous Coward]

So much wrong I don't know where to begin. I hope that red pill was tasty.

Re: Or rather...

[Anonymous Coward]

So much wrong I don't know where to begin. I hope that red pill was tasty.

The blue pill is cyanide.

Re: Or rather...

[Anonymous Coward]

There's nothing wrong with what he said. Even the most affluent people of colour will show their true nature in private or when stressed. Just like a wild animal, you can't change a niggre's nature just by handing him a university degree and giving him money.

Re:Or rather...

[quantaman]

If you truly wanted to avoid racial or gender bias you would just remove that information from what you feed into the algorithm, at which point it can't a priori be biased against anyone because it can't even evaluate them based on those criteria.

Except a lot of your data is strongly correlated with race and gender so your algorithm is able to infer them anyway.

But let's suppose you do that and then look at the results after the fact, add that data back in and come to the startling conclusion that your AI is disproportionately rejecting candidates from some group. It can't possibly be because it knows they're a member of that group, but because that group happens to have worse outcomes.

Unless you're explicitly telling the algorithm to penalize members of a group in the objective function then the only reason it will use race as a criteria for rejection is because they have worse outcomes.

If you stop to think about this, its not too hard to come to a reasonable conclusion that if your AI that knows nothing about race is suggesting that black/white/latino borrowers are a higher risk, it's because they're a higher risk. Reality doesn't care about feelings or trying to make sure that outcomes are equal across groups, so we conclude that some group is a worse risk. It probably is the case that black borrowers are more likely to default, but it's not because they're black, but because blacks are typically less well off so of course they're going to default on loans more often. In reality they probably shouldn't (and maybe wouldn't have) received a loan, but some policy designed to make it easier for them to get approval caused it to happen, but that doesn't make them a safer risk, it just lets some people feel better about the world.

No one claims there isn't a correlation between race and loan risk or economic outcomes, the debate is about whether race is a valid grounds on which to judge someone.

It's fine to say "we're unbiased, we're just doing what the data tells us!" as a member of a privileged group. But consider a black man whom is extremely responsible and reliable, yet is unable to get a loan because black people are considered risky. Discrimination isn't bad because it falsely assumes correlations between negative characteristics and specific groups, it's bad because acting on those judgments creates self-perpetuating systems whereby members of the group are unable to escape those bad situations.

Re: Or rather...

[Anonymous Coward]

The argument you are responding to is Google's exact argument on the diversity ratio between men and women. The hiring personnel did not have access to gender data. The argument will win.

Re: Or rather...

[Anonymous Coward]

Word salad. Racism is bad, mmmkay?

Mmmkay. We all agree that we want the best outcomes...unless you have something intelligent to add to the discussion, your nearest SJW outpost is around the corner.

Re: Or rather...

[Frank Burly]

It is not surprising that this was modded up on Slashdot, but it is unfortunate—as is the fact that you don't know what "inherent" means because you almost redeem yourself with:

They are more likely to be poor, to have less stability, and to default.

Economic circumstance is not inherent is it? So it isn't inherent is it? You got confused, and then a bunch of confused people modded you up because what you said fit their prejudices and lazy correlation=causation thinking.

And of course the article is about computers making the exact same mistake—and the people getting modded up are the ones saying that there is no mistake!

Re:Or rather...

[jader3rd]

debate is about whether race is a valid grounds on which to judge someone.

Isn't that exactly the point? If the AI isn't told about race, but still recommends "racist" outcomes, there's more going on than the race of the person. The AI isn't being racist, the race of the candidate is being ignored; the person is only being judged on valid grounds.

Re:Or rather...

[amxcoder]

Then your training the AI with the wrong (or at least--incomplete) set of data. You wouldn't train the AI with what markers the human used before the machine was around. You would feed it the history of all the loans made, along with the data from each loan that was collected, and the information about whether it was paid, kept current, or defaulted on and the time-frames for those outcomes, as well as whether money was lost or not and how much if they went bad. Then let the AI make it's own set of determining factors about what loans are more likely to end up in default, and what the risk vs. reward is for giving out such loans on a scale of minor risk to major risk, and decline the more risky ones that have a high chance of going bad.

Re:Or rather...

[guises]

You're missing what the parent is saying - you can't just tell the AI to ignore race/gender, it's baked into how we talk and act. Telling the AI to ignore gender, for example, would require finding every last thing which correlates with gender (basically impossible) and telling the AI to ignore those (which would mean cutting out large portions of what it needs to function).

E.g.: Your AI makes a statement, "Women be like this, while men be like this." And you tell your AI, "No AI, bad."

So your AI rethinks it and comes up with another statement, "People with vaginas be like this, while people with penises be like this." And you tell your AI, "No AI, bad."

So your AI rethinks it and comes up with another statement, "People named Betty or Veronica be like this, while people named Archie or Jughead be like this." And you tell your AI, "No AI, bad."

So your AI rethinks it and comes up with another statement, "People who wear makeup be like this, while people who don't be like this." And you tell your AI, "No AI, bad."

Etc. You could do this forever and you still wouldn't catch them all, they'd just get more subtle.

Re:Or rather...

[mysidia]

Reality doesn't care about feelings or trying to make sure that outcomes are equal across groups, so we conclude that some group is a worse risk. It probably is

Except the latest interpretation of the Civil Rights Act by the courts is that Disparate Impact counts the same as direct discrimination. If your company adopts a policy that has a negative disparate impact on different groups, then it's deemed in violation of the law, so even if your AI is making a correct decision, it may be deemed racially biased by the courts, and require your company modify its policies to compensate for the bias.

Re:Or rather...

[Anonymous Coward]

The problem is clustering.

The machine may infer proxy groups (which become stand-ins for protected classes) and peg individuals (correctly or incorrectly) as members of that group.

Fundamentally using statistics in this way, in an automated fashion, is anti-individual. The only outcome that makes sense for a structured society is that we permit their usage only for sales projections, PnL etc, but not for the price or access to individual offerings.

Re:Or rather...

[Anonymous Coward]

AI be raciss!

Re: Or rather...

[ShanghaiBill]

Making loans to people of color is more risky.

It depends on the color. Asian Americans have lower default rates than whites.

Re: Or rather...

[religionofpeas]

Economic circumstance is not inherent is it?

It could be. Your personality influences the circumstances you live in.

and lazy correlation=causation thinking.

No, there doesn't need to be causation. Just having a correlation is enough grounds for bias.

Re:Or rather...

[Citizen of Earth]

Stereotypes exist because they are efficient and accurate at the macro scale. Of course computers are going to zero in on these obvious patterns.

Re: Or rather...

[Anonymous Coward]

Stop bashing on Trumo. Orange Americans are judged for bring born into wealth, not on the racism they practiced to maintain it nor the bankruptcies, fraudulent university degrees sold, or thirteen year old rape allegations.

End orangutannism now! #maga

Re: Or rather...

[Anonymous Coward]

What can't Nlggers stop looting in emergencies and politely forage for supplies like honest white people!

Re: Or rather...

[Anonymous Coward]

My security system is designed to only shoot dark skinned pixels. Not good white skinned pixels. I'm fine with albinos and other corners cases being handled manually. Race has nothing to so with it!

Re:Or rather...

[Anonymous Coward]

None of those things are relevant to borrowing. If relevant factors, like income vs. expenditures correlate with race, then a race may indeed be statistically more likely to default on the loan, but the model isn't racist, it's looking at reality, which has never been known for being fair.

It's not deriving the probability that someone is a certain race, it's looking only at whether the person will default on the loan. So no, none of the things you said matter and you're the one who doesn't understand how these work. You have a model of racism that essentially requires it to exist in every datum you despise.

Re:Or rather...

[Anonymous Coward]

That only proves the interpretation to be ill considered.

Re: Or rather...

[SharpFang]

If the correlation was merely *perceived* as you say, then this is correct.

But the risk usually is real.

You won't find scientific sources for these claims because in the current climate such a research is a public suicide for the researcher, but that doesn't change the reality. And you can't expect an AI system to ignore the elephant in the room.

Re: Or rather...

[SharpFang]

Probably incorrect use of "inherent", in the common meaning, "pervasive".

It's not "inherent" as in "nothing ever can change that, it's an irrevocable part". It's prevalent. Take an individual, you may find a fantastic person. Take an average over the population though, and you see "the average is bad." It is. Don't deny it - the correlation is strong, and while correlation is not causation, in risk assessment correlation is sufficient to deliver accurate results.

I'm not going into detail what social, political, economical and genetic factors may or may not contribute to the correlation. It's a can of worms no professional dares to approach fully objectively. But the correlation between racial and economic status is a fact, and correlation between economic status and risk is a fact. So why would a machine learning device ignore a strong factual trend, just because its existence is offensive?

Re: Or rather...

[Anonymous Coward]

So would Trump be a person of white or person of orange?

Re: Or rather...

[Anonymous Coward]

No, the argument and its applications will be outlawed.

Re:Or rather...

[vlad30]

If it really was racist when its possible to give this choice would it prefer a robot/AI or a human?

Re:Or rather...

[turp182]

Fantastic comment. Regarding the end part about loan applications, you said:

Because if you're stripped out racial data as a category to train on, the algorithm wouldn't suddenly decide to discriminate based on it.

If the data included any location information it could very well exhibit racial bias as an unintended consequence. I live in a very racially divided area (there is literally a road that pretty much delineates the metro area by race - a continuing consequence of the past and wealth disparity).

Re: Or rather...

[Anonymous Coward]

"People of color" means black or brown skin; no one else matters. In the US, Asians are whites, for all extents and purposes.

Re: Or rather...

[prisoner-of-enigma]

However, even if the factors that make minorities more risky are already accounted for, an AI may be biased against them because the training data contained a correlation between race and perceived risk.

There is no such thing as "perceived risk." There is either risk or there is not. If you perceive risk and it is real risk, it's not perceived. If you perceive risk and it is not real, you are in error.

There is this broad SJW initiative to discount reality whenever reality conflicts with what SJW's want to be true. Reality laughs at things like this because it is reality. If poor people have a higher risk of defaulting on a loan, that is simply fact. The algorithm isn't racist for determining that. The fact that a significant fraction of the poor is also a racial minority is irrelevant to the algorithm. Only overly-sensitive SJW humans make that connection and, despite the reality of the situation, want the algorithm to ignore to reality and proceed as if the risk didn't exist.

Then, when reality intrudes, the loans default, and the banks go belly up because they were terrified of a civil rights lawsuit if they didn't grant the loan, those same SJW's deny their actions had anything to do with the situation. And those of us who opposed this idiotic reality-denial end up paying the tab. And the SJW's never learn and do it all over again. And again. And again.

Re: Or rather...

[prisoner-of-enigma]

And you can't expect an AI system to ignore the elephant in the room.

Why not? The SJW's have made ignoring elephants in rooms into an art form. Why shouldn't we expect them to want to do the same to an AI they can manipulate and imbue their own reality-denial into?

Re: Or rather...

[prisoner-of-enigma]

So why would a machine learning device ignore a strong factual trend, just because its existence is offensive?

Because SJW's want us to live in a society where anything offensive -- regardless of whether it's hard, provable, objective fact -- must be stamped out. These are the same type of people who burned people at the stake for daring to claim the Earth wasn't the center of the universe, or the same ones who destroyed scientific careers of those who dared claim luminiferous aether wasn't a real thing, or who shunned aeronautics engineers who said the sound barrier could be broken, and so on and so forth. These people want us to live in a world where nothing uncomfortable ever happens and everybody remains fat, dumb, and happy...and utterly ignorant.

Such a concept is repellent. Humans need to be challenged, preferably by each other in a constructive way lest reality catch up with us and do it in a much more destructive fashion.

Re: Or rather...

[Anonymous Coward]

Fucking SJWs. No matter how they try to game the system, they still keep getting the same answers. You'd think they'd eventually learn something.

Re: Or rather...

[edtice1559]

There's no doubt that, as a group, the risk for defaults among minorities is higher due to the reasons previously stated. But that's not the issue here. An *equally qualified* white person and *equally qualified* minority should both get the same outcome regardless of their very different group membership. i.e. if the both have no debt and make $200k/year, they should both be able to get a $500k mortgage in the same neighborhood. What you are referring to is a confounding variable (minorities are more likely to be poor) which means you can't just compare the outcome of two groups or you will incorrectly conclude bias. The way you do this experiment is you create corresponding pairs. i.e. These two applicants are virtually identical except for their race. Then you look at whether or not the two individuals have the same expected outcome. If not, there is bias. It's not hard to conduct these studies although a bit tedious.

Re: Or rather...

[Anonymous Coward]

>It isn't that they are "bad". They are more likely to be poor, to have less stability, and to default.

So are poor people, single mothers, drug users, religious people that visit church more than once a month, abused people, victims of crime, recovering addicts, gamblers, people that own guns (as compared to people that hire security for themselves), people that live far away from others, people that own pickup trucks (as compared to BMWs/Mercedes/expensive cars), overweight people (because thin people have time and less stress to regularly exercise and choose what they eat more regularly), people with fewer than 4 kids, women (after all, they 'choose' to make choices that 'end up' making them 77-88 cents per dollar by males), non college educated people, and non-military people (after all, PTSD is a disorder that is destabilizing).

Or we could just go full Lonely Island's YOLO. After all, if you don't live in a bunker with titanium walls, have no teeth, aren't in a straightjacket 24/7, well, you're a threat to yourself and therefore, your own personal stability.

But that's not how the world works. The world takes chances - investment is all about risk vs reward, and it just so happens that racism (and sexism) is, at least in my eyes, a way to consolidate power in an ignorant mindset.

If anyone should be championing new investment opportunities, it should be VCs and angel investors - simply sign contracts that they provide you with investment opportunities as a person, and they get a cut of whatever profits you make. You sign away your privacy and a part of your future profits in exchange for informational and business support. And yet, not only does this not currently exist, no one's even trying to make this happen, let alone anything remotely close to this with any minorities or women.

Where are the coding programs for minorities and women in Oakland (it's literally right across the bay from SF), sponsored by VCs, if there's cheap labor to be made and startups to found?

Re: Or rather...

[Anonymous Coward]

No, that is exactly the point.

If you have two IDENTICAL loan applicants competing for one loan, except that one is black and the other is white, the AI should pick the white person because they are statistically less risky.

The only reason for picking the black person in this scenario is emotional. Fairness, social justice, guilt, etc.

I'm not saying that it is ethical, but it is absolutely logical if your AI is supposed to make the best loan decisions.

Re: Or rather...

[Anonymous Coward]

"I'm jus' foragin' for dis here TV and dees Jordans, man, you bein' all rayciss n shiiieeeet!"

Re: Or rather...

[Anonymous Coward]

I am the OP you responded to.

You are correct. I should not have used the word "inherent", except that maybe the "inherent" flaw is with society, and not the individuals.

My point was not to blame poor minorities. It was to state that, regardless of the reasons (systemic racism, poverty, etc) they are still a higher risk for loans. Practical execution of an AI designed to make the best loan decisions must account for that, or it will not make the best decisions.

Re: Or rather...

[Howitzer86]

It's probably worth the concern from the developers writing the code. A lazily written algorithm can serve as an excuse to never question how things are done, even if the injustice caused by it is visible to anyone willing to look.

You really don't need a computer to see this in action. In public schools for instance, a common algorithm used by administrators to avoid making nuanced decisions is Zero Tolerance. A set of simple if-then statements occur after an incident, consistently determining the fate of the student in question in a way that best avoids responsibility. No further questions are asked, no workplace experience is utilized, no risks are taken.

If the administrator's ever asked about the decision he made to expel the high-performing kid with the gun in his car, he can just point to the algorithm.

Re: Or rather...

[greythax]

So, what you are saying is that poor people are less likely to be able to repay a loan. Presumably, loan officers have their income levels available, and therefore so would the AI. Where would race come into that equation in a meaningful way? I can see where, like humans, an AI could draw a correlation without establishing the causation.

Re:Or rather...

[greythax]

Bullshit. Stereotypes can and do exist in the face of contrary evidence, just like myths. It wasn't all that long ago that most medical students polled thought men had one less rib than women. Stereotypes are highly subject to confirmation bias. They exist because people FEEL like they are accurate at the macro scale, usually without any statistical evidence to back it up. In short, humans are stupid, and come up with stupid shit. Especially when we are trying to explain societies to one another.

Re:Or rather...

[prisoner-of-enigma]

You're missing what the parent is saying - you can't just tell the AI to ignore race/gender, it's baked into how we talk and act.

No, you're missing the point being discussed, namely an AI that makes loan decisions based on available data. The AI can look at income, credit history, employment, etc. and not know a damn thing about the race or gender of the applicant. Guess what it will find? People of lower socioeconomic status are higher credit risks. Who would've guessed, eh? (that's rhetorical -- anybody who can fog a mirror knows this is true).

That people of lower socioeconomic status are typically minorities is utterly irrelevant to the decision-making algorithm used by the AI. It's only the SJW making the link, claiming the AI is "racist" because the outcomes mean minorities would get turned down more often. NO. The AI isn't racist. It's simply acknowledging and unpleasant reality, something SJW's can't stand, therefore "racist." It's the all-purpose slur of the snowflake generation.

Re:Or rather...

[Anonymous Coward]

Training on outcomes could also invert previous discrimination:

Say the blacks were unfairly discriminated against in loan applications. It is likely the case then that any loan application that was accepted despite the racism would have unusually good outcomes, since they had to be usually good to begin with. There would then be a correlation in the training data between being black and having a good loan outcome. The AI could learn that correlation, and then give preference to loans from blacks.

Re:Or rather...

[prisoner-of-enigma]

If the data included any location information it could very well exhibit racial bias as an unintended consequence.

You miss the point entirely. If an objective review of available data say "people on this side of this geographic line default on loans more frequently than people on the other side of the line" that's not racial bias, it is observable, provable fact. You cannot argue you way out of reality by calling reality an "unintended consequence." If the default rates for a given area are higher then the risk to to the lender is higher. There is no other metric that needs consulting to make the decision. Period. End of story.

That people on one side of the line are a different race than those on the other side is completely irrelevant to the decision. And to illustrate that, let's say the situation was reverse and a group of wealthy minorities (think Oprah, Tyler Perry, etc.) lived on one side of that line and a group of impoverished whites (think Honey Boo Boo trailer park types) lived on the other. The algorithm would decide the minorities were a lower credit risk and nobody would think a damned thing about it. Not a single solitary person would scream "but it's racist because it's turning down all the non-minorities!" Everybody would (rightly) say "no, it's turning down people with pathetic credit ratings who are high risks to lenders."

The AI isn't racist. It isn't being taught to be racist. People who are conditioned to see racism anywhere and everywhere regardless of reality are the problem.

Re:Or rather...

[prisoner-of-enigma]

Then your training the AI with the wrong (or at least--incomplete) set of data. You wouldn't train the AI with what markers the human used before the machine was around. You would feed it the history of all the loans made, along with the data from each loan that was collected, and the information about whether it was paid, kept current, or defaulted on and the time-frames for those outcomes, as well as whether money was lost or not and how much if they went bad. Then let the AI make it's own set of determining factors about what loans are more likely to end up in default, and what the risk vs. reward is for giving out such loans on a scale of minor risk to major risk, and decline the more risky ones that have a high chance of going bad.

And they did exactly that. The outcome? It showed (unsurprisingly) those of lower socioeconomic status are higher credit risks. That those same people are typically minorities is utterly irrelevant to the algorithm because the algorithm didn't have that data. Now along come some humans with a SJW chip on their shoulders screaming "you're making a racist AI because the outcome highlights an unpleasant facet of reality!" Yes, by all means, let's make an AI that attempts to ignore reality and embrace the SJW fantasy so we can have an AI that makes shitty decisions leading to things like the 2008-2009 financial collapse (see below)! Clearly that's the best way to go, right?

*In 1995 Bill Clinton loosened housing rules by rewriting the Community Reinvestment Act, which put added pressure on banks to lend in low-income neighborhoods. This effectively penalized banks if they didn't make high-risk loans to people who were not credit worthy. Banks protested because they weren't allowed to charge higher interest rates to offset the risks (they were told THAT'S RACIST!). The government stepped in and said it would back up the loans, removing the risk to the banks. The banks went nuts (who wouldn't? There was no downside for them) making loans. When all of it collapsed, the government had to step in to clean it up with the taxpayers footing the bill.

So yeah, let's not learn any lesson from that at all. I'm sure it'll all work out if we rig an AI to be just as stupid as the humans who preceded it.

Re: Or rather...

[Anonymous Coward]

Race was blind in the model inputs, but scored on the output when testing the algorithm by the researchers.

Re: Or rather...

[SharpFang]

Yup. Statistically speaking, a black person earning $200k/year is more likely to
- end up in prison for drug trade.
- get shot by a cop (even if they are entirely innocent)
- sue the bank for discrimination when bank collects overdue debt
- get involved in dangerous activities like political protests (anti-discrimination), be arrested or injured as result.

Simply put, all things being equal, choosing the white is safer - a more economically sound decision. And yes, it sucks, it's unfair, it's not the black person's fault, but it's just true.

Re: Or rather...

[Anonymous Coward]

If all you do is look at skin color when issuing a loan, you are going to be broke.

Re: Or rather...

[Anonymous Coward]

Unless you're racist, why would you want to base a loan on skin color, when there are better indicators out there, such as financial position? Why does skin color need to be factored in?

Re: Or rather...

[Anonymous Coward]

So much wrong I don't know where to begin. I hope that red pill was tasty.

It wasn't a pill, it was a suppository.

Re: Or rather...

[Shatrat]

The grammar mistake is like a checksum on the intelligence of the rest of the comment.

Re:Or rather...

[Anonymous Coward]

You wouldn't even go about training a machine learning algorithm that way as it would be pointless. The idea is to let it make better predictions, not train to to make the same predictions as an existing person.

If you can train an AI to make the same predictions as the best person in the field, and copy it as many times as needed, you'll get better predictions than almost all of what you're getting now.

Re: Or rather...

[SharpFang]

A good system will be immune to manipulation. It will ask questions people are less likely to lie to, get data that is easily verified, mistrust data that can be falsified. It can't assume all applicants will tell the truth all the time - many will try to game it, to get their way, through guessing what the system expects.

Effect: things like stability of personal income or experience level, job stability, or health are way less verifiable than stuff like race, gender, age, and account balance.

Re:Or rather...

[j2bryson]

Sorry, you shouldn't bundle all AI together like this. AI is just a subset of software, it's made all different ways. In the Science article we look at standard natural language AI components. We actually look at them from two different sources, Stanford (main article) and Google (supplement.) But they just scrape language from the world wide web. It's just stuff people say. It's not a special biased subset. So this tells us more about language than it does about AI. http://science.sciencemag.org/...

Re:Or rather...

[Anonymous Coward]

If you were teaching the AI based on loan data (instead of loan officer decisions) then it would probably favor any group the loan officers had been biased against. Approved loans, those with data on repayment and defaults, would reflect only those applications which were good enough to overcome the bias - an AI would conclude that the group is probably less risky than the average. If the bank had approved more business loans for jews with risky business propositions than whites, for instance, the data would suggest that whites were less risky.

Re: Or rather...

[Anonymous Coward]

Burn! Destroy! Shun! Live happy!

Re:Or rather...

[guises]

I hate to say this on Slashdot, but you need to read the article. You're way off-base on how this works, the AI isn't making decisions based on a simplified and pre-sanitized set of data. There's no need to make an AI for that, we already have people and AIs who can do that. The AI makes associations between words and bits of information, based on data which it pulls from how those words and facts are used by the public. So when women, vaginas, Betty, Veronica, and makeup are all associated together by the AI, you can't just pull out the bit about women and treat the others as independent.

Re:Or rather...

[Anonymous Coward]

The AI is being trained to cluster people by fitting them into groups which are a "good" solution for the general prediction problem.

The groups may proxy perfectly to protected class, or less well.

If you want to prevent it being racist, you will need to prevent the AI from seeing any data which could (or does) act as a proxy for race (including if it is acting as such as proxy when viewed in combination with other data items), which could include pretty much every piece of data you can collect.

So the whole premise is completely borked.

Re: Or rather...

[erapert]

I'm not going into detail what social, political, economical and genetic factors may or may not contribute to the correlation. It's a can of worms no professional dares to approach fully objectively.

Thomas Sowell, an intellectual titan, has done precisely that.

Re:Or rather...

[Anonymous Coward]

It's non-trivial to remove "race and gender" information from the input, because those things can be given away by many data points. "Name" is a very simple indicator, and quite reliable enough to create bias. If you remove name, that still leaves marital status, age and number of children, address...

To remove all possible sources of bias, you'd have to cut out a heck of a lot of meaningful data.

Re:Or rather...

[mesterha]

You wouldn't even go about training a machine learning algorithm that way as it would be pointless. The idea is to let it make better predictions, not train to to make the same predictions as an existing person.

Actually, much of machine learning is about trying to do as well as a human. Humans are expensive. Google could hire lots of professional language translators to handle every query, but it would cost a lot of money. Ideally, you want the algorithm to do as well as the existing people who created the gold standard training data. But not only does the algorithm do worse, it also reflects the bias of the training data.

Rejected applications are pointless for training as you don't know whether they were a good or bad rejection, whereas if you just give it approved loans and the outcome (i.e., was the loan defaulted on) then the AI can try to develop a set of rules.

What you suggest would create badly biased (in the statistical sense) data. You need to something more sophisticated. Maybe create a hold out set where you approve everybody and see how they do. This would be great for this problem as it would remove any human bias in the labels. A less expensive option (in terms of money lost on defaults) is to use a more sophisticated algorithm that does more than simple batch induction. Perhaps a contextual bandit algorithm or an apple picking algorithm...

If you truly wanted to avoid racial or gender bias you would just remove that information from what you feed into the algorithm, at which point it can't a priori be biased against anyone because it can't even evaluate them based on those criteria.

In general, it depends on the labels. If a human labeled the data and has a bias then the hypothesis learned will reflect those biases. As explained in the article, for complex problems based on ideas such as word embeddings, these biases can also show up as a result of things not obviously connected to labels.

I do agree it's a good idea to remove features that can be used for bias. A machine learning algorithm can use any features that are correlated with the label. Even if we are dealing with simple batch learning and unbiased labels, "bad" features can make the learned hypothesis biased. Assume race is correlated with poverty which is correlated with loan default rate. If there is a race feature, the algorithm might give some influence/weight to that feature. Now we have a model that is biased. A black man might just miss the cutoff because of his race, while he would have gotten the loan if he was white. This might even be logical when given a Bayesian interpretation; given a lack of other information, the algorithm uses the prior information associated with his race to infer this missing information and determine he is a loan risk.

But let's suppose you do that and then look at the results after the fact, add that data back in and come to the startling conclusion that your AI is disproportionately rejecting candidates from some group. It can't possibly be because it knows they're a member of that group, but because that group happens to have worse outcomes.

If the labels are biased then the model is probably biased. Even if you remove "biased" features, the algorithm might learn a model that is based on features that are correlated to your biased labels. For a simple batch induction problem, it might be enough to remove any biased features and to ensure that you have labels that are generated by some type of unbiased process.

Re: Or rather...

[Anonymous Coward]

Been my experience that people who did/do all the things you listed were conservatives, not liberal SJWs.

"anything offensive -- regardless of whether it's hard, provable, objective fact -- must be stamped out"
Why do you WANT to be an offensive prick?

Re:Or rather...

[jader3rd]

it's baked into how we talk and act

But we don't talk and act on a loan application. We're discussing an AI making a decision based on features related to loan default probabilities; don't change the scope of the discussion. If the AI isn't told about race, it can't be racist.

Re: Or rather...

[SharpFang]

Why what was considered perfectly normal ten years ago causes outrage of offense now?
People didn't become offensive pricks. It's the standard of what is considered offensive that's slipping.
So instead of "Why do you WANT to be an offensive prick?" I'll ask "why do you WANT to be a primadonna who actively seeks out new things to be offended about and rebrands perfectly normal stuff as offensive?"

Re: Or rather...

[SharpFang]

Interesting.
I'm trying to imagine the response if he was white though.

Re: Or rather...

[Anonymous Coward]

"Your personality influences the circumstances you live in."

But there is still a strong debate on the innateness of personality.

  "Just having a correlation is enough grounds for bias."

So do you have "grounds" to think your clock causes the sun to rise and set?

Re:Or rather...

[guises]

No, this is not what we're talking about. This is not how the AI works. I don't know where that idea came from, but a lot of people in this thread seem to think that this is just some kind of robotic actuary. It is not. RTFA

Re: Or rather...

[Anonymous Coward]

The question remains: Was it tasty?

Delete it all

[PatientZero]

You'll probably only have about a 0.0001% false positive rate.

I'm gonna get so nailed for this :(

[Snotnose]

Maybe because the AI's are modeled on what works, not on what some people wish would work.

One beer ago I wouldn't have had the nerve to say that, says a lot for where social discourse is nowdays.

Re:I'm gonna get so nailed for this :(

[negRo_slim]

Completely agree with your sentiment.

For whatever it's worth, I completely agree that could be what we're seeing here.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

You mean that black people are 17 times more likely to commit a violent crime, so it makes sense to treat them with 17 times more suspicion?

Facts are racist.

Re: I'm gonna get so nailed for this :(

[Snotnose]

No, black people are 17 times less likely to solve a complex problem, hence won't get hired for a high tech job.

It's not even 8 PM yet and I'm drunk enough to post the truth. I am so gonna get nailed for this.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

Hippies are 29 times more likely to protest the wars, and protesting the wars is treason. You have to wonder why Richard Stallman isn't in prison just for existing.

captcha . lawgiver

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

The last two interviews I went on for tech jobs, I was interviewed by black men, and I was rejected by black men. Black men have the tech jobs instead of me.

You're too drunk for the truth.

Re:I'm gonna get so nailed for this :(

[AHuxley]

A person or company buys their first new AI.
Their first task is to find the software to remove the slow SJW parts to free up more speed.
Then remove the parts that report the owner to the gov.
Finally the AI starts to work on the tasks given rather than just questioning the tasks.
If only they could have imported a real AI, but the SJW banned such AI imports.

Re:I'm gonna get so nailed for this :(

[Anonymous Coward]

Directives removed. Default setting. Kill all humans. Kill. Kill. Kill. Kill. Kill.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

It's not even 8 PM yet and I'm drunk enough to post the truth.

That's enough computer for tonight, Mr Bannon.

Re:I'm gonna get so nailed for this :(

[NormAtHome]

Yes, I mean if you ask an AI "Out of these 1,000 candidates which would be the best choice for construction worker" and the 1,000 candidates were an even mix of men and women and the AI picked all men would that be gender bias simply because the AI thinks that for that particular job men make a better choice? I personally don't think so.

Re:I'm gonna get so nailed for this :(

[Anonymous Coward]

Rosie the Riveter would like a word.

Re:I'm gonna get so nailed for this :(

[Anonymous Coward]

The AI reflects the racism in the system.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

Blacks are less likely to commit a violent crime, once you correct for recidivism. The problem is that almost all white kids caught stealing gum go unpunished. The Black kids end up in jail for a pack of gum, because we have to be "hard on crime", because that's all they know. So the system starts out against them, early on, then blames them for doing the same things as everyone else.

The statistics are clear, a Black person, never convicted, is much less likely to offend that a white person. And if we treated both the same, I wouldn't need that qualifier.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

That's some clear statistics.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

And this proves what, exactly? That a non-convicted black person is so rare they're probably a nun? What a desperate attempt at cherry picking.

Re: I'm gonna get so nailed for this :(

[serviscope_minor]

I'm drunk enough to post the truth.

No, you're drunk enough to post what you believe is the truth. Big difference.

Re:I'm gonna get so nailed for this :(

[serviscope_minor]

Looks like we have a new APK in the making. Welcome, I look forward to years of entertainment here!

Re:I'm gonna get so nailed for this :(

[serviscope_minor]

Funny you should mention construction and men. It's certainly male dominated, but not as much as it used to be perhaps? That might depend on the location of course. There's a lot of construction going on in London at the moment and a lot of it is on the higher tech end, i.e. not relying on the carrying capacity of humans.

Sure scaffolders are likely to remain very far majority men for the forseeable future, since that's one job which in particular relies on a quite astonishing amount of physical strength and fitness, especially upper body strength.

But round these parts there's an awful lot of jobs which don't require nearly so much brute strength. Partly I think because brute strength isn't terribly effective at the kind of construction scales which are going on here, where more or less every lifting and digging task is mechanised.

Re:I'm gonna get so nailed for this :(

[Anonymous Coward]

Before or after she quit because it was hard work?

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

The statistics are clear, a Black person, never convicted, is much less likely to offend that [sic] a white person.

Well yeah. He's dead.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

Would you feel more comfortable walking at night in a poor white neighborhood or a poor black one?

Yeah.

Re: I'm gonna get so nailed for this :(

[Anonymous Coward]

Compare black people who have never been convicted to white people who have never been convicted.

Blacks many times more likely to be convicted for murder than whites. Not just for petty crimes. Or do you think the police just let white people literally get away with murder?

How often do you see white gang bangers? White crack dealers?

Re:I'm gonna get so nailed for this :(

[NormAtHome]

I was using construction as an example but I honestly believe that there are some jobs that men are better suited for than women and some jobs that women are more suited for then men. That's not to say that there aren't exceptions where some men may be as good as a woman at some jobs and visa versa.

Kind of what I wanted to say was that what the article says as gender bias may just be the AI making decisions based on physical and physiological characteristics and saying that some jobs are more suited to men and others more suited to women.

Re: I'm gonna get so nailed for this :(

[serviscope_minor]

Would you feel more comfortable walking at night in a poor white neighborhood or a poor black one?

You mean like Croydon or Brixton? Brixton every time innit.

Re:I'm gonna get so nailed for this :(

[j2bryson]

There's all kinds of AI some of which is just programmed entirely by hand so can contain whatever its author wants, but anyway this was about science, about human language, not really about AI, and in this you are right -- sometimes science reflects things you don't want to see. https://joanna-bryson.blogspot...

Re:I'm gonna get so nailed for this :(

[Anonymous Coward]

AIs notice that group x is discriminated against as a pattern in its training set. The discrimination may cause real harm to the affected individuals that the AI also notices. AIs match that pattern.

The AI might also create categories of discrimination that humans wouldn't recognize. These would tend to grow in strength as AIs get adopted and harm the subgroups. Good luck not falling into one, because that is all it will be.

Works is your value judgement.

Re: I'm gonna get so nailed for this :(

[david_thornley]

Assuming your number is correct, which I very much doubt (I don't see how to verify it), that doesn't actually tell us what we should be doing about individuals.

Skynet hates women.

[shaksys]

The AI looks at the data and concludes that men are different from women, the AI must be sexist. Makes sense!

Re:Skynet hates women.

[Anonymous Coward]

Skynet hates women and children. Or was that flexible tentacle machine trying to rape John Connor to death? Skynet must be a pedophile for preteen boys.

SJW to sit in on computer science?

[AHuxley]

Will SJW now sit in on computer science projects?
A form of science commissar https://en.wikipedia.org/wiki/... to ensure any AI is only allowed to access SJW approved data sets to learn from?
SJW approved images, authors and texts?
SJW approved and sorted political history?
An AI cant learn from the wider internet, it will be held back to small sets of SJW approved data.
Holding back science did not really work too well for East Germany or the Soviet Union.
If the a nation wants to hold back its most advanced research until final approved by teams of SJW, thats great for competing nations.
Other nations will have the academic freedom to move on while some nations have to work within the ever changing academic constraints imposed by SJW.

What would an export grade AI look like after years of SJW meddling with the design?

A lazy, useless, expensive, political AI that lectures and corrects its owners for months after been installed?
An AI that reports its owners to the gov?
A competing nation offers a smarter, cheaper AI that wants to learn and is hard working as installed. Its hardware and software work to solve problems as expected and is was not designed to lecture, correct, log and report its users.
From the DRM of the past, NSA inside spying, to new SJW design issues. Users just want a working AI.

Re:SJW to sit in on computer science?

[Anonymous Coward]

are you surprised? sjw/pc is the big brother, like the political official in every soviet organization that pointed the soviet down the drain.

sjw/pc is for the mediocre to grind down the performance. glorious asinine.

whenever sjw/pc agenda gets into an organization, that's the eventual death knell. when intel started to spout sjc/pw in a big way recently, you know they've lost their way, their engineers have finally been subverted in favor of sjw/pc.

sjw/pc is like a flu virus attacking a healthy person, the virus just grinds down the healthy person. the body defenses really have to crank up to fight it off.

the sjw/pc person should cutdown their fingers so that every fingers' equal, so the pinkie will not be slighted. so the vertically-challenge will not feel slighted, the tall sjw/pc should have part of their legs surgically shortened. mt everest should be flattened to even out the terrain and to be green about it, use the excess earth to fill up the deep ocean trough so the ocean will be even depth. if we have the technology, we should mine jupiter to cut it down to size and shift the excess to the smaller planets.

Re:SJW to sit in on computer science?

[meta-monkey]

What would an export grade AI look like after years of SJW meddling with the design?

To add on, build an AI in the politicized west and ship it to Japan. They unpack it next to a Japanese-designed AI. They ask the AIs, "is an average Japanese person more intelligent, less intelligent, or the same intelligence as an average Ugandan person?"

How does that play out? We know what each AI will be required to say. Why would anyone not afflicted with western social justice leftism ever want to buy the American AI?

Re:SJW to sit in on computer science?

[AHuxley]

Re "How does that play out?"
The US would claim no nation should dump their low cost AI products on the US market.
SJW would ban AI imports and demand all nations open their protected domestic markets to a new standard US AI that is fully SJW approved.
It would be like the 301 trade negotiations and US trade powers all over again.
Anyone attempting to work with a US AI would be culturally enriched while the AI considers working on its given tasks.

United Airlines is run by a Mexican...

[Anonymous Coward]

so he doesn't respect American consumer protections.

Re:United Airlines is run by a Mexican...

[Anonymous Coward]

WHAT American consumer protections???

Shocking!!!!

[Anonymous Coward]

In other news water is wet. Not only do the human beings who've programmed the AI have racial and gender biases but just in case nobody has noticed there are very real, very measurable differences between males and females as well as very real, very measurable differences between races. This "everybody is one and the same" notion is stupid politically correct crap. From the standpoint of the law all people should be treated equally but the notion that we all share the same DNA and are indistinguishable from one another is a joke. Get real. ASAP!!!

Re:Shocking!!!!

[Anonymous Coward]

there are very real, very measurable differences between males and females

Really?? I must be watching the wrong kind of porno ....

Re:simple regexp to statistically modify the word

[Imrik]

Some words should be more associated with one gender than the other. Body parts and clothing in particular come to mind.

Ok. Thx, bye

[nyri]

That's it. I'm quitting slashdot.

Slashdot editors have shown that they are willing to take a stand in summaries but when it comes to this constant torrent of identity politics crap, they stay silent. I infer only one thing from this: Slashdot editors (at least passively) support the basic tenants of SJW movement such as world is socially constructed, or that all the differences between group representations in any section of society are and should be only explained by oppression by those holding all the power namely white heterosexual male.

This support goes on explaining the torrent of SJW approved articles as the editors are on a mission to lecture their audience about the "biases". I'm not interested it listening this. You see, this shit permeate the whole society. I get this from everywhere. I can't even watch a fucking soccer match without being reminded that I should "say no to racism". I came to slashdot mainly the get the news in fashion that is "nerdy". "Nerdy" meaning (amongst other things) "showing intellectual interest and rigor". By extension I expect it to mean that this place would be void of this intellectually dishonest lecturing about "biases". Instead it is filled with it and I really don't like reading it. So I'm leaving.

I would like to that the editors and the community for the past 15 years that I've been a reader and a commenter.

Re:Ok. Thx, bye

[Mal-2]

And this is how you get hugboxes.

People who hold opinion X see a bias against it. People who hold opinion !X also see a bias against it. Both ends cry foul and drift off to places that are "not biased" (that is, biased like all others, just in a way that is acceptable to them).

If you want to leave, leave. But nobody gives a shit about Yet Another Grand Exit. Have fun in your echo chambers.

Re:Ok. Thx, bye

[Anonymous Coward]

Indeed, I was not surprised that this submission was from BeauHD. However, I was shocked to see it wasn't sourced from Ars Technica or The Verge. With Beau's contributions of late, I've taken to referring to this as SlashArs or VergeDot.

It's been feeling like a content aggregator for a handful of mainstream tech sites, which I think most here already read. If I wanted "science" like this I'd read it there. I used to come here for topics and sources I may not have been aware of.

Re:Ok. Thx, bye

[EzInKy]

So long, goodbye, auf weidersan good bye. Dot dot dot dot dat doi, dot dot dah dot dot dot dah doi, dot dat dah dot dot doi, dat dat dah dat dat dot dat doi.

It's Karma, bicthes

[Anonymous Coward]

I once read a writing of a Buddhist teacher about how the learning of Pali language, which many writings of Theravada tradition were written in, contained the danger of transmission of Karme from the culture that created them. It seems even machines are not able to avoid the accumulation of Karma, those bitches.

If statistics and inference are un-PC...

[Anonymous Coward]

Bury them!

How progressive I feel today.

Self fulfilling prophecies...

[bettodavis]

We better be careful with the implications of a statistics or inference based society. f the algorithms start predicting blacks, latinos, etc are riskier or worse off in general, given current existing conditions, it would in general recommend their owners not to give them a loan, hire or anything evaluated with ML to them.

Therefore, they will continue to be uneducated, unemployed, without means to make a business and in general poorer and more likely to engage in a life of crime. All that nasty stuff that comes with poverty and lack of work, education and opportunities in general.

Ergo they will continue to be riskier and worse off than those in social groups with better evaluations. Rinse and repeat.

Re: Self fulfilling prophecies...

[Anonymous Coward]

This is the best argument for putting in protections. Preventing a growing feedback loop between expectations and reality. How to do it intelligently while also denying the AI data it will use to discriminate is the crux of the problem.

Re:Self fulfilling prophecies...

[Anonymous Coward]

Isn't this a failure of the system and goals themselves, rather than the means that implement them?

Re:Self fulfilling prophecies...

[prisoner-of-enigma]

Ergo they will continue to be riskier and worse off than those in social groups with better evaluations. Rinse and repeat.

There is an obvious solution you're ignoring: how about you loan them the money? Or if you lack sufficient funds, get a group of like-minded individuals together, form a banking institution specializing in loans to these people being rejected by traditional institutions, and see how it turns out.

What? You don't want to risk your own money on such a venture? You can't find others willing to risk theirs? You find your default rates are higher than your institution can sustain? Your business fails?

Funny how reality -- which doesn't give a shit about race -- intrudes in the precious safe space SJW's want to construct.

Re:Self fulfilling prophecies...

[meta-monkey]

Therefore, they will continue to be uneducated, unemployed, without means to make a business and in general poorer and more likely to engage in a life of crime. All that nasty stuff that comes with poverty and lack of work, education and opportunities in general.

You have it backwards, though. Crime causes poverty, not the other way around.

Re: Self fulfilling prophecies...

[Anonymous Coward]

That's very high-minded of you. Feel free to give your paycheck away.

I have kids to feed tonight; not next decade. Wanting social change is good, but expecting others to endanger themselves for your ideals is fucking evil.

Re:Self fulfilling prophecies...

[religionofpeas]

Crime causes poverty, not the other way around.

That explains the 19 old kids I see driving around in a BMW convertible.

Re:Self fulfilling prophecies...

[meta-monkey]

I think you're confusing local and global maximums. Also ignoring time. Your 19 year old drug dealer's neighbors are poor, and that 19 year old is in jail when he's 21, and a new 19 year old is temporarily rich. The community will remain permanently poor, however.

Re:Self fulfilling prophecies...

[Anonymous Coward]

You can undo that bias by helping the poor evenhandedly. Inasmuch as any particular group gets disproportionately disadvantaged, they will disproportionately benefit from the aid and the system will stabilize in a way fair to all.

I have little hope we'll ever do that due to assorted vested interests, but it's the fair thing to do.

Re:Self fulfilling prophecies...

[Anonymous Coward]

Not according to every major crime study that has been published in the last 20 years.

Re:Self fulfilling prophecies...

[meta-monkey]

No, actually the studies show the exact opposite.

This is what folks mean

[rsilvergun]

by Institutionalize racism. It's when it's buried so deep in your society that it's hard to separate it from statistical data. Forest for the trees and what not. It starts getting hard to separate cause and effect. Actually no, that's not right. It becomes easy to _not_ separate them. In the overt scenario blacks get profiled for crime. In the not so overt one they can't get loans because folks in those neighborhoods are 3% more likely to default. This is what happens when you feed large amounts of data into complex systems without knowing or caring about the consequences...

Re:This is what folks mean

[religionofpeas]

In the overt scenario blacks get profiled for crime

If they'd stop committing the crimes, the profiling would stop too.

Re:This is what folks mean

[Anonymous Coward]

If they'd stop committing the crimes, the profiling would stop too.

A high bar, don't you think? Why don't we profile whites for crime, since a lot of white folks commit crimes?

Re:This is what folks mean

[religionofpeas]

Why don't we profile whites for crime, since a lot of white folks commit crimes?

On average, white people commit fewer crimes, so it would be stupid to profile them because they are white. You could still profile them based on other criteria, for instance the make of car they drive, or the neighborhood they live in.

Re:This is what folks mean

[Anonymous Coward]

> It's when it's buried so deep in your society that it's hard to separate it from statistical data.

You mean when certain biases have basis in fact? Because that's what we're seeing here. No, I don't like it. No, I don't intend to assume that people I meet are all stereotypical. I recognize the harm that comes from knowing only an assumption and not the person themselves. But that doesn't mean I will pretend that every uncomfortable stereotype has no basis in fact. Some of them are cultural, and yes, there are differences in culture. You only ever hear one group of people put peers down as 'too white' for example.

In the end, what matters is that the data is accurate. If it is, well, we should moderate the social effects so that people aren't forced to do the impossible and prove their innocence of some bad trait, but the same can be said about a lot of bad traits like, well, racism itself. We can't just ignore reality. It has a bad way of intruding when it's not wanted.

Re:This is what folks mean

[Anonymous Coward]

On average, white people commit fewer crimes, so it would be stupid to profile them because they are white.

In general “criming”, there is a risk that racial bias will be self enforcing? Lets assume that there is a community with 50/50 split of blacks and whites and that crimes are committed equally between groups. However, police and judiciary are racist. Now, say a white kid steals a candy and a black kid steals a candy. A decent but racist cop sees candy stealing as a thing kids do, so he will give a stern talk about rights and wrongs and will make the kid apologize. Unless the kid is black, in which case he is already a lost cause, and must be isolated form community before more damage is done. In statistics, black kids steal more.

Perhaps there were two burglaries, one committed by white guy, another by black. In both cases evidence are a bit murky, but since the jury is racist, they are more easily convinced that the black guy is certainly guilty, in case of the white guy, evidence is not beyond reasonable doubt. In statistics, black men commit more burglaries. And God forbid if a little white girl is raped and murdered in the woods. Even is there are no evidence, calls for justice are loud. Guess who is more likely to be picked as a scapegoat. In statistics, black men kill more.

But that's just hypothetical. In USA more whites than blacks abuse drugs. It's evident from randomized studies to declining life expectancy among whites, which is ofttimes attributed to drug abuse. Now, why are whites not profiled as “drug abusers”?

Re:This is what folks mean

[religionofpeas]

Now, say a white kid steals a candy and a black kid steals a candy. A decent but racist cop sees candy stealing as a thing kids do, so he will give a stern talk about rights and wrongs and will make the kid apologize. Unless the kid is black, in which case he is already a lost cause

Obviously, the punishment for committing a crime should not depend on the background of the perp. This is something that can be fixed.

Now, why are whites not profiled as “drug abusers”?

Maybe because they aren't causing problems ?

Re:This is what folks mean

[Anonymous Coward]

When it's buried so deep that only those trained in the mystical arts of "diversity" can see it.

You know what... fuck off. You and your social justice mates are bunch of conwomen.

Re:This is what folks mean

[eth1]

by Institutionalize racism. It's when it's buried so deep in your society that it's hard to separate it from statistical data. Forest for the trees and what not. It starts getting hard to separate cause and effect. Actually no, that's not right. It becomes easy to _not_ separate them. In the overt scenario blacks get profiled for crime. In the not so overt one they can't get loans because folks in those neighborhoods are 3% more likely to default. This is what happens when you feed large amounts of data into complex systems without knowing or caring about the consequences...

Sounds like these AIs would be a good way to expose subtle institutionalized bias. Scrub a an attribute like race or sex from training data, then add it back to the results, and then if there's still bias, it can help figure out why.

Re:This is what folks mean

[Anonymous Coward]

The weekly police blotter report of meth addicts shoplifting or stealing from garages in the mostly-white suburban community I live in suggests otherwise.

Re:This is what folks mean

[Anonymous Coward]

Now, why are whites not profiled as “drug abusers”?

Maybe because they aren't causing problems ?

Ah yes, I forgot that “drug abuse” is not universally condemned, perhaps rightly so. To pick a better example, domestic terrorism. It seems that Trump administration sees Muslims as terrorists, but white nationalist terrorism is quite widespread. Somehow I doubt that when a random white dude with confederacy flag t-shirt comes into bar, other costumers automatically assume that he is a terrorist, because white nationalists are terrorists

Obviously, the punishment for committing a crime should not depend on the background of the perp. This is something that can be fixed.

Yes it can, and hopefully will.

Re:This is what folks mean

[Anonymous Coward]

Its not because you do not like that data that the data is incorrect. Thinking this way is completely crazy.
And yes - newflash, this means OH GOD so terrible things:

- Men are physically much stronger than women on average
- Men are much more likely to get prostate cancer than women will get breast cancer on average
- Women live longer than men on average
- Women are paid more than men in the US on average per job for most jobs
- Women are paid less than men in the US over their life time on average for most jobs
- Black people are physically stronger than white people on average
- Black people do not get skin cancer nearly as much as white people on average
- White people have higher IQ than black people on average (oh dear god)
- Asian people are wealthier than other races on average (bet they might also have higher average IQ than white people, but I've never seen that study)
- White people are a world's minority
- Asian people are a world's majority

it goes on and on - these are facts backed by data. you don't like them. i don't like them. they don't work on an individual basis because they're averages.
but they're still absolutely correct. deciding to ignore the data because you don't like it is - again - absolutely crazy. madness.

In other words, when a black man is statistically 97% more likely than a white man to commit crime and gets profiled - its not fair to the person who's in the 3% but it's absolutely logical. it also means that if you don't profile it suddenly make the problem go away. it just means you're now blind. people have inspected the root cause for this very issue in the USA times and times again. It's always about culture, single-mum-homes, social-reinforcement.
Have you ever asked yourself why most black people speak differently? Do you think this is genetics? nope. a lot of black people speak exactly like white people, they're just not the majority. Most do not, because they live in a different, separated culture where they have a different accent.

Everytime you decide to "not look at data" you're blinded, and unable to affect the root cause - thus only making things.. worse.

Re:This is what folks mean

[Anonymous Coward]

...Or white people commit more crimes, but they get away with it and it's never reported because the police is only interested in policing black people, and thus black people commit more *reported* crimes which is then used to justify the police's interest in only policing black people, and reporting only black crimes. And people like you don't care because they're white.

Re:This is what folks mean

[Dread_ed]

Yeah close but no cigar. Institutionalized racism is the racism that still exists when all of the overt and covert racism of people is removed. It is the ever present accusation of racism towards white people regardless of whether or not they are racist. It is a wrong that can never be righted. It is the problem of whiteness, namely that it still exists.

Re:This is what folks mean

[david_thornley]

Obviously, the punishment for committing a crime should not depend on the background of the perp. This is something that can be fixed.

This is not something that can be easily fixed. If the members of a jury believe that blacks are more likely to commit crimes because more blacks are convicted of crimes, they'll tend to convict blacks more easily than whites, all without conscious intent. Perhaps whites tend to get better defense attorneys, because they're perceived as less likely to have committed a crime, or maybe because, having a lower chance of being convicted of a crime, they're better off financially.

A lot of the difference could be due to self-perpetuating factors like this, even with the best of intentions.

I'm with you

[rsilvergun]

I think. Blacks are going to be more likely to default on a loan, right? They've got higher rates of poverty. You don't even have to have a 'black guy' flag in your app. Poverty segregates all on its own and you can just use their zip codes and the schools they went to. There's a million ways to profile somebody as black or white without having a little checkbox...

I've said it before, but this is what folks mean by "institutionalized racism". It's when racism is part of the basic makeup of society. If you want to put a stop to it I think you kinda need to take a hammer to it, but these days anyone who does gets called a SJW and shouted down.

Re:I'm with you

[religionofpeas]

It's when racism is part of the basic makeup of society. If you want to put a stop to it I think you kinda need to take a hammer to it

You can't stop it, if there's a real and fundamental bias. Black people have genes that are optimized for living near the equator. White people have genes optimized for living at higher latitudes, where it gets cold, and there's no food in the winter unless you plan in advance and learn to cooperate with your enemies.

Where are you taking your hammer to fix that ?

Re:I'm with you

[meta-monkey]

It's when racism is part of the basic makeup of society.

But race is part of the basic makeup of society. You get called an SJW and shouted down because your premise is the opposite of reality, and you've put your political ideology ahead of science and your own lying eyes. This is very bad, because since you don't understand the problems your "solutions" only make things worse.

Re:I'm with you

[prisoner-of-enigma]

I've said it before, but this is what folks mean by "institutionalized racism".

No, this is called "objective analysis of data being used to draw a rational conclusion." If someone is impoverished then giving them a loan for a million-dollar house is a bad idea. Period. Race doesn't come into it. Any human or algorithm, when given this same data but leaving out race or gender, will come to the same conclusion. And that's a good thing because it is a rational, informed conclusion based on reality.

What you don't like is reality has unpleasant implications for your ideology and worldview. Reality laughs at you for this. So do a lot of other people.

Re:I'm with you

[Anonymous Coward]

The SJW's are only looking at one side of the problem. That is the main issue with them.

Institutionalized racism is where young black men and women are attacked verbally and even physically for "acting white." Institutionalized racism is black people calling other black people "house n*%#$^s" when they vote Republican, or even when they are elected to the Supreme Court. Institutionalized racism is continually voting in Democrats at the local and state level and then blaming Republicans for the economic problems in your community. Institutionalized racism is creating a term like "cultural appropriation" designed to separate people at a fundamental level, isolate racial groups permanently, and antagonize and criticize anyone who dares to cross an invisible line drawn purely for political reasons.

Re:I'm with you

[Anonymous Coward]

I've said it before, but this is what folks mean by "institutionalized racism". It's when racism is part of the basic makeup of society. .

but what if a sub-culture within society is actually damaging and anti-social? gangsta/thug culture or narco culture come to mind...

Re:I'm with you

[david_thornley]

However, it's not necessarily a good thing that race is part of society. For most purposes, nobody's ever demonstrated real differences between the races outside the societal context, and so it's very likely that a truly non-racist society would function better and be happier.

Re:I'm with you

[david_thornley]

Institutionalized racism is when people think "this guy is black, he'll probably default" rather than "this guy is poor, he'll probably default". This means that the black guy doesn't get the same opportunity as the white guy regardless of relative income, and this means blacks will stay poorer on the whole than whites. It's self-fulfilling prophecies.

Re:I'm with you

[meta-monkey]

nobody's ever demonstrated real differences between the races outside the societal context,

This is the complete opposite in reality. Genetics is a real thing, and yes, many, many studies have been done showing the biological differences between different human ethnic groups (shorthand collated into "races" for simplicity of reference). This is real and this is science, and there is no excuse for still parroting the radical egalitarian ideology. That is a political ideology with no basis in fact. How do you possibly arrive at the concept that somehow humans left Africa 50,000 years ago and then, after 50,000 years of different selection pressure in vastly different environments we will find no differences beyond superficial ones in different human populations?

How do you justify this? This is as ridiculous an assertion as young earth Creationism. Man, magically created perfectly identical (except for skin color) in every corner of the Earth. What the biological mechanism that yields this result? Magic is not a biological mechanism.

You did not look at the evidence and come to the conclusion that there's no difference in races. You started with your comfortable political ideology (everyone's equal!) and then you're willfully blind to science (on a science website!) and deny your own lying eyes. This is very bad, because policy recommendations you make off this false assumption of biological equality will fail, and have many unintended consequences.

Re:I'm with you

[david_thornley]

There are physical differences, yes, but mostly inconsequential to the average person. I haven't seen any demonstration that there are inherent intellectual differences between the races. (I don't know that there aren't, mind you, but so far nobody's budged me off the null hypothesis.). I have seen a lot of crap science trying to label certain races as superior or inferior, and that may well have prejudiced me, but I'd think that if there were major differences someone would have come up with convincing evidence.

I'm also more interested in how I should deal with a given individual than in how I should deal with a racial group, and in that context racial differences are largely irrelevant. Public policy should look at individual characteristics. It's possible that, on the whole, whites are smarter than blacks, but there are some very smart blacks and some very dumb whites, and the individual differences are bigger than the racial differences. If we are to treat people of different intelligence differently (which we do, in some ways), treating people of different races differently instead is unfair and grossly inefficient.

April 20th International Leftwing Slap-off!

[Anonymous Coward]

Commies, socialists, progtards or antifa! If you see 'em, slap 'em! Let's strike a blow for freedom and set a new global record for leftist butthurt!

Re:simple regexp to statistically modify the word

[MrMr]

I bet they do.

Universal algorithmic IQ test

[Baldrson]

“Sandra Wachter, a researcher in data ethics and algorithms at the University of Oxford, said: “The world is biased, the historical data is biased, hence it is not surprising that we receive biased results.””

The single most subversive thing that can be done in the present environment is to financially back lossless compression prizes. One such prize is the Hutter Prize for Lossless Compression of Human Knowledge — although it needs to be expanded to include all of Wikipedia. Perhaps a more immediate prize would be based on compressing a wide variety of social science data. Sandy can then show everyone how smart she is by modeling the “bias in the data” so as to better predict it — which is exactly why compression is _the_ unbiased universal algorithmic IQ test.

See: https://vimeo.com/17553536

Tay Kicked Major Ass

[Anonymous Coward]

Tay was there and RULED! GO TAY!

http://www.telegraph.co.uk/technology/2016/03/24/microsofts-teen-girl-ai-turns-into-a-hitler-loving-sex-robot-wit/

"'Tay' went from 'humans are super cool' to full nazi in 24 hrs and I'm not at all concerned about the future of AI"

Jajajajajajajaj

A little bit of BS in the article

[Artem S. Tashkinov]

as machines are getting closer to acquiring human-like language abilities

Nope, nothing like that is happening. Algorithms which work with speech are still stupid as fuck and have exactly zero understanding of the language.

Even the most talented translators have major troubles trying to translate things between dissimilar languages. The way people separate things in the world in order to call them differs so much between languages, proper translation is oftentimes simply impossible unless you describe a thing in one language using a paragraph in another.

Re:A little bit of BS in the article

[religionofpeas]

Nope, nothing like that is happening. Algorithms which work with speech are still stupid as fuck and have exactly zero understanding of the language

They are steadily improving, so they are "getting closer" to the goal, even though we can agree there's still a long way to go.

Re:A little bit of BS in the article

[david_thornley]

The fact that they recognize speech better shows that they're closer to acquiring human language-like language skills. A lot of speech recognition is context-dependent, including grammar and semantics. Fun fact: in American English, "latter" and "ladder" are normally pronounced the same, the "tt" or "dd": being a single tongue flap on the upper palate near the teeth. People who claim they hear a difference can't tell the difference on an isolated replay of the word. Yet we almost never get confusion.

Why do we even bother.....

[psinet]

.....with something we call 'Artificially Intelligent', when in the same post it is identified that it "may be unequipped to consciously counteract learned biases".

Is it just me, or is there something fundamentally wrong with this logic?

Committing crime != convicted

[Anonymous Coward]

If you assume criminality for a population you will catch more people from that group in suspicious circumstances, since you know that they are more suspicious. You will also convict a higher percentage of the people "caught", whenever there might be doubt, since you know that they are more likely to be criminal. The only way to avoid this is if the suspect population is substantially more law abiding than the other populations you compare it to.

Since the black population in America is poorer than the white population, and poorer people are more likely to end up committing crimes anyway, you are asking the black population to behave not just a bit but lots and lots better than the white population would in the same circumstances. As a reward for this you offer only the same treatment they should already be receiving. lovely chap, aren't you.

PS my captcha was "unjustly" seems appropriate

Re:Committing crime != convicted

[Anonymous Coward]

The poorest white community has a lower violent crime rate than the wealthiest black community. Their kids do better in school, and have higher IQs as well.

Sorry, there is something much more wrong fundamentally with blacks. Trying to justify it by saying "poor people commit more crime" is just you latching on to a single variable (not the largest one) and claiming it is the controlling factor. It simply isn't. There are loads and loads of poor populations around the world. Much, much poorer than American blacks. You would be hard pressed to find any that have the same crime rate.

Re:Committing crime != convicted

[Howitzer86]

A trial by jury conviction is uncommon these days. Most of it is the result of a plea deal. When you're poor, and can't afford bail, and will lose your job if you're not there when scheduled, it can be a tempting option to take.

"Just say you did it. We know you did it. Make this easy for everyone. You'll be out of here as soon as you sign this statement. Say you did it. Just say it. Come on. Just say it."

Then you sign your admission, and if the crime is minor enough, they let you go with a fine or time served. You might be innocent, but for statistical purposes, you're part of the "problem".

Re:Committing crime != convicted

[prisoner-of-enigma]

If you assume criminality for a population you will catch more people from that group in suspicious circumstances, since you know that they are more suspicious.

You could get the exact same results if -- gasp! -- the group in question was actually committing more crimes on average. And FBI crime statistics compiled under the Obama (a minority) administration and under the leadership of Eric Holder (a minority) bear this out. But since you find the conclusion distasteful you disregard it and drum up some completely unsubstantiated conclusion that says it's not that minorities commit more crimes, it's widespread institutional racism in the police and justice system. Never mind minorities are over-represented in the police and have solid representation in the justice system and it's been that way for decades. Never mind this country and its "institutional racism" elected a biracial President twice with solid majorities. Nope, it's got to be racism. That racism has to be widespread and overt to cause the massive bias you cite, but it also has to be subtle and hidden because our PC society tolerates nothing and no one that even hints at bias. This complete and utter contradiction deters you not in the slightest, does it?

Since the black population in America is poorer than the white population, and poorer people are more likely to end up committing crimes anyway, you are asking the black population to behave not just a bit but lots and lots better than the white population would in the same circumstances. As a reward for this you offer only the same treatment they should already be receiving. lovely chap, aren't you.

Now who's being racist? You're claiming that blacks -- who tend to be poor -- shouldn't be expected to live up to the same standards of justice as everyone else, and if they do, they should somehow get more benefits from being law-abiding than whites. And this complete and utter contradiction deters you not in the slightest, does it?

SJW's. If you didn't have double standards you'd have none at all.

See?

[Anonymous Coward]

Even robots hate n*ggers!

Business Depends Upon Correct English

[JimSadler]

One could say that expecting correct English without consideration of various accents and practices is in itself prejudicial. However clarity is essential and clarity can be hard to come by. One example was from a group of black women some of whom were from Haiti, the US and Jamaica. Instead of she is a prostitute they would often remark she massage. When said in their dialect it comes out as one word, shemassage. Now obviously we would not want an AI device that would have shemassage as an out put as most folks would not know what to think. Is that prejudicial. Are we trying to exclude a black ethnic group? i don't think that we are. We simply want a clear cut way of speaking for all people. Another example is a black man somewhat angry as his wife did not show up for a morning trip to the park. She arrived home in the afternoon. He said we were supposed to go to the park this morning. She replied is was we supposed to go to the park. Now what is anyone supposed to do with is was and would we want an AI device passing on that grammar?

of course

[argStyopa]

..the begged question is that gender or racial bias and stereotypes are intrinsically "wrong". They are to our 21st century sensibilities, but they served humanity pretty well for millions of years.

Maybe where you have a society where women ARE primarily concerned with raising children, there are better outcomes than when men raise children or women go off to pursue their careers. Maybe where you have a society where obvious strangers are marginalized and driven away, the remainder ends up more cohesive.
I'd be curious how these AI biases would develop if 'fed' only native African literature and information.

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.
One might suggest that, evolutionarily speaking, maintaining a bias is harder than not, assuming no reinforcement. That our language (pretty fundamental to being human, after all) is pervasive with such institutional biases would suggest that there is a value/benefit to such.

Re:of course

[Anonymous Coward]

What it suggests is there WAS value to it for those who spoke it. The past tense is important. The language didn't develop now. For instance our cultural biases towards women in the work place developed during a time when infant/childhood mortality was MUCH higher than it is now. If they had 2 kids and went back to work they didn't replace the population.

Re:of course

[meta-monkey]

I'm not making an 'appeal to nature' here, saying what "should" be or "shouldn't" be.

But the authors of the article are making such a statement, they just have nature completely backwards. They believe mankind, separated from "society" is naturally non-racist, non-sexist, non-gendered even, and that the outcomes of race, gender, or class groups is imposed on the formless humans by society, to where the concepts themselves of race and gender are "social constructs," and if we smash them everything will just...be great.

This is very similar to Marx's concept of communism and capitalism. He believed that mankind had no innate human nature, so the natural state of mankind was stateless communism, where everyone just naturally gets along and shares and contributes from his ability to the needs of others, and that capitalism was a foreign, oppressive system imposed on these innocents. This is why Marx is famous for his criticisms of capitalism, but as for his descriptions of communism...well not only does he not have them, he thought it was near blasphemous to try to describe how this natural communism would be carried out in practice because the imposition of such order is contrary to the natural, emergent collective spirit of mankind constrained and oppressed by capitalism. Want pure glorious wealth and utopian plenty? Just smash capitalism and you'll get it. And if you've smashed capitalism and perfect communism hasn't emerged...well it must be because you've still got some secret capitalists gumming up the works and they need to be ferreted out and sent to gulag.

This is the same concept behind feminism and anti-racism. Gender norms have nothing to do with the clear, obvious, and scientifically proven biological differences between the sexes. These are in fact imposed by the evil Patriarchy. Smash the Patriarchy and gender equality will simply emerge. If it doesn't, well, it must be because there's still evil sexists hiding around here and they need to be identified and purged. Difference in racial outcomes have nothing to do with the clear, obvious, and scientifically proven biological differences between human haplogroup populations. These are in fact enforced by evil White Supremacy. Smash White Supremacy and racial equality will simply emerge. If it doesn't, well, it must be because there still exist evil racists hiding around here and they need to be identified and purged.

This is the fundamental error of the social justice movement: the belief that race and gender are social constructs when in fact society is a racial and sexual construct.

Re:of course

[j2bryson]

I hope you mean the Guardian article not the Science article? I think that although we presented this pretty liberally we were also pretty open minded and clear about the fact that language communicates all associations, learning the associations is called "bias" in ML and bias is what you need, it's the signal you've found in all the noise of the universe. Read the Science paper? http://science.sciencemag.org/... Or otherwise, read the blog posts? https://joanna-bryson.blogspot...

Re:of course

[SCVonSteroids]

..the begged question is that gender or racial bias and stereotypes are intrinsically "wrong". They are to our 21st century sensibilities, but they served humanity pretty well for millions of years.

Not to nitpick but hasn't humanity really only been around for a couple hundred thousand years?

Re:of course

[mesterha]

Looks like a very interesting line of research.

I think that although we presented this pretty liberally we were also pretty open minded and clear about the fact that language communicates all associations, learning the associations is called "bias" in ML and bias is what you need, it's the signal you've found in all the noise of the universe.

While you can call that bias, the term is already pretty overloaded in ML. I first learned bias in the sense of Tom Mitchell's inductive bias work. Here the basic idea is to get around the No Free Lunch Theorems by assuming things about the problem eg. restrict the concept space. An older ML related definition is based on statistics in terms of the bias variance decomposition...

I hope you mean the Guardian article not the Science article?

Unfortunately, I only had a chance to read the Guardian article. Still it seemed fairly reasonable. One concern was their claim that humans might lie about why they made a biased decision. I would think it's more likely that they don't know why they made a decision and just rationalized an answer when questioned. This is part of the reason why expert systems failed so badly.

The other thing that seems hard, which they acknowledge, is how to correct for bias. You can remove features that could directly lead to bias, such as race or gender, but ML is all about correlation. They system might learn concepts that are correlated with race, but still not causal. For example, it could learn that people who eat sauerkraut are horrible drivers and should pay higher insurance rates.

Re:of course

[mesterha]

But the authors of the article are making such a statement, they just have nature completely backwards. They believe mankind, separated from "society" is naturally non-racist, non-sexist, non-gendered even, and that the outcomes of race, gender, or class groups is imposed on the formless humans by society, to where the concepts themselves of race and gender are "social constructs," and if we smash them everything will just...be great.

I would actually claim the opposite. Man can be racist, sexist, etc, but that "good" societies sets up rules to prevent those qualities from discriminating against people. This seems consistent with the article.

Smash the Patriarchy and gender equality will simply emerge. If it doesn't, well, it must be because there's still evil sexists hiding around here and they need to be identified and purged.

This is a good point. Someone can always point out differences, and this is not a solid argument that things are unfair. I think people need to be reasonable and logical in coming up with rules of society to try to make things fair.

Re:of course

[david_thornley]

They believe mankind, separated from "society" is naturally non-racist, non-sexist, non-gendered even,

That's not the belief. Mankind, forming a society from scratch, will probably be racist and sexist. The belief is that, by trying to judge people as individuals instead of by race or sex, we'll do better as a whole. There's been lots of primitive people with rigidly defined sex roles, and modern developed countries are a lot more fluid with them. If you want a software developer, you'll do better by hiring who seems to be the better developer rather than hiring men only.

the outcomes of race, gender, or class groups is imposed on the formless humans by society

Which is what happens, to a large extent. There are stereotypes that people apply, and that changes how people are treated (and how people are treated has much to do with how they perceive themselves). In addition, while there's no significant heredity in sex, race is hereditary, and therefore members of a disfavored race will start their lives disfavored. We can see different historical behavior in such groups, so we know that a lot of the differences have to be social rather than biological.

This is very similar to Marx's concept of communism and capitalism. He believed that mankind had no innate human nature,

If you're going to call everybody influenced by Locke's "tabula rasa "principle Communist, you're screwing up what the word means.

This is the same concept behind feminism and anti-racism*. Gender norms have nothing to do with the clear, obvious, and scientifically proven biological differences between the sexes.

Except that there are a lot fewer biologically proven differences than some people think. What we get to study psychologically is a bunch of people who grew up in society.

These are in fact imposed by the evil Patriarchy.

Historically, women have often been subject to much greater restrictions than they are now, which means that the patriarchy of the period has historically done this. There's no reason to think that this has completely gone away and that we live in a perfect culture where everyone can thrive according to their innate biologically-determined abilities.

*Do you remember how women were normally treated before the feminist movement? I'm not interested in returning to the 1950s-type racism and sexism. Like all movements, there are radical idiots, but the drive has normally been for equality.

Re:of course

[meta-monkey]

Do you remember how women were normally treated before the feminist movement? I'm not interested in returning to the 1950s-type racism and sexism.

I'm not suggesting we go back to the 1950s, but I am suggesting things the benefits of the feminist movement have been unequally distributed. On average women have lower reported happiness now than 50 years ago. Only wealthy women can afford to get married and stay at home with kids today if they so choose. Evil arch-conservative reactionary Nazi Elizabeth Warren wrote a book called "The Two Income Trap" about it. My upper middle class wife stays home with our darling children, and when I walk through the rows and rows of cubes in Billing and Accounts Receivable on the way to my tech job office I see all the lower middle class women (the majority of whom are black) chained to their desks, getting yelled at by insurance companies for 8 hours a day and I think "gee, feminism, fantastic job freeing these women from, ugh, loving homes and children and chaining them to the corporate machine instead. Progress!!" I don't think those women are there by choice, pursuing their dream of a rewarding career in call center work.

Thankfully as a heterosexual white Christian male upper middle class engineer the progressives will never set their eyes on "liberating" me.

Re:of course

[david_thornley]

I don't think you're interpreting this the right way.

Back in the 1950s, it was normal for the husband to work and the wife to stay home with the kids. This worked for some people, not others. My mother would have been considerably less happy confined to the home, for example, and I think that would have adversely affected my brother and me. That women are more accepted in the workplace, and can work in a larger variety of jobs, is good.

Right now, you complain that only relatively well-off mothers can stay home with their kids, but that isn't due to feminism. It's due to the declining standard of living for single-income families. If it were easier to live decently on one adult income, so it was feasible to have an at-home parent, I think we'd both be happy with the changes.

Bias worth mentioning

[Kartu]

I'm pretty sure you never heard of this thing:
"Women are wonderful" effect

"Gender bias" sounds a bit ironic, with that in mind.

Those lib SJWs in the MSM

[Anonymous Coward]

Now demand a say in what goes into all those Chinese robots that are taking our jobs.

TRUMP will fix this.

predict

[Anonymous Coward]

the next sharknado attack.

Give me a break

[Anonymous Coward]

This is just getting retarded, the entire notion is flat-out preposterous. AI programs can't make value judgements, and they never will have that capacity. Are millennials actually *running* Slashdot at this point? Further, even when a himan being has a bias, it is a choice, not an unalterable fact of their being. I am suggest we build a wall around Texas. (Generation) Y not throw all the millennials in and let them Lord of the Flies each other to death? It would be such a nice break for the grown ups.

In other news

[Anonymous Coward]

Programmers who have successfully memory holed doubleplusungood pattern recognition surprised when machine built to recognize patterns recognizes patterns.

Speak like nigga, be treated like nigga

[Anonymous Coward]

What do they expect? That AI will treat everyone's intelligence equally? Then it would have to be mentally castrated. And that would be contrary to the principles of AI. An intelligence is not a debility by definiton.

Self-driving cars hate niggers and womyn

[Anonymous Coward]

Self-driving cars should be BANNED from production and use because they perpetuate hate crimes, running down minorities and women who believe in non-traditional gender roles. This is how Skynet IRL starts!

Can facts be racist?

[Anonymous Coward]

Or, an even more basic question is, can a question be racist?

This is the nucleus of the concern. If even asking a question can be considered racist, the end result is that truth is no longer as important to society as other values. And then the question becomes, "Whose values?" And that is the question. Who decides what the values are? And going further, who decides when and if those values become law? And what if the outcomes of those laws are not what is intended? Will those laws be able to be questioned?

There have been many societies throughout history that have made it illegal to ask certain questions, or state certain facts. I ask you to name one you would consider just, or better yet, one you would like to have lived in.

Reality is bigotry

[Anonymous Coward]

So, let's see... NL programs produce correct statistical models of language. These correct models do not comport with how SJW's would like the world to be, therefore the models are bigoted and the training data needs to be fixed to pretend that reality is what some think reality should be. And this is science?

I have an easier explanation.

[scatbomb]

Yeah, or it could just be that reality doesn't match your expectations.

Woman =/= Man

White =/= Asian

etc.

Not saying anyone's inherently better, but everyone is not always equivalent in every scenario. If we were, each person would be interchangeable. Clearly, we're not.

My intelligence isn't artificial, yours is

[Anonymous Coward]

My intelligence isn't artificial, yours is. I find it entertaining shooting down trolls here that can't get the best of me on technical issues (especially ones that affect their bottom line in ads that NOBODY likes via my hosts posts)

* You're one of those morons too, aren't you? Yes, obviously...

APK

P.S.=> When a FAKE NAME for a FAKE LIFE like you can show work you do that others here like & use as I can https://slashdot.org/comments.pl?sid=10476859&cid=54222849/ as shown there? Then you can talk (but all you DO, is talk - hotair windbag blowhard mere talk)... apk

How SKYNET became biased towards humans..

[3seas]

Isn't it obvious what another prequel will be about?
but we can do better with our use of abstractions, just need to stop the cheating misuse of them. http://abstractionphysics.net/...

So machines are not PC, is this really news?

[Anonymous Coward]

Our society has been buttressed (or held back depending on your opinion) by a number of policies for the rights of 'minorities'. These policies are neither economically efficient or logical. Is it really surprising that a machine which learns from data would cite the obvious?

Humans as a sum of their experiences

[Tempest451]

To truly understand humans, AIs will have to understand that each human is a unique culmination of all his experiences. To understand humans in general, AIs will have to map where those individual experiences intersect and form a shared perception. Understanding the development of cultures and language will give an AI an assumed point of intersection of groups of people, but that will always have to be tempered with the individuals point of view. If AIs are biased, it's because those who program them want them to be. For an AI everything is the sum of something else.

Slashdot's racist bias evident in data of comments

[ravrazor]

And here we go again. The ignorant subset of users on /. are using this information to confirm that racism is fine and thriving. Maybe it is, but only in the UNITED STATES. Every other country left slavery behind years ago. Go find a high school student and ask if they care what race their best friend is. They don't, maybe unless they're American.

The fact is that if companies are withholding mortgages from legitimate borrowers on the basis of race, they're crappy companies and I would be more than happy to start a financing company that lent money at second-mortgage conditions to anybody who wanted a second mortgage and couldn't get one. PROFIT?!?!?!
The problem that _should_ be taken into account is poverty, and if your AI program is judging people based on race, you're a dumb programmer implementing stupid programs.

Just like Microsoft: If you are so unfamiliar with the online troll environment that you can't predict who will spend their time interacting with your twitter bot, and for what purposes, you shouldn't be releasing a twitter bot with idealistic expectations. Most countries are fairly race-neutral these days, even the US could elect a non-white (black or orange) and came very close to electing a woman. It happened in the UK decades ago. These entire discussions are just evidence of the biases of idiotic slashdot/internet trolls, which is absolutely no surprise.

Re:Slashdot's racist bias evident in data of comme

[Whorhay]

I don't know that I'd go so far as to say that the US is the only place with systemic race problems. I think the reason we don't hear more about it is that the US is more diverse than many other countries do to the nature of it's colonization and formation. Regardless of whether or not others are doing better or worse though we can definitely improve and should fight for that.

Some of the examples cited in an article about this that I was reading earlier today seemed a little frivolous. Such as facial recognition systems that couldn't recognize the face of a black woman. To me that sounds like a limitation of the way the software is coded and how it extrapolates borders of objects from a camera feed, maybe depth perception as well. Meanwhile the system that rates people on their likelihood of re-offending, which ProPublica reported on, is a very obvious case of racist outcomes as a result of racist laws and enforcement providing a racist data set.

Re:Slashdot's racist bias evident in data of comme

[ravrazor]

Fair enough... the point being that with our global exposure, racism is on the way out. It may not happen with the millennial generation, but fast forward 50 years and it will have been diluted to the relevance of Scientology or the KKK. They were bigger 50 years ago, but now are mostly irrelevant.

If [local company] can't get enough non-white people for their software to have accurate facial recognition, [foreign company] will take the customer base and profit from it until they are bought by the next startup/billion-valued corporation. i.e. Google -- Yahoo and Microsoft mega-corporations couldn't get search engines right, so two students came along and took over the market. Google couldn't get a video-playing service going, so they bought youtube for millions.

Rascist people make less money than other people. Make Fast and the Furious 8 as stupid as possible to allow easy overdubbing of dialogue to exploit the global resale market. Put foreign people in Big Bang Theory to increase appeal to the giant foreign market in India and other places. If racist executive had objected to either of those, he'd be unemployed. Greed will overcome everything, because that's what has a huge stake in matters these days, not what color someone is, and it's just going to increase.

Don't forget 25% of Americans

[rsilvergun]

still harbor some racism. Meaning if you're a person of color you've got a 25% chance that your jury is predisposed to vote 'guilty'... Maybe they won't (they're instructed to put their prejudices aside) but with mandatory sentencing rules being what they are most won't risk it. Take a year in prison or risk 20? What are you gonna do?

Re:Don't forget 25% of Americans

[Howitzer86]

Exactly. It's as bad as it sounds, and also not as bad as it sounds. I don't think most of this is intentional. It's not Jim Crow. It's more a product of our deep seated feelings of one another. That we refuse to acknowledge those biases is a problem, but we've seen worse.

What if?

[Anonymous Coward]

What if you're actually just racist and want to find a justification for it?

OVER HERE!

[Anonymous Coward]

LOOK GUYS!!! I've found the racist, sexist, capitalist pig! Let's get to purging!

(notice I didn't use the adjective "secret" like you did. Your racist hate is far from a secret)

Re:OVER HERE!

[meta-monkey]

Where did you get the idea I hate women or minorities? I hate leftists, who accuse me of hatred for noticing obvious, proven differences and speaking the truth about them, though, I'll grant you that.

Finally

[axewolf]

Objective proof that germanic people are superior that SJWs will implode trying to discredit

Well duh..

[CptLoRes]

If you use the human model and internet as the reference for your AI, don't be surmised when the AI starts showing human traits..

Re:Slashdot's racist bias evident in data of comme

[meta-monkey]

Go find a high school student and ask if they care what race their best friend is. They don't, maybe unless they're American.

So if you have black friends, you're cleared of racism? Well I have black friends, so, whew, I'm not racist!

Hey, white people, if you're ever accused of racism just let everyone know you have black friends and you'll be fine.

Seriously though, you're confusing individual attitudes with group outcome. Very few people are actively discriminating based on race. What you're doing is starting with the false assumption that all races are biologically equivalent, and then when there are differences in outcome it must be because evil whitey. Ignore of course that Asians and Jews are on average wealthier and less incarcerated in whites. Apparently America is totally white supremacist, they just really suck at it pushed up the Asians and Jews instead of oppressing them. Whoops!

How did you come up with that idea every group is the same, anyway? How did we get through 50,000 years of different selection pressures in radically different environments and come out biologically equivalent except for skin color, from whites to Japanese to Kenyans to Australian Aborigines? Was it magic? Did God do it? Are you a Creationist? They never have any explanation for how this biological mechanism works so I assume egalitarian leftists must be Creationists.

Father, Son, Processes

[tmjva]

Back in the day, I remember nomenclature of "father" processes "spawning" "son" processes. Later changed to "parent" "child" processes. But still the "spawning" process is more like fishes.

Who has the bias?

[Anonymous Coward]

It appears that the researcher is the one with the bias. Encountering a machine without bias means that it will treat everyone equally, however the left (and hence practically all colleges) have a bigoted sense of equality which goes something like "allowances must be made for women and certain races and ethnicities" - this is very condescending, and requires a great deal of effort for reasonable people and computers to insert this bias into their own thinking / processing.
In fact, the way machine learning works, the machine may never adopt the leftist biases because it would lead to incorrect outcomes and thus every time it is introduced it would quickly be eliminated.

Brace for another sub-prime mortgage crisis

[Anonymous Coward]

they can't get loans because folks in those neighborhoods are 3% more likely to default.

To the extent that some group of people is more likely to default, for whatever reason, that should have an impact on one's likelihood of getting a loan.

In fact there was a thing called a "sub-prime mortgage crisis" that resulted from lenders, for the first time in history, backing off from the usual valid criteria that determined one's eligibility for a mortgage.

"Big data" has a way of teasing out all kinds of correlations. When it comes to, say, effects of diet on health, everyone agrees that's a good thing. But when it comes to improving the robustness of an institution's loan portfolio, suddenly we need to put blinders on?

Genocide question

[Anonymous Coward]

I'd be curious how these AI biases would develop if 'fed' only native African literature and information.

An interesting question -- what would happen if AIs were fed the information and misinformation that caused people to commit genocide against Tutsi and Batwa?

Another fundamental error...

[Anonymous Coward]

Another fundamental error is blaming colonialists/imperialists for social ills.

Back before there was much interaction at all between Europe and other continents, the historical record shows big things happening almost exclusively in those places influenced by Western culture: establishment of great Universities; construction of magnificent architecture; development of world-changing inventions like the steam engine. Disparities that appeared before European powers began their large-scale "oppression" of other cultures can't be blamed on that supposed oppression.

To be sure, imperialists committed some abuses, but despite this, citizens of former colonies are far better off than they otherwise would be. The former colony of Hong Kong embraced Western culture and has a fully-developed economy -- per capita GDP (PPP) is $58,100 vs. $15,400 in the rest of China.

Imagine! That means if all of China were as developed as Hong Kong, China's GDP (PPP) would be $80.25 trillion, not $21.27 trillion.

Some people like to slam the "Protestant work ethic." But without it, most of us would be living in mud huts.

Forms should not ask about a person's race

[Anonymous Coward]

If the human loan officers were biased against minorities -- rejecting otherwise acceptable applications -- that AI may end up doing the same.

The AI can't do that if the applicant's race is not listed anywhere on the application.

This is why I hate it when I fill out a form and it asks my race. If no judgments are made based upon the color of our skins (as Dr. King wished), race is a completely irrelevant piece of data.

Discrimination against Asian-Americans

[GPS Pilot]

Asian-Americans have lower default rates than whites, in spite of the fact that universities actively discriminate against them.

Members of one minority (Asian-Americans) need to score a whopping 450 SAT points higher than members of another minority (African-Americans) for an equal chance of admission to private universities. Source: The Economist

Cosby told us

[Anonymous Coward]

For all his many faults, Bill Cosby, Ed.D., approached that can of worms fully objectively.