Slashdot Mirror
Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women (reuters.com)
Jeffrey Dastin, reporting for Reuters: Amazon's machine-learning specialists uncovered a big problem: their new recruiting engine did not like women. The team had been building computer programs since 2014 to review job applicants' resumes with the aim of mechanizing the search for top talent, five people familiar with the effort told Reuters. Automation has been key to Amazon's e-commerce dominance, be it inside warehouses or driving pricing decisions. The company's experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars -- much like shoppers rate products on Amazon, some of the people said. "Everyone wanted this holy grail," one of the people said. "They literally wanted it to be an engine where I'm going to give you 100 resumes, it will spit out the top five, and we'll hire those." But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way. That is because Amazon's computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.
[...] Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said. The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity.
[...] Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said. The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity.
Train algorithm with data in hand, algorithm's output mirrors data provided. They can't possibly be shocked by this, can they?
Which has more power: the hammer, or the anvil?
When government reviews your hiring they expect you to show that your diversity level is consistent with the normal spread of minority groups ( some consideration of candidate pool MAY be given.)
In other words, if your only criteria is hiring whomever best for the job, you will likely be operating illegally and subject to fines and lawsuits. This is the product of laws that are designed to create social engineering based restrictions based on someones religious idea that any measurable discrepancy in minority placement must be corrected.
âoeTolerance applies only to persons, but never to truth. Intolerance applies only to truth, but never to persons.
Amazon trained their AI using the dataset that reflected their business practices as they currently are (flaws and all) but what they wanted was a data set for the practices they wanted to become (i.e. the ideal).
Finding a training dataset that reflects the ideal is going to be extremely difficult, particularly in an area where that ideal is so poorly defined.
Bias is a non-factual prejudice against someone. That is why it is considered unfair. If the facts are that 80% of the population of people who do the work you want are named "Dave", then it is not a sign of a moral failing if your AI exhibits a strong preference for another Dave.
"We receive as friendly that which agrees with, we resist with dislike that which opposes us" - Faraday
Garbage in, garbage out.
If the training data has bias, then the AI will learn to have that bias.
The trick is developing training data that doesn't reflect the biases of the humans that performed the task in the past.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
If the results are biased, the data is biased and the process is biased, maybe the bias is normal?
Purge any submission to the system of a gender identifier... women's or men's anything... remove names in case that is factored... literally provide nothing in the submission that would definitively define a gender.
Then see what it does.
My experience with these systems is that they don't actually factor gender but that the end result of is that there is a gender imbalance.
However, if there is an imbalance and the system was given no indication as to gender then there is no gender bias.
You can't cite persecution or preference if the system can't even know. And generally these fairly common and consistent imbalances are made without reference to gender itself.
Generally it is factoring on other criteria that give the same result but which are not gender. Work experience is a big one... breadth of skill set is another.
And if you took the total population and look at which portion of the population had that work experience and breadth of knowledge, you'd find it more closely matched the hiring patterns of these systems. Which means it isn't factoring on gender.
Now... this is assumption to some extent on my part. I've audited these systems in the past and what I am describing above is the pattern I've seen.
As to what the Amazon system was doing... I'd have to audit it.
What I'd probably try is a word replacement/purge of all terms that would signify gender or I'd just change a bunch of rejected female resumes to say they were male and see if they got accepted and vice versa.
If the system actually changed its decision based on gender then that's a smoking gun that it is doing things on the basis of gender.
But I'd find that very surprising.
Machine learning is unpredictable so I'm hardly going to claim to know what the damned thing was doing. For that reason I wouldn't actually use machine learning in this application. I'd use a very clear rules based system where everything it was doing was known to the programmers.
Those systems are completely fine for this sort of work and you can very easily audit the code for them.
The best way to deal with this is to first be gender blind. You literally do not factor for gender at all.
That will give you an imbalance probably... you can make as many diversity hires as you need to after that. But your core hiring pool should be merit based unless you want to go out of business.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
Well, there is a huge imbalance between genders in IT, due to different reasons (which I don't want to speculate about, James Damore tried and we all know what happened). So the system must have a bit of positive bias towards women to correct for this - to help restoring balance. However, the cold heartless AI can't factor in this ... what data you feed in defines what model you get. So, is the data biased?
The claim that "the industry is dominated by men and therefore we couldn't train this in a gender-neutral way" is totally bogus from a machine-learning perspective. All that is needed to eliminate a bias arising from dataset imbalance is to balance the dataset.
More likely they realised that when using dispassionate criteria for optimal hiring, it would become very likely they'd not get the desired "Women > Men" politically correct outcome for all sorts of statistically valid reasons, and figured such optimal hiring was not worth its salt against all the money lost from lawsuits and bad PR in a time of a politically tense climate favouring women.
I completely agree with their choice, and would do the same. No need to feed oil to the fire
Yes.
Both this ridiculous garbage reporting and the apoplectic shitshow from ideologues in the press over James Damore's memo are not just the usual bland claims of sexism.
There are long known and well researched gender differences in interest preference going all the way back to infants - long before any possible way to for the results to be explained by 'societal sexism' or other such nonsense.
Feminist dogma is 100 percent counterfactual this basic and well researched science.
Hence why the over the top attacks on anyone and anything that brings to light these fundamental differences in the abilities of men vs women in technological jobs.
The reason there is such a huge disparity in male hires in tech companies is a direct result of those well established gender differences. The candidates being selected are at the very, very top end of the bell curve in both intelligence(where men have a significant advantage) and a lifetime of interest in and drive compared to female applicants in general.
Of course the usual 'argument' and response anyone pointing these basic facts out is screetching that the claim is women aren't as capable as men.
Any individual woman can be just as capable as a man in tech.
However, that is not true at the population level where men will significantly outpace women in the number of highly qualified candidates.
Never. Any discussion of gender imbalance is always started by the people who don't have any followup evidence that their inclusion will improve the sector.
A system cannot be changed, and has no responsibility to change, unless the opposing system seeking to substitute it proves that it can create out of scratch what the original system in-place did from zero, and improve upon it, making the ideas of social justice evangelists nothing but empty words without any substance.
The problem is that the social justice evangelists always latch onto existing business instead of proving their competence by carving out their own segment in the industry independently from scratch as the "patriarchy" did on day 1 before any human was involved in the field, thus making their entire cause suspect.
It's this shit that i disdain the most: Milking existing businesses and calling for unjust redistribution of existing resources by venue of empty words and philosophizing instead of creating your own and thus adding to existing wealth and showing qualifications that you can mingle with the rest (therefore leading to inclusion). East Asians proved themselves in this manner by involving and showing skills rather than being idiotic pussy-hat wearing whiny children begging for daddy for allowance.
They are being fixed. There are programs by nurse's unions on hiring more male nurses, and there are programs for teachers as well in increasing the number of male teachers teaching elementary school.
Don't assume that because you don't know about it it's not a big deal. There are programs for increasing the proportion of females in trades (construction, welding, etc) run by various trades organizations, and plenty of pilot groups also aimed at increasing the gross under representation of women in the cockpit. (Funny enough, the same complaints show up as well when special women-only events aimed at getting girls interested in aviation.). Oh yeah, many of these organizations are also trying to get under-represented minorities in as well.
And yes, all these groups recognize it's not just diversity that's a good thing, but having people to "stick together". Male nurses are very valuable when dealing with obstinate male patients who are sexist, for example. As are older doctors for those patients who cannot fathom being treated by a doctor who's younger than their kids.
Yep, and you thought that #MeToo being weaponized was bad. You *might* get away from that label with false accusation....maybe/
But if anyone every accuses you of a child related crime, again, even if FALSE....good luck on having a life after that.
Guilty until proven innocent...and even then, well, you know....can't trust you.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
The thing is, most men are not going to make advances on a teaching assistant, or seek to ruin that person's career.
All men are however at risk of a false accusation.
Teaching assistants can make their own choice of which risks to their career they accept. The person to whom you replied is doing exactly the same, by eliminating an obvious risk vector.