Slashdot Mirror
Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women (reuters.com)
Jeffrey Dastin, reporting for Reuters: Amazon's machine-learning specialists uncovered a big problem: their new recruiting engine did not like women. The team had been building computer programs since 2014 to review job applicants' resumes with the aim of mechanizing the search for top talent, five people familiar with the effort told Reuters. Automation has been key to Amazon's e-commerce dominance, be it inside warehouses or driving pricing decisions. The company's experimental hiring tool used artificial intelligence to give job candidates scores ranging from one to five stars -- much like shoppers rate products on Amazon, some of the people said. "Everyone wanted this holy grail," one of the people said. "They literally wanted it to be an engine where I'm going to give you 100 resumes, it will spit out the top five, and we'll hire those." But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way. That is because Amazon's computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.
[...] Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said. The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity.
[...] Amazon edited the programs to make them neutral to these particular terms. But that was no guarantee that the machines would not devise other ways of sorting candidates that could prove discriminatory, the people said. The Seattle company ultimately disbanded the team by the start of last year because executives lost hope for the project, according to the people, who spoke on condition of anonymity.
Train algorithm with data in hand, algorithm's output mirrors data provided. They can't possibly be shocked by this, can they?
Which has more power: the hammer, or the anvil?
When government reviews your hiring they expect you to show that your diversity level is consistent with the normal spread of minority groups ( some consideration of candidate pool MAY be given.)
In other words, if your only criteria is hiring whomever best for the job, you will likely be operating illegally and subject to fines and lawsuits. This is the product of laws that are designed to create social engineering based restrictions based on someones religious idea that any measurable discrepancy in minority placement must be corrected.
âoeTolerance applies only to persons, but never to truth. Intolerance applies only to truth, but never to persons.
Amazon trained their AI using the dataset that reflected their business practices as they currently are (flaws and all) but what they wanted was a data set for the practices they wanted to become (i.e. the ideal).
Finding a training dataset that reflects the ideal is going to be extremely difficult, particularly in an area where that ideal is so poorly defined.
When you read this article it doesn't say anything about this algorithm not 'liking woman'. Based on the parameters it was given it chose to rank candidates based upon the factors it was trained to look for. It's also somewhat telling how the writers of this tripe chose to specifically highlight how the algorithm chose to downgraded candidates from two all female colleges without saying why they were downgraded. As if the fact that it's an all female school is more important than the quality of the candidates that came out of the school.
At the end of the day this bullshit is more about how the media writes headlights to illicit emotional reactions instead of reporting the hows and the whys of a situation. And on that note I'd like to see someone actually start writing algorithms to to replace tech reporters so we can get ride of garbage tier activist journalism like this article.
Purge any submission to the system of a gender identifier... women's or men's anything... remove names in case that is factored... literally provide nothing in the submission that would definitively define a gender.
Then see what it does.
My experience with these systems is that they don't actually factor gender but that the end result of is that there is a gender imbalance.
However, if there is an imbalance and the system was given no indication as to gender then there is no gender bias.
You can't cite persecution or preference if the system can't even know. And generally these fairly common and consistent imbalances are made without reference to gender itself.
Generally it is factoring on other criteria that give the same result but which are not gender. Work experience is a big one... breadth of skill set is another.
And if you took the total population and look at which portion of the population had that work experience and breadth of knowledge, you'd find it more closely matched the hiring patterns of these systems. Which means it isn't factoring on gender.
Now... this is assumption to some extent on my part. I've audited these systems in the past and what I am describing above is the pattern I've seen.
As to what the Amazon system was doing... I'd have to audit it.
What I'd probably try is a word replacement/purge of all terms that would signify gender or I'd just change a bunch of rejected female resumes to say they were male and see if they got accepted and vice versa.
If the system actually changed its decision based on gender then that's a smoking gun that it is doing things on the basis of gender.
But I'd find that very surprising.
Machine learning is unpredictable so I'm hardly going to claim to know what the damned thing was doing. For that reason I wouldn't actually use machine learning in this application. I'd use a very clear rules based system where everything it was doing was known to the programmers.
Those systems are completely fine for this sort of work and you can very easily audit the code for them.
The best way to deal with this is to first be gender blind. You literally do not factor for gender at all.
That will give you an imbalance probably... you can make as many diversity hires as you need to after that. But your core hiring pool should be merit based unless you want to go out of business.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
there's a huge gender imbalance in nursing and primary education as well; when will society get around to 'fixing' that?
The thing is, most men are not going to make advances on a teaching assistant, or seek to ruin that person's career.
All men are however at risk of a false accusation.
Teaching assistants can make their own choice of which risks to their career they accept. The person to whom you replied is doing exactly the same, by eliminating an obvious risk vector.