Slashdot Mirror
When the Hiring Boss Is an Algorithm
Hugh Pickens writes "Joseph Walker writes at the WSJ that although personality tests have a long history in hiring, sophisticated software has now made it possible to evaluate more candidates, amass more data and peer more deeply into applicants' personal lives and interests. This allows employers to predict specific outcomes, such as whether a prospective hire will quit too soon, file disability claims, or steal. For example after a half-year trial that cut attrition by a fifth, Xerox now leaves all hiring for its 48,700 call-center jobs to software. Xerox used to pay lots of attention to applicants who had done the job before. Then, an algorithm told the company that experience doesn't matter. It determined what does matter in a good call-center worker — one who won't quit before the company recoups its $5,000 investment in training. By putting applicants through a battery of tests and then tracking their job performance, Evolv has developed a model for the ideal call-center worker (PDF). The data recommend a person who lives near the job, has reliable transportation and uses one or more social networks, but not more than four. He or she tends not to be overly inquisitive or empathetic, but is creative. 'Some of the assumptions we had weren't valid,' says Connie Harvey, Xerox's chief operating officer of commercial services. However, data-based hiring can expose companies to legal risk. Practices that even unintentionally filter out older or minority applicants can be illegal under federal equal opportunity laws. If a hiring practice is challenged in court as discriminatory, a company must show the criteria it is using are proven to predict success in the job."
tends not to be overly inquisitive or empathetic
Well, if the bean counters consider the lack of those qualities to be what makes for a good callcenter worker then it's no wonder that the quality of support has gone down as fast as it has. Six or seven years ago when I called into support there was about a 50% chance of reaching someone who was smart and could solve my problem without relying on a script (which never solve my problem because if it can be found in available documentation I've already tried it before calling support), today there's maybe a 5% change if that.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
The irony here is Xerox using this to "copy" their ideal brainless worker to pair asses to seats. Well played, Xerox, well played.
http://marshallbrain.com/manna1.htm
One massive computer controlled database that marks you hireable or not hireable.
I had a hell of a time landing a federal position in the department that I had been working for years as a contractor because the automated system at OPM kept kicking my resume out of the candidate pool. If you fail to get past that, then local hiring managers aren't even aware you have applied, and have no recourse. A co-worker finally gave me pointers on "faking out" the word filters, and I went from "unqualified" to "highly qualified" overnight.
"The U.S. ranks 23rd among developed nations in the percentage of students with undergraduate degrees in science or engineering who are employed in related fields."
That couldn't possibly be due to years of massive overproduction of American STEM graduates, now could it.
They want robots they can put in carbonite at night and not pay.
That wouldn't be efficient at all, the hibernation sickness would have them wiped out for the first half of the shift at least!
Since it doesn't guarantee success, it's a heuristic...and I wouldn't trust anybody trying to sell me one, who doesn't know the difference
It'd be pretty funny when the live HR person pulls up your resume and sees that it's just a word cloud... or scary when you get hired, have been sitting in the cube for a week, and get called into the office over it.
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
Meh. I consider it a heuristic that I use to filter out the employer if they require read access to my Facebook. My Facebook is locked the fuck down; they'd find my name but not much more. If that's a problem, well, I have recruiters emailing me every day, so good luck with your search.
It's better to vote for what you want and not get it than to vote for what you don't want and get it.
- E. Debs
There's a growing trend of hiring intelligent Japanese, Chinese and Indian workers at a fraction of cost to U.S. ones
You think labor rates are cheap in Japan? GDP per-capita in Japan is about 4X that of China and about 10X that of India. Japan has plenty of talent but it isn't particularly cheap or abundant talent. Japan, like the US, relies heavily on automation. Labor intensive industries left Japan years ago just like they did in the US.
The U.S. ranks 23rd among developed nations in the percentage of students with undergraduate degrees in science or engineering who are employed in related fields
Now figure out what that means. It's not at all clear what significance is in having a lower percentage of engineers at a portion of the population. The US is also the third largest country in terms of population so even if they produce a lower percentage of engineers than some other countries they still will produce larger absolute numbers than most of them. You seem to be implying that graduating a lower percentage of engineers/scientists will result in negative consequences. While that might be true you have to back it up with more than just vague implications.
Funny how our education ranking has dropped considerably once the 'No Child Left Behind' bill went into service.
Enforcing everyone passes education at the detriment of our more intelligent children does us no good.
It depends on the job.
For cases of trying to prevent theft people who are working as a Call-Center employee people who have High-Self esteem (AKA Diva's) Will often feel that they are too good for this work, and feel justified of stealing stuff as compensation for the extra pay that they are not getting because they are so great.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
I know there is a general backlash to the increasing use of algorithms in determining major decisions such as hiring. However, from a quantitative standpoint interviews have been shown to be extremely inaccurate as a judge of future job performance. There are simply far too many opportunities for bias on the interviewers part and so they tend to be neither reliable nor valid. Irrelevant characteristics such as appearance end up having far too much weight due to the halo effect. If you want the best result, depending on faulty human judgement is often the wrong choice.
For example, the Apgar score for judging the stability of newborn babies was designed to combat biases on the part of delivery room doctors. Prior to the use of this score, doctors rated how healthy newborns were based on a wide-range of criteria, and each doctor did it differently. When the Apgar score was introduced, it standardized the process by rating newborns on five categories: skin complexion, pulse rate, reflexes, muscle tone, and breathing. The result was that the error introduced by human bias was reduced and countless babies have been saved by quick intervention.
No article on hiring algorithms is complete without mentioning the secretary problem.
In brief, how do you decide that you've interviewed enough people and select a candidate, even though that means ignoring anyone you have yet to interview?
Use of the words "good", "bad" or "evil" is almost invariably the result of oversimplification.
All generic tests for employment, whether marked by hand or by computer, are based on statistical likelihood of success based on past performance indicators. They therefore reduce the range of abilities of workers and guarantee stagnation.
In essence, saying "this appeared to work in the past therefore it's ideal in the future" is the antithesis of progress. And you can't monitor the usefulness of different characteristics because you've already rejected all the employees who don't conform to your ideal.
Big business in the West today is all about low risk mediocrity, i.e. just enough "cost cutting" and profit to maintain a few years of healthy executive bonus. Our performance shows it. The bright, naturally, remain in academia or ensure they have a sufficiently good reputation that they bypass all these stupid tests when entering the commercial world.
Some of the better tech firms understand this: MS abandoned most of its silly puzzles and Google had progressively reduced "college quiz" style interviewing (not quite there yet, though!). When IBM was king, it applied the most costly but effective way of selecting employees: huge probationary periods. Not sure what it does now. I wonder when the average company will catch up and learn?
The U.S. ranks 23rd among developed nations in the percentage of students with undergraduate degrees in science or engineering who are employed in related fields
Gee, do you suppose that's somehow related to lost Science and Engineering jobs due to offshoring in the past decade?
Join the Slashcott! Feb 10 thru Feb 17!
Indirect discrimination is discrimination too.
If your algorithm does discriminate against a group because you are discriminating against things common to that group, you may still have a problem.
That couldn't possibly be due to years of massive overproduction of American STEM graduates, now could it.
Of course not, there's a slashdot article every semester about how we need more people in STEM degrees, especially women.
Especially hot women.
Especially hot women with a fetish for nerdy men, and possibly a tendency for bisexuality.
That $5k is an average number for call center training. For professional positions, it's between 1 and 1.5x annual salary.
Sadly, your brother needs to adopt better parents, because that's how you get jobs. Do you think Mitt Romney, son of a Mexican immigrant who was a migrant farmer and never made more than a subsistence wage and never interacted outside of the migrant community would have had job offers in big firms or ready-made partnerships with well-connected businessmen? Of course not. Take your brother, add in a network of hundreds of friends and colleagues in various fields, have someone prominent in the community and in business vouch personally for his abilities, and I can almost guarantee him a job in under a month, and a 6 figure job in under 5 years - far less if it turns out your brother is both personable and responsible. Add in some ability (numbers, management skills, sales ability) - it doesn't even need to be technical in any way, and he'll be on his way to a very comfortable lifestyle.
Can you claw your way up from the bottom? Yes, but you have to be exceptionally lucky in finding a job with growth and a manager who sees ability and is not threatened by it. Or you have to just be downright good and start your own enterprise from the ground up. The latter generally requires the moral flexibility to spend a lot of time in the gray area of the law (skirt regulation as much as you can) and personal relationships (be a ruthless backstabbing sonofabitch).
Is it just my observation, or are there way too many stupid people in the world?
I see a bunch of problems, including a few that'll leave the company circling the drain down the road. But one obvious one is that the whole thing depends heavily on what you're selecting for. I know my experience on the hiring side is that HR tends to filter out the best-qualified candidates and leave the ones that aren't qualified. That doesn't bode well for their ability to decide what constitutes a successful employee. It may work OK for tier-1 call-center support, but what happens when eg. you decide you want software developers who fix the most bugs the quickest and deliver the most new features the fastest? You end up with developers who write buggy code that can't be maintained or enhanced. You can't fix a lot of bugs quickly unless the code's got a lot of bugs in it, after all, so the criteria would filter out the developers who avoid creating bugs that'd need fixing. And thinking about what the system will need to do 2, 3 or 4 years down the road and coming up with ways of doing things now that'll accommodate those future needs takes more time than duct-taping together something that just about works right now, so you end up selecting for developers who'll hamstring your ability to enhance your system in the future.
In college math we called it the local-optimization problem: you get so caught up in finding the best way to find the maximum/minimum of a function that you end up missing the maximum/minimum.
As usual, most of the respondents either did not RTFA, or simply did not understand it because many of the respondents have got it exactly backwards.
Management did not just make up a set of characteristics they thought would be good (in this case hire local drone) and hire those after doing a drone-test. That's the way it had been done for the last few thousand years.
So here's what happened.
A company tests applicants for a very broad set of characteristics.
They track the performance of the hires.
They compare the success of the hires back to the characteristics found in the test.
They make a model of the successful hires and then use that model to select future hires.
Scientific model:
Construct hypotheses
Gather data
Conduct test
compare result to hypotheses
refine hypotheses
Anyone that is complaining about the algorithmic process and it's outcome has no idea how most people are typically hired.
For the most part, It still boils down to 1: being someone's buddy/relative and 2: looking like someone the HR boss would like to hang out with.
So I, for one, welcome our new algorithmic masters. ( having neither buddy nor looking like someone you would want to hang out with)
Also, this is very far from being new. I know of one upscale hotels started doing this a couple or three decades ago.
They gave all their employees a variety of tests and observed what characteristics were associated with the successful ones in the various positions.
Then, when people apply, they assign them to the position they'll be successful in. The end result is that successful floor-cleaners are happy and productive floor-cleaners, and people whose profile fits the front desk are happy and successful there. And it should be obvious that swapping those two people might create two very resentful employees. It really shows, too, if you ever stayed in a place like that how the good moods of the employees is almost Stepford-spooky.
I've heard my share of stories of Japanese workers not doing much the first many hours, then cramming at the end only because it's taboo to leave before your superior, so long hours with not much more accomplished.
Not sure how true, but if it sounds too good to be true, it probably is. I can't see Japanese not getting burnt out if they actually worked 2x as much.
I'm an industrial/organizational psychologist at Evolv. I help build assessment content and I work closely with our predictive algorithms. A few clarifications from the WSJ article & responses to /. comments:
Yes, creativity and empathy are important for some positions, even in call centers! We're not looking for hateful drones who will hang up on you when you call in. In addition to staying longer, our recommended hires perform better as well. That means increases in both customer satisfaction and efficiency (we call it "average handle time"). But it's a curvilinear relationship - somebody who is too inquisitive is going to tend to waste your valuable time (and their employer's) while trying to resolve your issue. There's a balance.
Most test vendors put a test in place and walk away. At Evolv we take all the post-hire data from our clients and continually feed it back into our algorithms. The content, scoring, and weighting adjust over time to be more predictive.
At Evolv, we don't pair obvious responses when we create questions. So no "I like to steal office supplies" vs "I always show up to work on time" questions. Coupled with the continual refresh & validation of the content, there is no "answer key" that will get you a job. One of the neat things about this approach that we've found is that people applying to entry level positions often don't know what they're good at. Either they've bounced around a few jobs or they're just out of high school. So when somebody applies to a call center job that's hiring for both customer service and sales positions, and we can recommend the position for which they're likely to be "fitter, happier, and more productive"... that's kind of cool. Their employer will make more money off a more stable employee, and the employee ends up doing something they will enjoy just a little bit more. I know some folks will see it from the Radiohead point of view, as creepy (and I respect that), but we think it's better than dumping somebody into a position they're not going to enjoy just because they had the right keywords on their resume or they BS'd their way through an interview.
Science & statistics help eliminate some crazy gut-based hiring decisions. Some hiring managers want to ask call center applicants what they'll be doing in 10 years with an expected response of "I'll be working at this call center". But let's be realistic - while some people enjoy them and thrive, call center jobs are typically not where you plan to be in 10 years. We've also found that resume experience for entry level positions is less important than basic skills and attitude. It's easy to look at that and say "duh" but you'd be surprised how many people hiring & screening for these roles want to exclude applicants who don't have prior experience. So we can cut things out of the interview and hiring process that just don't mean anything.
Evolv doesn't just do employment screening. We periodically follow up with people after they're hired. We find out what information wasn't communicated well during the hiring process, get their feedback on how their training is going, their thoughts on their supervisor, that sort of thing. We feed all of this back in to improve the process. In some cases, that means identifying the trainers whose students perform poorly when they start working. Other times it could be flagging a tenured stellar performer whose numbers are starting to dip for a new position to help reinvigorate them. We strive to improve profitability across the workforce, and do so in an employee-friendly way.
Last but not least, we're still expanding through Xerox, so if you've called their customer service and had a bad experience it must not have been one of our hires. Joking aside, agents are people too, and even our top recommendations have a bad day. We're working hard to to make it better though!
Hope that helps! Yes, there definitely are risks with employment testing, but we try to avoid them and build solutions that make everybody's life a little better.
Cheers,
Tim
I was sought out specifically by a government agency because of some research work I had done and some tools I had developed - they basically had a position that was an EXACT match for my skills, just in a broader scope.
I had to submit my CV to their automated system and was rejected because there was a typo in one of the filter criteria for their automated screening system. Then when they fixed it and I resubmitted, because I was found unqualified previously I was booted out.
They reset the job listing, triple checked the criteria, had me re-format my resume and submit it from a different email address just to make sure it wouldn't reject, but then when a human HR manager looked, she noted I had been rejected previously (but not why) and rejected it again.
Bottom line, you need smart people handling your hiring, and you need to make damn sure your automated systems are helping rather than hindering getting good people in there.
What's funny is that they wound up hiring me as a consultant (costing them at least 3x as much as hiring me on staff would cost) for the work, which worked out great for me since I was able to keep my old job and do the new work telecommuting with only the occasional trip to various sites.
Since I can't tell them apart, I treat all ACs as the same person.
I wouldn't worry about that, employment laws are well entrenched. IANAL (though I am an industrial/organizational psychologist at Evolv), but the employment laws are pretty clear when it comes to discriminating against protected classes. We've also known for years that intelligence is the single best predictor of performance across all job types, but as an industry we can't really use it because intelligence tests tend to discriminate. That's why you see so many personality-style tests.
There are a lot of specific questions employers cannot ask (personal, disability, some criminal history) as well as protected classes which cannot be arbitrarily discriminated against. Protected classes include ethnicity/race, gender, and age (people over 40). We're constantly checking our assessments to ensure they do not discriminate against women, any ethnicity, or older applicants.
Things get trickier when you add the notion of job relevance. IF you are using a screening tool that discriminates, it MUST be job relevant. You cannot disproportionately screen out women who can't lift xx lbs over their head from a firefighting job if that's not something a firefighter actually has to do on the job. You CAN disproportionately screen out blind people for the job of fire truck driver because vision is obviously job relevant.
Most employers try to stick with screening tools that are job relevant AND don't discriminate. Rarely do you see an idiot employer who screens based on some illegal, discriminatory criteria. More commonly it's an edge case, where a tool is job relevant but is discriminatory against some protected class (and isn't as obvious as the blind driver). Somebody sues and it gets settled out of court. It's just safer legally to not discriminate at all, job relevancy or no. At Evolv we include job relevant questions (i.e., call simulations for call center applicants; retail simulations for retail applicants) and situational judgment tests AND check to make sure they don't discriminate against a protected group. We could use job relevant questions that do discriminate since it would be legal, but it's just not worth the risk so you typically don't see it done.
I was once asked (on a formal test, the last hoop) in a job interview for a big box store whether I would turn my mother in if I caught her stealing.
I was entirely unsure of what answer they were looking for. On one hand you would think they would want employees to be that dedicated to protecting their assets. But lets be real, is ruining your family worth your part time job, over a petty theft? Bitch at them and return the item to the store, yes...
I can't imagine anyone (who is 100% sane) would. This would indicate that anyone who answers yes is either a damned liar or not mentally stable. But by saying "no" means you may be immoral (yet, honest?).... so... which was correct? I answered "no" for the sake of honesty. And did not get the job. Not sure if that is why or not, but still wonder.
I would suggest that you check out the concept in employment law known as disparate impact. When combined with the Civil Rights Act of 1964, the Heart of Atlanta Motel v. United States Supreme Court case, and the commerce clause of the US constitution you get a situation where it does expose the employer to a law suit. This was exemplified by the Supreme Court case of Griggs v. Duke Power Co. in 1970 where the employer has to show that a requirement or filter is required to ensure that the individual can do the job. On the other hand the Supreme court case in 2009 of Ricci v. DeStefano does seem to put some limits on this exposure but didn't change it much as the test was designed to ensure competence in the new role.
Time to offend someone
They have already ruled on this. As well as defining how the law is to be interpreted in regards to this. I would suggest reading the following in the order provided:
Civil Rights Act of 1964
Commerce Clause
Heart of Atlanta Motel v. United States
Disparate impact
Griggs v. Duke Power Co.
Ricci v. DeStefano
Time to offend someone
Japanese workers are extremely hard working and incredibly unproductive. I hope they serve as a warning on the importance of work/life balance.
I have worked in Japan, this is VERY true. During much of the work day, and especially the late afternoon/early evening was almost official goof-off time. Then everyone buckled down and got to work in the overtime hours. And if you left before 9pm you were supposed to apologize to everyone. It was weird.
Also, many Japanese Engineers are still paid hourly instead of being salaried, so it is to their advantage to work long hours. Plus, white collar workers wore "uniforms" of some sort everywhere. Often it was just the same color pants and shirt for everyone. And it was a different color for females.
And then there was always the morning "chant" meeting where everyone gathered and did the weird company chant. Of course when I asked my co-workers about any of these things I was always told "It's a Japanese thing."
I used to work in a materials analysis lab and probably 70% of the lab technicians and scientists were female. And probably 70% of those were actually very attractive women. Pretty much everyone was married, including me, but it was still a fun place to work. We had great parties.
Age 25 or under with 9 years experience in a 2 year old technology willing to work for minimum wage.
According to the American Chemical Society the employment rate in 2011 for young chemists is 38%. That's right, a little more than one in three young chemists has a job, and this is from a body notorious for down playing the un- and under employed within the discipline. It isn't significantly better in the other sciences. We might need more programmers. We don't need more scientists, we don't need more engineers, we don't need more mathematicians. The soup lines are already full of them.