What narrow thinking in these comments. I know, "get off my lawn, Slashdot used to be amazing!" has been said before, but all of these comments are simple "LULZ BofA is stoopid!" Come on people, think bigger and more cynically! Why not devote a few analyst cycles to ponder the reality vs simulation question if you're a major finance company? If we are in a simulation, isn't there a healthy chance that simulation includes bugs that could be exploited by economists living within it?
How many billions could you make if you were able to predict glitches in the Matrix?
Now before I'm told how horrible a parent I am, let me explain. Like anything in life or parenting, I think it's about moderation and structure. We gave our daughter one of our older Android phones when she was about 2.5 years. We were embarking on a 5 hour flight with a toddler and a baby so yes, I was looking for distractions. I gave her a locked down phone (no dialing or data connection possible) with two educational apps and one simple drawing app. We play the apps with her and monitor her play. She can also take photos. That's it. It held her interest for a while, as did a number of other activities we brought with us on the plane, but with a limited amount of distraction available she eventually grew bored. She still has access to her phone at home now and she rarely uses it. We've taken it as a backup to many outings but find we only pull it out rarely (specifically dinner at a restaurant, close to or past bedtime, aka the witching hour).
If you give a kid a phone with the entirety of the internet and app store available with no structure, sure they'll become overwhelmed or addicted. Same thing goes for most stuff we give kids as they get older (insert car analogy). But I also think it's important to have access to the tools, connectedness, and creativity that computers and smartphones can unlock. So as my kids get older and they demonstrate they are ready, I'll unlock more functionality. But if and when I do, they can expect a lesson in how to use that new functionality safely and responsibly. If they abuse it, they lose it. I think that's about the best you can do. Maybe I'll be singing a different song in 7 years.
I see a lot of people concerned about what will happen when Uber overturns regulations and kills traditional taxis. Don't be.
Uber is a commodity. They have to grow as fast as possible if they want to remain the top brand. But they're spending tens of millions of dollars of investor money to overturn laws not just for themselves, but for the dozens of competitors waiting in the wings. Once the laws are overturned, it's not like the barrier to entry is that high. I ride Uber 5-10 times per month on business trips, and the drivers typically have both Uber and Lyft running simultaneously. Drivers have 0 commitment to Uber or any other company (in fact most of them grumble about Uber when I ask), they just want fares where they'll earn a reasonable amount. The rating system means that crappy drivers or dirty cars will not last long. Uber can't jack up the rates because passengers will just fire up the competitor apps and strike a free market happy median with the drivers, who also have the competitor apps open. I mean raw capitalism has its problems, but when you ride in Uber of Lyft or your local equivalent a few times, you can see it working pretty damn efficiently.
Of course the whole concept of a driver is now just a short-term thing. As soon as self driving cars show up, these ride share services will truly be a commodity. Ride share will just be a default OS app from Apple and Google and when you summon a car you will not know which ride share company is facilitating the actual vehicle on the back-end. Uber? Lyft? Apple? Google? So long as the car shows up, who cares?
Thanks for the background info! I'm curious, do state DOTs do their own thing or are they like other agencies where the large states tend to force the standard? In other words, if we pressure California DOT to build these platforms responsibly, would that be felt elsewhere in the country as well?
The same challenges exist elsewhere (e.g. hotels) and have been pretty well solved by rating systems, brands/chains (you expect and pay for different levels of cleanliness at Motel 6), and general human decency. I know gross hotels make for good 6 o'clock news footage, but most people still use them and automated cars should be even cleaner. I expect the major fleets to be chock full of cameras, both inside & out, for liability purposes. Trust me, Google/Uber/Hertz will know who is messing up their cars and cut them off quickly. Johnny Cab covered in vomit shows up? Report it via the app and get a new car, a whopping 20 second delay. The previous rider gets slapped with a fat cleaning fee and/or ban.
It may take a while, but the benefits are so great that we'll eventually get out of the personal car mindset. Think about it - eventually there will be no need for parking spaces. If all the cars on the road are shared for-hire, estimates are we will need only a tenth as many cars to meet demand (think 1 automated car running at 100% capacity vs. 10 cars running at 10%). All the cars available during peak hours will be driving, none of them will be parked. During off-peak times (say 4am) the automated cars can just settle down and park on the unused highway lanes or the side of the road. Who needs garages and parking anymore? City density will rocket up as those spots get turned into more housing and office space.
Unlike the hard sciences, awareness of classic social science findings can loop back to impact the phenomena in question or they can change in response to society's evolution. Take the bystander effect for example. How many thousands or millions of college students have learned about the bystander effect in Psych 101? Hypothetically, now that they're aware of it, the effect should diminish and not be quite as reproducible as it once was. Then you layer on societal changes (oblivious smartphone/iTunes users increase the effect, but ubiquitous phones may decrease barriers to reporting and responding to violent crime, etc) and the ability to reproduce an earlier effect becomes muddled.
When a physicist announces a new particle, nothing changes. All the particles keep behaving how they were behaving before the announcement, and they don't care how society changes. The findings should be reproducible 100 years from now.
Many other comments have correctly pointed out that studies in general often focus on the new and shiny and statistically significant rather than reproducing prior results or reporting null findings, but the issue of settling on "truth" is made that much more difficult in the social sciences due to the existence of moving targets.
Hah hah, reminds me of high school when a friend and I started a small web hosting company (pre dot com years). One night we received a frantic call from the local dial-up ISP where our server was co-located. Turns out one of our customers had started hosting some pretty tame stuff (mostly models in swim suits or underwear, but a few nudie pics) and it completely maxed our both our server and the ISP's bandwidth. It created an interesting legal situation given that the two web host owners were also minors - needless to say we had to turn off that user's account. Good times.
As an analyst, to me it's a question of data cleanliness. Yes, people should be able to look at the facts (i.e., crime rate) and route around a higher risk area if they so choose. Trouble is, there's a partial racial component driving those crime statistics (i.e., minorities more likely to be arrested) which probably inflates the "true" crime rates for those neighborhoods. If people are going to get all bent out of shape, they should do so up-stream. Tackle the issues that inject a racial element to crime statistics and leave the people looking for an objective measure of risk assessment alone - they're only using the best available data to make a decision.
No psychologist worth their salt puts any stock in polygraphs, it's law enforcement that loves them. Psych 101: a good test should be reliable and valid. Polygraphs do not meet the criteria.
Unless you're applying for a job that requires security clearance (no presence might be good) or a marketing/PR/public-facing position (having a presence is good), it's really only going to be used to screen people out. If a Google search turns up a red flag about someone else with the same name, you might want to create a LinkedIn profile for yourself to SEO your results and easily distinguish yourself from your negative doppleganger. Recruiters are also using LI much more frequently now to look for talent, so it can't really hurt to have a profile. You might get some leads. Other than that though, there's not much reason to go out and create a presence just for your job search. It's only going to hurt you if you post something a recruiter or hiring manager doesn't like and you're not going to get many brownie points for a post they do like.
Sure, one question type we use a lot is the forced-choice dyad. For example, pick between "My friends say I'm a good teacher" and "The customer is always right" (they're not actually this obvious but you get the idea). #1 might be more predictive of success for tier 2 tech support agents, #2 for basic customer service. But they can vary from company to company, that's where the big data and the specific competencies required for success kick in. So while #1 may predict for most TS agents, at a company that is truly passionate about customer service #2 may still be the best predictor. No single question is going to make or break an applicant, but in aggregate and across many hires you get very real differences.
I work on the pre-hiring screening tool validations at Evolv (full disclosure: Lazslo sits on Evolv's board). I am not at all surprised that silly tech interview questions predict next to nothing. What I can tell you is that validated personality and work-style questions absolutely do predict success among entry-level workers (and if you do it right, professional individual contributors). Like they touch on in the interview, a combination of a structured behavioral interview plus some simple personality screening can be a great screening tool, but you have to balance the raw "big data" results with practical, legal, and applicant experience concerns.
Company gets greedy, company raises prices, opportunities become more enticing for competitors. Sure it will take the market a little while to react, but if the vacuum at the reasonable end of the price spectrum creates more competition from paid or FOSS alternatives, I'm cool with that.
Hi there, I'm an I/O psychologist at Evolv. To dispel a few myths and FUD:
- I absolutely agree there are a lot of HR folks & hiring managers out there who don't do a good job of using scientifically-based, job-relevant screening criteria. And no, simple correlations are not science.
- We don't actually score or weight every statistical relationship our marketing team mentions to the press:)
- Something like this wouldn't be used in isolation as a cut-off. Good selection systems use experience, work samples, appropriate personality & cognitive testing, relevant skills tests, behaviorally structured interviews, etc to come to a holistic hire/no-hire decision, and even then there are always allowances for people who don't fit the mold but who may still do a great job.
- The forums seemed to take this in an anti-M$ direction, but the finding is independent of OS. Installing a non-default browser on a Windows or an Apple machine is linked to positive post-hire outcomes. Having said that, there are many other predictors that are more job relevant.
Ditto! The irony of not researching the viability of your software platform, which is designed to assess the viability of other software platforms, is a little too much.
I generally exercise some degree of distrust towards computer manufacturer recommendations when my product is no longer under warranty and their legal team likely has them relatively well protected against your situation, but I'd definitely name names. Send a note to the Consumerist, find a few execs and contact them directly. It may be legal, but it's a dishonest approach for those companies to take. It doesn't cost you much time and energy to bring unwanted attention to the companies and that attention is sometimes enough to suddenly get your components replaced. It won't cause systematic change, but at least you're better off.
Not one to miss an opportunity for a car analogy: if a critical recall fix bricked your ride, I think most everyone would agree it is the manufacturer's responsibility to make things right even if the vehicle is out of warranty. Of course, there's obviously more regulation involved and a more direct correlation to physical safety in the case of cars (i.e., you are putting yourself at risk of bodily harm if you choose to disregard the recall fix).
I signed up for some term life insurance a few years back and the contract explicitly permitted suicide. That surprised me but I confirmed it with the agent. I image this doesn't apply to everyone (maybe it's not allowed in whole life plans), but I have a standard plan from a major carrier.
Unfortunately my wife also read the fine print so now I have to watch my back...
This may be overly cynical, but I think it's the lesser of two evils. Historically, we've done an awful job of matching people to jobs. It's depressing to learn what really gets people hired and fired. As an I/O psychologist, I cringe whenever an under qualified applicant gets a job just because they know somebody. So as a field, in general, our goal is to make the process more scientific. As a field I/O is generally pretty positive, but it can definitely be used for evil. The strongest force stopping those folks are probably the equal opportunity laws and regulations.
If you want creepy though, the screening piece is just the beginning. Someday careers could be like your Netflix queue. Based on your past jobs, your performance at those jobs, your ratings of those jobs, your knowledge, skills, aptitudes, and interests, and other factors, your career queue could literally suggest new jobs for you and even line up start dates. That's terrifying for many of us, but for others that peace of mind would be a godsend. If it was mandated it's definitely Brave New World. But if it's voluntary? Then it's not so bad. You don't have to watch the movies Netflix suggests... But how would you ever break out and try something new? I think a poor implementation would be linear, single-career (which we know isn't realistic). A smart implementation which could actually enrich a lot of lives, would encourage career jumps every so many years to help keep people engaged and motivated.
Sure, at the end of the day the employer pays our bills. That said, the highly inquisitive employee may not do so well in a customer service roll, but they can often make a great technical support agent. When an employer is trying to fill three positions and we can tell them which one an applicant is most likely to succeed in, we're helping the employer but also indirectly helping the employee. Many applicants for these types of jobs are taking the shotgun approach and apply to everything they find. They just want any job, so if we can match them with one where're they'll tend to do better, it's a least a small improvement over the old way of doing things.
What worries me more is somebody who scores poorly across our entire assessment. Somebody who you might say "does not play well with others". Personality isn't quite as static as pop psychology would have us believe, but that doesn't mean it's easy to change. How do we as a society engage with somebody who wants to work (they're applying for a job) but doesn't have a good attitude for it? Are there really that many entry level jobs to go around for the poor scoring applicants?
You raise very valid concerns. Faking is addressed through two main approaches: different scoring patterns for different jobs and adaptive tests (the content you see today may not be there tomorrow). Many of our clients are trying to fill two or three positions at the same time. That could mean customer service, sales, and tech support at a call center, or cashier and back room at a retailer. A lot of skills and aptitudes are important to both positions, but there will be some differences and even different weighting within the same trait. So a question often has multiple "correct" responses, but one is more likely to point towards sales and the other customer service. In these cases, faking doesn't really help an applicant. Adaptive tests also include a concept called "item exposure" which is simply the number of times a question has appeared in a test. Item exposure becomes part of any adaptive testing algorithm to help determine which question to present next - the more times it's been exposed (thus more susceptible to cheating), the less often it will appear.
Absolutely a concern. Evolv (and any assessment vendor worth their salt) checks this to confirm their products don't cause adverse impact on age, ethnicity, or gender. With that in mind, there's data we've looked at but cannot use. For instance the distance from work variable -- yes it's reasonable to think that people who live further away are more likely to quit, but that data is often tied to socioeconomic status, which is often tied to ethnicity. Poorer people typically don't live next to employers. So we're aware of the correlation but we can't use it.
The big data approach works best in hourly positions where there's plenty of feedback for machine learning. When the inputs are fuzzy, it's a less useful solution. When companies do a poor job measuring performance to begin with, no amount of processing power will save them.
You're absolutely correct, I do think this is the best currently available method for hiring hourly workers. The assessment itself is just one component. For instance, call center applicants also do a mini voice audition (clarity, tone, etc), take a typing test, and complete a behavioral descriptive interview. There's always a human element in the process, we're just trying to make sure the interviewers are asking consistent, job-relevant questions.
I completely agree that people can learn to be good at their job! Our assessments don't quiz people on knowledge they'll pick up in training or on the job. We don't pay much attention to resumes because great hires can come from any background, regardless of whether they have a specific skill set, and be a great fit for the role. When we start working with a new client, this is often a paradigm shift. But we do look at some relevant skills. If two people apply for an email support position and one of them has better typing WPM than the other, it only makes sense to hire the better typist even though the slower typist could learn.
At my previous Fortune 50 employer, top level executives absolutely did take tests like these. The analytics weren't as sophisticated as what we do at Evolv, but the same basic concepts applied. In either case though, these are never the single deciding factor for C-suite or front line positions.
Touche, you got us (I work at Evolv). But our heuristic is accurate enough for us to guarantee our results. Is a "guaranteed heuristic" semantically possible?:)
Slashdot Mirror
What narrow thinking in these comments. I know, "get off my lawn, Slashdot used to be amazing!" has been said before, but all of these comments are simple "LULZ BofA is stoopid!" Come on people, think bigger and more cynically! Why not devote a few analyst cycles to ponder the reality vs simulation question if you're a major finance company? If we are in a simulation, isn't there a healthy chance that simulation includes bugs that could be exploited by economists living within it?
How many billions could you make if you were able to predict glitches in the Matrix?
Now before I'm told how horrible a parent I am, let me explain. Like anything in life or parenting, I think it's about moderation and structure. We gave our daughter one of our older Android phones when she was about 2.5 years. We were embarking on a 5 hour flight with a toddler and a baby so yes, I was looking for distractions. I gave her a locked down phone (no dialing or data connection possible) with two educational apps and one simple drawing app. We play the apps with her and monitor her play. She can also take photos. That's it. It held her interest for a while, as did a number of other activities we brought with us on the plane, but with a limited amount of distraction available she eventually grew bored. She still has access to her phone at home now and she rarely uses it. We've taken it as a backup to many outings but find we only pull it out rarely (specifically dinner at a restaurant, close to or past bedtime, aka the witching hour).
If you give a kid a phone with the entirety of the internet and app store available with no structure, sure they'll become overwhelmed or addicted. Same thing goes for most stuff we give kids as they get older (insert car analogy). But I also think it's important to have access to the tools, connectedness, and creativity that computers and smartphones can unlock. So as my kids get older and they demonstrate they are ready, I'll unlock more functionality. But if and when I do, they can expect a lesson in how to use that new functionality safely and responsibly. If they abuse it, they lose it. I think that's about the best you can do. Maybe I'll be singing a different song in 7 years.
I see a lot of people concerned about what will happen when Uber overturns regulations and kills traditional taxis. Don't be.
Uber is a commodity. They have to grow as fast as possible if they want to remain the top brand. But they're spending tens of millions of dollars of investor money to overturn laws not just for themselves, but for the dozens of competitors waiting in the wings. Once the laws are overturned, it's not like the barrier to entry is that high. I ride Uber 5-10 times per month on business trips, and the drivers typically have both Uber and Lyft running simultaneously. Drivers have 0 commitment to Uber or any other company (in fact most of them grumble about Uber when I ask), they just want fares where they'll earn a reasonable amount. The rating system means that crappy drivers or dirty cars will not last long. Uber can't jack up the rates because passengers will just fire up the competitor apps and strike a free market happy median with the drivers, who also have the competitor apps open. I mean raw capitalism has its problems, but when you ride in Uber of Lyft or your local equivalent a few times, you can see it working pretty damn efficiently.
Of course the whole concept of a driver is now just a short-term thing. As soon as self driving cars show up, these ride share services will truly be a commodity. Ride share will just be a default OS app from Apple and Google and when you summon a car you will not know which ride share company is facilitating the actual vehicle on the back-end. Uber? Lyft? Apple? Google? So long as the car shows up, who cares?
Thanks for the background info! I'm curious, do state DOTs do their own thing or are they like other agencies where the large states tend to force the standard? In other words, if we pressure California DOT to build these platforms responsibly, would that be felt elsewhere in the country as well?
The same challenges exist elsewhere (e.g. hotels) and have been pretty well solved by rating systems, brands/chains (you expect and pay for different levels of cleanliness at Motel 6), and general human decency. I know gross hotels make for good 6 o'clock news footage, but most people still use them and automated cars should be even cleaner. I expect the major fleets to be chock full of cameras, both inside & out, for liability purposes. Trust me, Google/Uber/Hertz will know who is messing up their cars and cut them off quickly. Johnny Cab covered in vomit shows up? Report it via the app and get a new car, a whopping 20 second delay. The previous rider gets slapped with a fat cleaning fee and/or ban.
It may take a while, but the benefits are so great that we'll eventually get out of the personal car mindset. Think about it - eventually there will be no need for parking spaces. If all the cars on the road are shared for-hire, estimates are we will need only a tenth as many cars to meet demand (think 1 automated car running at 100% capacity vs. 10 cars running at 10%). All the cars available during peak hours will be driving, none of them will be parked. During off-peak times (say 4am) the automated cars can just settle down and park on the unused highway lanes or the side of the road. Who needs garages and parking anymore? City density will rocket up as those spots get turned into more housing and office space.
Unlike the hard sciences, awareness of classic social science findings can loop back to impact the phenomena in question or they can change in response to society's evolution. Take the bystander effect for example. How many thousands or millions of college students have learned about the bystander effect in Psych 101? Hypothetically, now that they're aware of it, the effect should diminish and not be quite as reproducible as it once was. Then you layer on societal changes (oblivious smartphone/iTunes users increase the effect, but ubiquitous phones may decrease barriers to reporting and responding to violent crime, etc) and the ability to reproduce an earlier effect becomes muddled.
When a physicist announces a new particle, nothing changes. All the particles keep behaving how they were behaving before the announcement, and they don't care how society changes. The findings should be reproducible 100 years from now.
Many other comments have correctly pointed out that studies in general often focus on the new and shiny and statistically significant rather than reproducing prior results or reporting null findings, but the issue of settling on "truth" is made that much more difficult in the social sciences due to the existence of moving targets.
Hah hah, reminds me of high school when a friend and I started a small web hosting company (pre dot com years). One night we received a frantic call from the local dial-up ISP where our server was co-located. Turns out one of our customers had started hosting some pretty tame stuff (mostly models in swim suits or underwear, but a few nudie pics) and it completely maxed our both our server and the ISP's bandwidth. It created an interesting legal situation given that the two web host owners were also minors - needless to say we had to turn off that user's account. Good times.
As an analyst, to me it's a question of data cleanliness. Yes, people should be able to look at the facts (i.e., crime rate) and route around a higher risk area if they so choose. Trouble is, there's a partial racial component driving those crime statistics (i.e., minorities more likely to be arrested) which probably inflates the "true" crime rates for those neighborhoods. If people are going to get all bent out of shape, they should do so up-stream. Tackle the issues that inject a racial element to crime statistics and leave the people looking for an objective measure of risk assessment alone - they're only using the best available data to make a decision.
Easier said than done of course...
No psychologist worth their salt puts any stock in polygraphs, it's law enforcement that loves them. Psych 101: a good test should be reliable and valid. Polygraphs do not meet the criteria.
Unless you're applying for a job that requires security clearance (no presence might be good) or a marketing/PR/public-facing position (having a presence is good), it's really only going to be used to screen people out. If a Google search turns up a red flag about someone else with the same name, you might want to create a LinkedIn profile for yourself to SEO your results and easily distinguish yourself from your negative doppleganger. Recruiters are also using LI much more frequently now to look for talent, so it can't really hurt to have a profile. You might get some leads. Other than that though, there's not much reason to go out and create a presence just for your job search. It's only going to hurt you if you post something a recruiter or hiring manager doesn't like and you're not going to get many brownie points for a post they do like.
Sure, one question type we use a lot is the forced-choice dyad. For example, pick between "My friends say I'm a good teacher" and "The customer is always right" (they're not actually this obvious but you get the idea). #1 might be more predictive of success for tier 2 tech support agents, #2 for basic customer service. But they can vary from company to company, that's where the big data and the specific competencies required for success kick in. So while #1 may predict for most TS agents, at a company that is truly passionate about customer service #2 may still be the best predictor. No single question is going to make or break an applicant, but in aggregate and across many hires you get very real differences.
I work on the pre-hiring screening tool validations at Evolv (full disclosure: Lazslo sits on Evolv's board). I am not at all surprised that silly tech interview questions predict next to nothing. What I can tell you is that validated personality and work-style questions absolutely do predict success among entry-level workers (and if you do it right, professional individual contributors). Like they touch on in the interview, a combination of a structured behavioral interview plus some simple personality screening can be a great screening tool, but you have to balance the raw "big data" results with practical, legal, and applicant experience concerns.
Company gets greedy, company raises prices, opportunities become more enticing for competitors. Sure it will take the market a little while to react, but if the vacuum at the reasonable end of the price spectrum creates more competition from paid or FOSS alternatives, I'm cool with that.
Hi there, I'm an I/O psychologist at Evolv. To dispel a few myths and FUD:
Hope that helps!
Ditto! The irony of not researching the viability of your software platform, which is designed to assess the viability of other software platforms, is a little too much.
Bryan & Harter (1899) noticed telegraph operators could identify one another through rhythm and style, nice to see someone finally apply that! :-)
http://psycnet.apa.org/journals/rev/6/4/345/
I generally exercise some degree of distrust towards computer manufacturer recommendations when my product is no longer under warranty and their legal team likely has them relatively well protected against your situation, but I'd definitely name names. Send a note to the Consumerist, find a few execs and contact them directly. It may be legal, but it's a dishonest approach for those companies to take. It doesn't cost you much time and energy to bring unwanted attention to the companies and that attention is sometimes enough to suddenly get your components replaced. It won't cause systematic change, but at least you're better off.
Not one to miss an opportunity for a car analogy: if a critical recall fix bricked your ride, I think most everyone would agree it is the manufacturer's responsibility to make things right even if the vehicle is out of warranty. Of course, there's obviously more regulation involved and a more direct correlation to physical safety in the case of cars (i.e., you are putting yourself at risk of bodily harm if you choose to disregard the recall fix).
What are they talking about? Drones aren't murderous, they're just full of angst!
http://www.hulu.com/watch/426530
I signed up for some term life insurance a few years back and the contract explicitly permitted suicide. That surprised me but I confirmed it with the agent. I image this doesn't apply to everyone (maybe it's not allowed in whole life plans), but I have a standard plan from a major carrier.
Unfortunately my wife also read the fine print so now I have to watch my back...
This may be overly cynical, but I think it's the lesser of two evils. Historically, we've done an awful job of matching people to jobs. It's depressing to learn what really gets people hired and fired. As an I/O psychologist, I cringe whenever an under qualified applicant gets a job just because they know somebody. So as a field, in general, our goal is to make the process more scientific. As a field I/O is generally pretty positive, but it can definitely be used for evil. The strongest force stopping those folks are probably the equal opportunity laws and regulations.
If you want creepy though, the screening piece is just the beginning. Someday careers could be like your Netflix queue. Based on your past jobs, your performance at those jobs, your ratings of those jobs, your knowledge, skills, aptitudes, and interests, and other factors, your career queue could literally suggest new jobs for you and even line up start dates. That's terrifying for many of us, but for others that peace of mind would be a godsend. If it was mandated it's definitely Brave New World. But if it's voluntary? Then it's not so bad. You don't have to watch the movies Netflix suggests... But how would you ever break out and try something new? I think a poor implementation would be linear, single-career (which we know isn't realistic). A smart implementation which could actually enrich a lot of lives, would encourage career jumps every so many years to help keep people engaged and motivated.
Sure, at the end of the day the employer pays our bills. That said, the highly inquisitive employee may not do so well in a customer service roll, but they can often make a great technical support agent. When an employer is trying to fill three positions and we can tell them which one an applicant is most likely to succeed in, we're helping the employer but also indirectly helping the employee. Many applicants for these types of jobs are taking the shotgun approach and apply to everything they find. They just want any job, so if we can match them with one where're they'll tend to do better, it's a least a small improvement over the old way of doing things.
What worries me more is somebody who scores poorly across our entire assessment. Somebody who you might say "does not play well with others". Personality isn't quite as static as pop psychology would have us believe, but that doesn't mean it's easy to change. How do we as a society engage with somebody who wants to work (they're applying for a job) but doesn't have a good attitude for it? Are there really that many entry level jobs to go around for the poor scoring applicants?
You raise very valid concerns. Faking is addressed through two main approaches: different scoring patterns for different jobs and adaptive tests (the content you see today may not be there tomorrow). Many of our clients are trying to fill two or three positions at the same time. That could mean customer service, sales, and tech support at a call center, or cashier and back room at a retailer. A lot of skills and aptitudes are important to both positions, but there will be some differences and even different weighting within the same trait. So a question often has multiple "correct" responses, but one is more likely to point towards sales and the other customer service. In these cases, faking doesn't really help an applicant. Adaptive tests also include a concept called "item exposure" which is simply the number of times a question has appeared in a test. Item exposure becomes part of any adaptive testing algorithm to help determine which question to present next - the more times it's been exposed (thus more susceptible to cheating), the less often it will appear.
Absolutely a concern. Evolv (and any assessment vendor worth their salt) checks this to confirm their products don't cause adverse impact on age, ethnicity, or gender. With that in mind, there's data we've looked at but cannot use. For instance the distance from work variable -- yes it's reasonable to think that people who live further away are more likely to quit, but that data is often tied to socioeconomic status, which is often tied to ethnicity. Poorer people typically don't live next to employers. So we're aware of the correlation but we can't use it.
The big data approach works best in hourly positions where there's plenty of feedback for machine learning. When the inputs are fuzzy, it's a less useful solution. When companies do a poor job measuring performance to begin with, no amount of processing power will save them.
You're absolutely correct, I do think this is the best currently available method for hiring hourly workers. The assessment itself is just one component. For instance, call center applicants also do a mini voice audition (clarity, tone, etc), take a typing test, and complete a behavioral descriptive interview. There's always a human element in the process, we're just trying to make sure the interviewers are asking consistent, job-relevant questions.
I completely agree that people can learn to be good at their job! Our assessments don't quiz people on knowledge they'll pick up in training or on the job. We don't pay much attention to resumes because great hires can come from any background, regardless of whether they have a specific skill set, and be a great fit for the role. When we start working with a new client, this is often a paradigm shift. But we do look at some relevant skills. If two people apply for an email support position and one of them has better typing WPM than the other, it only makes sense to hire the better typist even though the slower typist could learn.
At my previous Fortune 50 employer, top level executives absolutely did take tests like these. The analytics weren't as sophisticated as what we do at Evolv, but the same basic concepts applied. In either case though, these are never the single deciding factor for C-suite or front line positions.
Touche, you got us (I work at Evolv). But our heuristic is accurate enough for us to guarantee our results. Is a "guaranteed heuristic" semantically possible? :)