Google's Answer to Filling Jobs Is an Algorithm

An anonymous reader tipped us to a New York Times article about Google's newest HR tool: an algorithm. Starting soon, the company (which gets roughly 100,000 applications a month) will require all interested applicants to fill out an in-depth survey. They'll be using a sophisticated algorithm to work through the submitted surveys, matching applicants with positions. The company has apparently doubled in size in each of the last three years. Even though it's already 10,000 employees strong Laszlo Bock, Google's vice president for people operations, sees no reason the company won't reach 20,000 by the end of the year. This will mean hiring something like 200 people a week, every week, all year. From the article: "Even as Google tries to hire more people faster, it wants to make sure that its employees will fit into its freewheeling culture. The company boasts that only 4 percent of its work force leaves each year, less than other Silicon Valley companies. And it works hard to retain people, with copious free food, time to work on personal projects and other goodies. Stock options and grants certainly encourage employees to stay long enough to take advantage of the company's surging share price. Google's hiring approach is backed by academic research showing that quantitative information on a person's background -- called 'biodata' among testing experts -- is indeed a valid way to look for good workers."

Comments

Bias

[Threni]

It will be interesting to see if any company using this technique ever get accused of racial,sexual etc bias.

"But the computer chose them! You're not going to sue my computer, are you?"

Re:Bias

[Shmooze]

It depends how good their algorithm is - let's say it looks at what proportion of your life since graduating you've been in work, where more is better. That's a disadvantage to women because they (generally) take time off to have/raise kids and so on, even though the algorithm isn't specifically designed to discriminate against them.

(OK, so it's a trivial case, but you get the general idea)

I suspect there could be plenty of arguments in court about whether some nuance of the algorithm treats some group unfairly or not...

Re:Bias

[inviolet]

It will be interesting to see if any company using this technique ever get accused of racial,sexual etc bias.

What if google's statistical data (drawn from its database of performance reviews) shows that some ages, genders, races, and cultures are objectively better at a particular job than others?

Google's test will obviously avoid asking any direct questions about age, gender, and race, because that's illegal (even when objectively justifiable). However, if the test is powered by a statistics engine drawing a database of past performance reviews, then the test could unintentionally evolve to ask about such things indirectly.

An example: perhaps cat-ownership is correllated with femaleness, and femaleness is correllated with superior performance in writing technical documentation. An automated test-generator would unwittingly evolve to ask applicants about cat-ownership, in order to unwittingly select superior female candidates.

It's an amusing possibility. Indeed, it would be the free-market's way of legitimately selecting candidates based on age/gender/race while remaining underneath the legal radar.

Re:Bias

[general_re]

Except "people with less work experience" is not a protected group, so it's not unlawful to discriminate on the basis of previous work experience, unless you do so with the intent of discriminating against an actual protected group. I'm just guessing, but I'd say it would be awfully hard to win a case based on such "discrimination", short of someone admitting that they did it to avoid hiring women.

No. See Griggs v. Duke Power Co., 401 U.S. 424, 431-2 (1971). A plaintiff can show that some employment criterion or criteria results in a disparate impact upon a protected group, regardless of whether discrimination is overtly intended or not. The burden of proof then shifts to the employer to show that said criteria are a necessary requirement for the job(s) in question. If they can't, they lose. Even if they can, if the plaintifss can come up with an alternate business practice that satisfies the employer's interests without resulting in a disparate impact, they lose. Good or bad, that's the law.

Re:Bias

[yali]

As you might guess, there is a whole tangle of legal and ethical issues surrounding testing in personnel selection.

My understanding (IANAL etc) is that you are supposed to assess only the skills, aptitude, etc. that you can defend as related to the job. If that happens to be correlated with sex, race, age, etc., the correlation is not a problem, but you cannot use those things as a proxy for what you're really interested in. For example, in a job that requires quick responses, you can test people's reaction times, but you cannot automatically exclude people based on age (even though age may be correlated with reaction time).

More direct assessment is better anyway. Suppose you are hiring for a job that requires math skills, which you believe is correlated with gender, which you believe is correlated with cat ownership. Even if those correlations exist, you'd still get more accurate results measuring the math skills directly rather than measuring cat ownership which is correlated with something that is correlated with what you need.

Today Google Jobs...

[Flimzy]

Tomorrow Google online dating?

Re:Today Google Jobs...

[panaceaa]

I'm feeling lucky.

Re:Psychological profile included ala The Game?

[gardyloo]

Do you feel guilty when you masturbate?
Do you enjoy harming animals?

      Huh. I must be really efficient. Killing two birds with one stone, no beating around the bush. Er... as it were!

Only 4% turnover? It's going to rise

[vidarh]

It's easy to maintain a low turnover of staff as long as the vast majority of your staff isn't fully vested, and the stock is moving upwards. As soon as the growth in staff numbers slow down, though, you're going to see the turnover percentage increase significantly as a larger and larger percentage of staff have been there for the full 4 year vesting period of their options, and the company starts seeing pressure for lower refresher grants.

A lawsuit waiting to happen

[hellfire]

You can't hide bad questions behind an algorythm. The interview process has lots of laws around it now, and it's well established that there are only some questions you can ask. Here's a great example:

The questions range from the age when applicants first got excited about computers...

This question doesn't directly reveal your age, but a clever interviewer can glean much from it. "Oh, got excited in computers at 22, eh? Probably older than I thought. We don't want old employees we want young ones."

It is illegal to ask some questions in an interview. Age related questions are one of them. You are only allowed to ask questions that pertain to your performance of the job at hand. For example, I can ask someone "would you have a problem lifting heavy boxes?" but I can't ask how old you are and make a judgement because you are 40 that you can't lift heavy boxes. The above question you as a logical geek might think is iffy, but to a lawyer, it's shark bait and they'll be all over it, so don't ask it. If you ask a question that falls into this category, you open yourself up to a gender/age/racial discrimination lawsuit. These and many others are protected classes under the law.

And there's a great reason why an interview is a poor indicator of performance... because people lie!!! It's a sales process. They want your job, and you want the best candidate. Last two people I let go both gave great interviews, but when they actually worked, they sucked. They had all the right answers in the interview, but there is no escaping performance reviews.

0% firing rate is impossible, as is 100% retention. 96% retention is a stellar figure, even for silicon valley. I think they should be pretty happy that number.

Re:A lawsuit waiting to happen

[Jerf]

Your post triggered an interesting thought process.

Google knows AI and machine learning; even if they don't use it they'll have people who know about it.

Suppose by asking certain questions, and doing some initial research and calibration, I can determine your age within two years with 97% certainty. Or marital status, or race, or any of the other protected categories. Have I broken the law? What if I don't actually do the computation? What if my computers do the computation but no human ever sees it? What if I do the computation and no human ever directly sees the result but the computer has enough power to say "No" to a hire in practice, thus still incorporating this potentially "forbidden knowledge" into the hiring decision?

(After all, asking someone about their marital status may actually be less reliable in the end; I can easily imagine 1 out of 40 people lying about something like that, or their true age/race/etc. if asked.)

This is extremely likely to be possible, and probably downright easy for Google, so this isn't just a hypothetical. And the problems this raises extends beyond this exact instance into any domain where for legal reasons, we have to cultivate ignorance; exactly what constitutes "ignorance" if you get right down to it?

Re:Thankfully they changed the GPA thing

[panaceaa]

I've been working at Google for four months, and of all the companies I interviewed at, Google seemed to care the least about my past projects, experience, or my GPA. Google's interviewing process is all about finding very smart computer people. You simply must know the core computer science principles, but it does not matter if you were able to regurgitate them on your college exams. It matters that you can explain them in an interview and use them towards solving a problem. Once I got here, I can understand the reasoning behind the hiring process: Lots of Google infrastructure and technology is unique to Google. Look at the published articles on Bigtable and MapReduce to get a glimpse of the unique systems used every day here. For people to learn these systems and begin being productive quickly, Google doesn't care if you have an MSCE or know the syntax of Apache's httpd.conf. Google just needs you to be smart.

note: These are my opinions and not necessarily those of Google's. And I try not to post on Google articles nowadays, but this doesn't pertain to our business strategy so I'm comfortable sharing it. BTW we had an awesome free lunch today here in Kirkland, Washington. :)

20,000 vs 200 x 100?

[cei]

At the height of the dotcom bubble, Bill Gross & Idealab! had the philosophy that no company should have more than 100 employees. If your business model got above 100 employees, there was a high likelihood that you were better off dividing and spinning off other business units. (Don't know if they still preach that or not, but that was the thinking "back in the day.")

I don't know that Google would be better served as two hundred smaller companies, but at the same time, it's hard to imagine managing 20,000 employees would be any easier.

the code..

[Mogster]

bool recommend = true;

if (surveyResults == Evil())
        forwardResumeToMicrsoft(bool recommend);

hire();

Re:Thankfully they changed the GPA thing

[panaceaa]

Smart, but not *too* smart. Too smart people have ideas and go and start their own companies.

This statement could not be further from the truth. One of my fellow co-workers is brilliant and he ran his own hosted content management company for years before joining Google. The three people who started a company that eventually became Google Talk are still working here in Kirkland, Washington. One of my friends here at the Kirkland office just moved to San Mateo, California, to work with the engineers at YouTube and learn from their entrepreneurial experiences. And just yesterday we had the founder of JotSpot, which Google acquired a few months back, come to help us with our latest product strategy. The people here are extremely smart, they have run their own companies in the past, and Google's very happy to have them. (And as far as I can tell, they're happy to be here :).)

Re:They're not losing money fast enough...

[dirc]

Lets see... 10,000 employees, on average, costing the corp ~$200k each... that's... $20 billion a year... in salaries/benefits/office space/etc. Are they even making that much? Are they paying their workers with ``profits'' from stock sales?

10,000 employees at $200k each is $2 billion a year, not $20 billion a year. Google is making enough to cover those costs even if they double the number of employees and do not increase revenue at all. You can look at a summary of their revenue, and their expenses as a portion of revenue here: http://biz.yahoo.com/e/061108/goog10-q.html

They are making a handsome profit.

Re:The Dot-Bomb Trap

[starglider29a]

No problem. Then, they can devise an algorithm to decide who to lay off.