Are Brain Teasers Good Hiring Criteria?

theodp writes "Your brain teaser prowess may win you a job at Google, but the folks at 37signals don't hire programmers based on puzzles, API quizzes, math riddles, or other parlor tricks. 'The only reliable gauge I've found for future programmer success,' explains 37signals' David Heinemeier Hansson, 'is looking at real code they've written, talking through bigger picture issues, and, if all that is swell, trying them out for size.'" Those of you who have hired employees: have you seen correlation between interview puzzle success and job competency? How should an interviewee best handle these questions?

Comments

Well, they're a good indicator of intelligence

[elrous0]

If someone is giving you one, they're probably not very intelligent.

Re:Well, they're a good indicator of intelligence

[ILongForDarkness]

Perhaps. The article says "looking at real code" is better. Again perhaps. For example the problem there is: did they really write the code, if so how long did it take? Did someone else suggest fixes etc? You don't know. I mean 300 lines of beautiful C is all fine and dandy but if it took you 3 months to write it and half of it is cut and pasted from the web how good is it really?

What brain teasers hopefully do is take a problem close to the types of things you see in the job. Even though it is all programming different companies either due to industry or existing infrastructure/policies tend to have different types of coding "puzzles" that come up again and again. Hopefully this test problem is one you haven't seen before and they get to see how you approach something you don't already know how to solve, how close to a good design do you get on the first interation, if they point out a problem how you go about fixing it etc. All real world important stuff to know about someone.

Re:Well, they're a good indicator of intelligence

[leonbloy]

In my experience: They're a moderately good indicator of a special kind of intelligence; which is not a very useful indicator in the typical hiring process.

Puzzles help to distinguish programmers from lawyers. Not to discriminate good programmers from bad programmers.

Re:Well, they're a good indicator of intelligence

[jimbolauski]

I agree. But I would prefer a puzzle to questions like "where do you see yourself in 5 years" and "what are your goals". I want to answer "My goal is to get hired. Why else would I answer such stupid questions?"

Those questions are more pertinent to MBA's where your ability to pile on BS in a believable way is an important skill.

Re:Well, they're a good indicator of intelligence

[w_dragon]

The 5 years question is really just asking if you intend to move up the management or technical ladder. If I'm looking for someone who may eventually become a software architect and a candidate tells me they want to be a VP in 5 years I may think twice before hiring them.

Re:Well, they're a good indicator of intelligence

[Whorhay]

Is using code snippets from the internet really an issue? Is there a good reason to re-invent the wheel, pulley, block and tackle, lever, wrench, lightbulb and whatnot every time?

Time constraints could certainly be a problem if it takes longer for someone to lookup a solution and implement it than if they just come up with the solution themselves and implement it. And there are legal issues with copying large chunks of code, but I wouldn't think that banning it wholesale is a very productive way to go.

Re:Well, they're a good indicator of intelligence

[asliarun]

If someone is giving you one, they're probably not very intelligent.

I completely disagree, or at least, your statement is so broad it is untrue.

Brain teasers are just like any other interviewing tool - what matters is how you use the tool.
As an interviewer, if you use brain teasers to determine *how* the candidate is attempting to solve the problem, you are probably doing it right.
If you are using the brain teaser to tick a box in your checklist based on the answer, you're probably doing it wrong.

The really neat thing about brain teasers or puzzles or the bizarre questions you sometimes encounter like "How many pigeons are there is Manhattan" is that they are a very good way to judge someone's unstructured problem solving ability. How someone approaches this kind of a problem is a good proxy for their ability to debug hard technical issues or their problem solving ability in general.

Making a statement like "hire a programmer based on their programming ability" is also an obvious statement to make, apart from being a bit grandiose (look at us , we are cool because we are contrarians and we swim against the tide). The reason why many interviewers resort to other techniques is two fold - one, lack of time or other constraints that prevent the interviewer from directly testing a programmer's programming ability, and secondly, judge the non-programming aspects of the candidate like how they react to an ill-defined problem or a fuzzy situation, how well they will get along with others, how much of a self-starter they are etc.

Or, if I put it another way, if you are not hiring a programmer on the, to quote, "code they have written", what are you doing, hiring candidates on their baking skills? I get what 37signals is saying and all this got messed up to begin with when HR took over the interviewing process from programmers (especially in large companies). However, the other statements that are flying around about how *any* non-programming related question is stupid is also frankly, over the top.

Re:Well, they're a good indicator of intelligence

[Anonymous Brave Guy]

The 5 years question is really just asking if you intend to move up the management or technical ladder.

Then why not just ask that directly? It might lead to a mutually interesting conversation, and at the very least you're either both going to be more confident in your fit or both going to know immediately that it's not going to work out.

These are my personal interview red flags for a software developer job, starting with the most likely to result in me throwing the interview/leaving early/declining any offer:

1. Hiring is based on brain teasers, $-based certifications, or other similarly irrelevant criteria.
2. Wants to know my current salary. (Bonus point: Doesn't give any indication at the same time of the likely compensation if I'm offered the job. Extra bonus point: Makes clear that the salary range given in the job ad was wildly optimistic.)
3. Won't show me an example of their production code, real documentation, etc. when given a reasonable opportunity to do so.

In each case, someone is skirting what matters, instead of finding out as fast as possible whether we are really a good fit and a mutually satisfactory hire might result.

Re:Well, they're a good indicator of intelligence

[delinear]

It helps if you have 3 months salary in reserve for emergencies like you should so you don't end up entering a bad situation out of desperation.

This is the best advice out there. If you enter any job interview from a position of needing the job rather than wanting it, you risk not being happy in your role. And don't be afraid to turn the question back around - if someone is asking you where you see yourself in five years, why are they asking that? Is it because they want someone who will grow with the company, do they have a specific path mapped out that they'd want you to follow, are they hoping for someone who will stick in the one position forever, etc - ask them, if you got the job, where they'd see your role in five years, because it depends more on them than you at the end of the day (it doesn't matter if you see yourself as CEO if they only see you in a junior role).

Re:Well, they're a good indicator of intelligence

[Anonymous Coward]

Same for 'why do you want to work here." "Because I need a paycheck?" Seriously? Look, we all laugh at Wally in the Dilbert strips, but I'm sure as hell not going to knowingly hire one.

You can delude yourself all you want, but ultimately people get jobs because they need the money. Most of us would love just to lounge around with no real responsibilities in life if we could, but we need to eat, clothe, and house ourselves. So hence, the job.

That isn't to say that some jobs are more fun or better/worse than others, but don't pretend that economic incentive isn't the primary reason why we look for jobs.

Re:Well, they're a good indicator of intelligence

[SecurityGuy]

The really neat thing about brain teasers or puzzles or the bizarre questions you sometimes encounter like "How many pigeons are there is Manhattan" is that they are a very good way to judge someone's unstructured problem solving ability. How someone approaches this kind of a problem is a good proxy for their ability to debug hard technical issues or their problem solving ability in general.

This is really not true. My response to questions like that improved dramatically when I read an article that explained questions way out of left field like that are intended to test your problem solving ability, so do your best to estimate an answer and explain your thought process. Reading that article didn't make me better at debugging hard technical issues, but made me dramatically better at handling off-the-wall interview questions nimbly. You're not measuring what you think you're measuring.

Re:Well, they're a good indicator of intelligence

[shentino]

The fact that you're willing to pay the guy you're hiring means "to get a paycheck" is implied.

The question is a test of your ability to think ahead.

And if it's an annoying question it's also a handy test of your ability to remain diplomatic and RESPECTFUL.

If I ask an irritating question and they blow their top, I've just saved myself from hiring a ticking time bomb ready to blow.

Re:Well, they're a good indicator of intelligence

[maple_shaft]

Wow, you really hate whiteboarding don't you? I am so completely the opposite of you that I must respect your opinion because I can't relate to it at all.

Whiteboarding conveys a few things, a high level of spatial intelligence required for diagramming and modeling complex problems visually. It also is an accessory to communication about a complex design or process that a group of colleagues or lay people wish to know more about. If a candidate is whiteboarding a process for me and he silently doodles on the board then that is a problem. You are supposed to talk through the problem primarily and cement your ideas in on the board so that everybody can see a visual summary of your explanation.

The fact that you despise it means that you completely fail to understand why somebody wants an engineer who can whiteboard. It is a sign of an individual who can communicate and discuss problems on multiple facets, while somebody who just wants to demonstrate skill by typing in a text editor tells me that this person doesn't care about communicating or discussing complex ideas, they just want to showcase their skill.

You may be extremely productive writing software or some such engineering activity, but you seem like an extremely low-level task oriented person and that is not what most companies want. We want critical thinkers who engage in higher level design, thought, and communication. They don't want cowboy coders. They don't want a lone wolf.

Re:Well, they're a good indicator of intelligence

[HornWumpus]

If it didn't suck a little they wouldn't have to pay you to do it.

How many job openings are there for 'hot chick pussy eater'?

That said many employers are forced to be assholes because they have employees that are assholes.

For every employer that times your bathroom breaks their are a dozen employees holding their shit on the drive in so they can crap on the clock.

Use the rule of threes. If you've had three employers in a row with the same problem, it's not their problem, it's yours. Works for girlfriends too.

Puzzles aren't to test programming skills

[joshamania]

The brainteasers are there to see if someone is smart. Could the employee escape from a paper bag if necessary? I'd say these puzzles are important for finding creative problem solving and would be just as useful if not more useful in a manufacturing/fabrication/maker type of job.

Think of that American drilling team that drilled the hole to free the Chilean miners. That engineer's rig wasn't meant to do what he did with it. Can't aim it? He aimed it with a hack. Hole's plugged? Fixed it with a hack? Don't have a 28" drill head for this rig? Let's hack one together in a week. If that guy with the big brain didn't pick up the phone and say "hey"...those 33 guys would probably have been entombed for half a year if not forever.

Dude did it in one month with a toolbox full of hacks. Fucking brilliant.

Re:It's important to understand

[jythie]

Actually, I would wager they are giving brain teasers as an amusing way to cut down the number of applicants. I think programmers and hiring managers tend to like the questions because they are fun to give, but they are also a quick way to sort through people who look pretty similar on paper. When you have that kind of application volume, figuring out how to reduce it becomes pretty important for one's sanity, but like any other HR technique, it is designed to reduce the pile, not find the best applicants.

Re:Bring puzzles as an applicant

[w_dragon]

HR shouldn't be interviewing for technical positions beyond a basic initial interview to ensure that the candidate actually exists and wants to work for the company.