Slashdot Mirror
Are Brain Teasers Good Hiring Criteria?
theodp writes "Your brain teaser prowess may win you a job at Google, but the folks at 37signals don't hire programmers based on puzzles, API quizzes, math riddles, or other parlor tricks. 'The only reliable gauge I've found for future programmer success,' explains 37signals' David Heinemeier Hansson, 'is looking at real code they've written, talking through bigger picture issues, and, if all that is swell, trying them out for size.'"
Those of you who have hired employees: have you seen correlation between interview puzzle success and job competency? How should an interviewee best handle these questions?
If someone is giving you one, they're probably not very intelligent.
SJW: Someone who has run out of real oppression, and has to fake it.
Two employers offer you jobs. One asks you a teaser and expects a solution but lies, the other asks for experience and references and expects performance but tells the truth.
What do you ask the parrot that lies about the job the truthful parrot is offering?
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Bring the puzzles as an applicant to the interview, and ask the interviewer to take them. If the company puts someone who isn't even smart enough to do brain teasers in a position as important as interviewing and hiring, then the upper management probably isn't very intelligent either.
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
Every organisation has different needs, but even so "looking at real code they've written, talking through bigger picture issues" takes time, and an initial interview with more basic questions should probably be there to weed out the weakest candidates (unless the people in charge of recruitment interviews have nothing else to do and want to look busy, of course).
Needless to say unless you spend time puzzling over this specific type of problem you don't have the skills to answer them.
The impression I had was they were going through a dog and pony show of "trying to find a candidate" for their position. I am not sure what they were up to. Whatever it was, they weren't looking for a candidate for the advertised position.
There was an absolute reek of duplicity, insincerity and dishonesty about every single employee I met on that interview, starting with the prostitute-cum-receptionist who greeted me to the project manager who wouldn't look me in the eye to the interviewer who looked over my resume (which had only a distant physics class) and said "we're not going to ask you about programming, I can see you've got that down, we want you to solve some puzzles" and sprang on me some physics puzzles I could only solve if I were a physics major.
I couldn't wait to get out of there.
I saw that ad for a few more months online. I always wondered what they were up to.
Brain Teasers definitely test something.
If for example you are applying to a brain teaser solving/creation company then it would be ridiculous not to have to solve a few to get in.
If you are using one to test mental flexibility, well that can be as useful as being able to churn out well made and documented code, for the appropriate job.
Troll is not a replacement for I disagree.
The brainteasers are there to see if someone is smart. Could the employee escape from a paper bag if necessary? I'd say these puzzles are important for finding creative problem solving and would be just as useful if not more useful in a manufacturing/fabrication/maker type of job.
Think of that American drilling team that drilled the hole to free the Chilean miners. That engineer's rig wasn't meant to do what he did with it. Can't aim it? He aimed it with a hack. Hole's plugged? Fixed it with a hack? Don't have a 28" drill head for this rig? Let's hack one together in a week. If that guy with the big brain didn't pick up the phone and say "hey"...those 33 guys would probably have been entombed for half a year if not forever.
Dude did it in one month with a toolbox full of hacks. Fucking brilliant.
Google isn't giving brain teasers to find good programmers.
Google does *not* give brain teasers for engineering positions, and haven't been for the last 5+ years.
The WSJ article is based on urban legends, and *very* dated information.
The "next" big ideas??? NO. You failed the test. For google to look for the next big ideas, they first have to actually have some big ideas, which has not yet happened, and is not going to happen soon. I realize that most of the kids believe that making a good program requires only one big bright idea, in the time span of 5min, but the reality is that the "big" idea happens when you finished the project, when you are able to grasp all the cons and pros of your decisions, when you are actually able not only to make it work, and to make it fast, but actually "make it right". KISS bro, and keep watching "hacker" movies.
Anyone can write code, but not everyone has the ability to think outside of the box. Brain teasers are probably a great way to weed out those that aren't creative, provided you follow them up with questions showing they know how to do the job.
I serriously doubt Google doesn't follow up with relevant skill questions. Fail the brain teaser first; you save interviewers time, and leave no question to why you didn't get the job.
I actually wrote a blog post on this very subject this morning (I pushed up the publishing when I saw this). The post
In short, I disagree. I find brain teasers invaluable. But not in determining skill, but in determining personality and how a candidate behaves when they are faced with a challenge that they aren't familiar with...
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
Yep. There are few companies I know of that can afford their devs to not be creative though (perhaps healthcare and military being the exception since they are highly speced out systems up front usually). Customer has a problem and dev/support can figure out to do it with existing program but if they aren't smart enough to come up with a better workflow/software to fix that repeating customer problem more cleanly your program will continue to be a piece of crap for example. For small companies creativity is what they need because they don't have a dedicated designer/architect etc so they need someone that can say, oh wait why don't we completely flip the approach to the problem around and use hadoop to make this run faster or something. Lastly: it is rare that a company is hiring someone that they just want to be a dev they want them to grow and be more valuable, some day lead a team, a project, etc. A company owner usually is dreaming how big the company will get if the company grows 10X than some of the people that are working for you now need to be able to become managers/architects.
Actually, I would wager they are giving brain teasers as an amusing way to cut down the number of applicants. I think programmers and hiring managers tend to like the questions because they are fun to give, but they are also a quick way to sort through people who look pretty similar on paper. When you have that kind of application volume, figuring out how to reduce it becomes pretty important for one's sanity, but like any other HR technique, it is designed to reduce the pile, not find the best applicants.
I've never liked Brain Teasers. Every time I try one I keep thinking about how I would write a program to solve it.
I guess we combine the two approaches: we send our candidates small coding problems to solve. So we see real code they create and have a standardized way of comparing it to what other candidates have provided.
It works really well at filtering out people we don't want to waste time talking to, and gives us a starting point for the technical interview. It isn't useful for deciding whether or not a candidate should be hired, since there are many other factors that come into play.
But the idea isn't to get an answer - and I am very up front that I don't care about the answer, and I already know it anyway. What I do want to see is how someone approaches a problem that they don't know how to solve. I had one candidate ask me the answer, I already know it after all - immediately top of my hiring list, and she was an awesome hire. Another asked if they could use google on their phone - again a pretty much perfect answer. The puzzle is completely irrelevant, the ability to question, put forward ideas and not just say 'I don't know' or, even worse, go completely silent and get embarrassed that you don't know, is pretty fucking critical. IMHO.
I also look at samples of previous work, and we make all candidates carry out real world tasks along side us.
The best is the enemy of the good
I've been at my job 10 years now, and if I interviewed for it or a similar one now would expect (and do well at) a detailed technical interview. But it is in a very different area than what I studied in school- when I was being interviewed, we all knew that what the interviewers needed to discover was whether I could learn what I needed quickly and then apply it to designing new things. They already knew I didn't know it yet. I didn't even know Verilog (I do the digital side of mixed signal chips).
The best question was a quick lesson in how one of the main building blocks of many of our systems works, followed by questions about implications and what would happen if various broad changes were made with the architecture.
But the puzzle questions (usually requiring broad math and science knowledge, no one asked me elephant in the fridge type questions) were a good way to get at whether I had a broad knowledge base and could apply it to new things.
So they have their place, probably more for people crossing fields than those doing something they are experienced with.
When interview product support people, I've used some extremely hard puzzle questions as a test, but I'm never looking for a correct solution - I'm watching to for signs of freezing/panic when confronted with something unexpected, and I'm looking for the applicant to be able to: ask clarifying questions if needed, remain calm, and when presented with a portion of the answer, be able to apply it further. But that's partly because the job was support of a complex product with a large number of components that can interact in unexpected ways. Being blindsided by a customer question isn't uncommon, and being able to reason through the process and explain the steps as needed meant that the customer wouldn't have to call back a week later having gotten themselves into the same boat again.
Then they are even dumber than we expected. Seriously? Think about what you are saying. There is little correlation between brain teaser ability and intelligence. There is no correlation with creativity.
He best wy to find mart creative people is to engage candidate in unscripted conversation with smart creative people. I may be a chicken and eg problem, but goofy tests are not the answer. It is childish and lazy. Most highly intelligent or creative people would not even consider taking a childish test.
I know some of you are saying, "hey! I took a test like that for my job.". Just know, what I said is true. ;)
You have to ask whether google actually has any competent developer at all....
To quote Bob Fosse: I don't want people who want to dance; I want people who *need* to dance. That is what I look for during an interview, someone who clearly loves what they do and doesn't just sit around waiting for orders or just did whatever was told of them. I typically ask them about a project they were on, and if they get into the details, even if it's not exactly specific to programming but that they understand the "big picture", as well as their role in it, and look to see the eyes light up. It's especially Then I move on to the question that a lot of people don't expect, surprisingly, but is very telling: "What got you into programming?" Any flavor of "because it's really really cool" works; sadly a lot of responses are "it was either this or becoming a lawyer | dentist | whatever".
If you're hiring a sales guy, your interview should test his persistence, resilience, likability, and perhaps ability to hold his liquor.
If you're hiring a customer service rep, your interview should test their patience, politeness, and thoroughness at collecting information.
If you're hiring an engineer, solving puzzles is part of the job, brain teasers are one quick way to gauge how a potential hire will respond to the kind of task the job requires.
As for hiring HR staff, I'm not really sure how to judge them, other than the fact that any good person I've ever encountered in HR didn't stay in the job for long.
If you're using brain teasers as pass-fail criteria, then you're stupid. And if you're using them in an interview process that lasts less than an hour or two, you're even more stupid. They can be good for understanding a person's thought process during problem solving, but that's it. It's not the answer, but how they come up with it. That being said, the "how many gas stations in such-and-such city" where you have to pull estimates out of your ass are not good for choosing programmers, so don't even go there.
I used to work for a big company that you've probably heard of. When we interviewed people, our group had a brain teaser that we liked to use, probably because the answer (and there was a fixed, definite answer) was not the obvious one. And we got to draw pictures of it on the white board while asking it. But it was the programming test that really mattered to our group.
First of all, we had them do it on a white board in front of the group. (This was after all the individual interviews were done, and we had warmed up the group part of the interview with a brain teaser or two.) We weren't looking for getting API parameters in the right order, just that you could do the algorithm on the fly, and in a less than quiet environment. (typical cubicle farm level noise from us chatting to each other during this) We didn't even care what language they wanted to write in, the point was getting the algorithm right. And if they got something wrong, we would tell them how the output would be wrong, and let them fix it. Again, the goal was to see how they write code, and show us how, not that they could spit out the right thing from memory.
First was to implement strcpy. Any C programmer (our stuff was mostly C++) should at least understand how strings and pointers work to build something around *p++ = *s++ with a loop. So you probably got an off-by-one error, so what, we point it out, you fix it, but you at least got the basic idea right if you got even that far. Second was to write code to copy a file, since you should also be able to understand how to get data in and out of files. Then we would ask how to make the file copy faster, since most people would try the one-byte-at-a-time approach, and you ought to know about buffering, too. Finally, reverse a singly-linked list. This is something that any CS student should learn in their second year Data Structures class. Not to memorize it (because it's kind of pointless to memorize such a function), but to figure out how to do it from scratch. If your degree says "Computer Science" on it, you should be familiar enough with how to walk down linked lists to at least make a decent start on this one.
Well, guess what. The fresh out of college CS grads generally failed horribly, especially the ones that had been weaned on Java, where you don't have to deal with pointers like you do with C and C++. It was really stunning and even sad to see people fail at this. (The EE grads did much better, FWIW.)
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
Ive been on both sides of the table plenty and have both faced and given brain teasers. To say they are inherently good or bad hiring criteria is thinking of it the wrong way. Its just one tool in the toolbox. A hammer isnt inherently good or bad, you use it when its appropriate and not otherwise.
I personally put little stock in them, except that I love to get a wiseass answer because it shows personality. For example, I got hit with the Google-ish "how many golfballs fit in a schoolbus?" question once. My answer, almost immediate, was: "Come on, thats just silly, everyone knows golfballs do not ride the bus when they go to school... they don't need to, they're balls, they just roll!" The interviewer absolutely loved that answer.
To me, there's far better ways to evaluate a candidate. For a programming job its actually easy: give them a real-world task typical of the position, tell them they have as much time as they need, set them up at a workstation, show them where the bathroom and snack machine is and give them some space. See what they produce in that situation.
If a pion (n-) collides with a proton in the woods & noone is there to hear it, does lamdba decay into the source pa
You're information is only somewhat accurate. A typical interview path for Google consists of about 7 interviews during which you're interviewed by at least a couple different people and although there is no official stance on asking such questions there are most definitely interviewers that do.
Last interview I did they gave me a real-world problem to solve, put me in a room with a virtual machine and a proctor, and let me go to town. Deliverables included not just a working tool to solve the specified problem, but also documentation for same and the ability to explain what/why... and while the VM had unfettered Internet access, I had/have to assume that even when the proctor wasn't there in person they had a VNC client sharing what I was doing from the outer host or such.
From our discussion afterward, I gather that that exercise filtered quite a few folks who otherwise might have slipped through but couldn't actually transform requirements to code in a timely manner.
Yup, that's important too.
Over 30 years of hiring programming staff, what I have found is the converse of your proposition.
People who are not good at solving brain teasers are poor at being good programmers. If they are good at solving brain teasers, that really doesn't say anything about whether or not they will be effective as programmers.
Also, I'd be remiss if I didn't add a few words about "being effective." It's not ability to produce code that counts, it's ability to produce code that can be maintained later by other, less effective programmers. A good solid foundation is absolutely necessary, as is sufficient commentary so that someone who is stone cold on the code can dig into it and fix things, or change parameters. The long term cost of code is not creation of code, it is maintenance of code.
Looking at code is an important hiring criterion, but it is also something that is simply and totally out of the ability of an HR person to achieve. Perhaps the idea of using brain teasers is simply because it is a screening process that can be carried out by HR.
Don't take life too seriously; it isn't permanent.
I was handed a stapler and a box of staples and asked to write detailed instructions that anyone could understand on how to load staples into the stapler at an interview. I imagine that they were looking for someone with good communication skills. That was my first job out of college desktop support.
There's a technical ladder? Anywhere I've ever worked, it's more like a stool - start a decent distance off the floor, then go nowhere.
That's a function of the size of the company and industry you are in. In general, the technical ladder (or stool) becomes steep very quickly. And as you climb it up, you start to see that you do more management than actual hands-on thingie-building. But do not delude yourself into thinking that this type of management is of a non-technical nature.
Software Architect. Enterprise Architect. Technical Lead. Principal Engineer. Technical Director. Chief Scientist. Let's call these upper-stool technical positions.
These types of positions require you to do less hands-on stuff, but the management you will have to do must be technical-oriented. How you assign technical tasks to people and teams will depend on whether tasks are technical feasible, on identifying the technical capabilities of your team, on understanding the resources required to complete technical tasks.
Granted that a lot of people who get into these positions let go of themselves, gradually detaching themselves from the technical realities on the ground, where the pedal hits the metal. And as a result, their decisions are no longer technical, with technical consequences that is beyond their grasp. But those are examples of doing a bad job in their positions. And that exists at all levels, from the uber-chief of technical reality down to the lowest code monkey.
These are the fabled paper tigers.
That is, being detached of technical realities is not an inevitability of working so high up the ladder/stool. Good technical people remain strategically and tactically technical always, regardless of their pecking order. A good above-the-clouds architect can drop back to code with only a few days to clear the mental cobwebs. A good technical foot soldier can extrapolate the reasons behind good high-level technical decisions, even if he/she does not have the management experience (which naturally they don't at their entry level of their careers.)
My suggestion to people who find themselves staring at the technical stool: put another stool over it, secure it with nails, crazy glue or some other good shit, and then climb it. That is, like a good engineer, you need to engineer and build your technical ladder.
This can only be done without realizing first that to climb it, you will have to gradually move away from hands-on work without losing your technical wits. You cannot allow yourself to become a paper-tiger.
This will also means that when you find yourself at a company where there is nowhere else to go but down (because the stool cannot go any higher for whatever reasons), then it is time to go somewhere else where there is a chance to nail/glue another stool over the one you have built so far.
Anyone who can clearly explain a topic in English probably won't write readable code either.
Is this a clever dig at native English speaking programmers?
(which 7 variables do you need to track in your mind at any one time)
After 15 years of business programming, a smattering of embedded and OS programming, and a lot of algorithm programming competitions, I can't imagine a kind of programming where "which 7 variables do you need to track in your mind" is of any relevance. If this normally comes up for you, you're doing something very wrong.
Let's not stir that bag of worms...
I once applied as a programmer to work on a server infrastructure for a next-generation search engine. They were looking in particular for people with great expertise in the C++ language and in the Boost libraries, areas in which I was a very good candidate.
They asked me to perform a task and send the result by email before meeting me in person.
The task was to write a program that would take an integer n, and display the nth integer that satisfies a particular condition involving primes (I have forgotten what the exact condition was). I was told I would be judged on the performance on my program.
It was obvious that what they wanted was for me to know the mathematics about primes so that I would know the right formulae to compute the nth value quickly. As I didn't know them, it was irrelevant to the job I was applying for, and I didn't want to spend time researching it on the Internet, I chose to fit their requirements differently.
I computed all the values beforehand, and simply made the program return the nth value of a table. Technically, it fitted the specifications they had given me exactly, and was the fastest solution possible.
Yet they chose not to make me go to the next stage.
Looks like brain teasers don't like being beaten at their own game...
(Another funny thing about this event is that I sent the code to the person as a tarball, and he was unable to open it and asked me to send him a zip instead.)
I never give candidates puzzles to solve during an interview. Why? Because they're not 10 years old; they're adults.