Slashdot Mirror
Are Brain Teasers Good Hiring Criteria?
theodp writes "Your brain teaser prowess may win you a job at Google, but the folks at 37signals don't hire programmers based on puzzles, API quizzes, math riddles, or other parlor tricks. 'The only reliable gauge I've found for future programmer success,' explains 37signals' David Heinemeier Hansson, 'is looking at real code they've written, talking through bigger picture issues, and, if all that is swell, trying them out for size.'"
Those of you who have hired employees: have you seen correlation between interview puzzle success and job competency? How should an interviewee best handle these questions?
If someone is giving you one, they're probably not very intelligent.
SJW: Someone who has run out of real oppression, and has to fake it.
Once again 37signals cuts through the bullshit. Brain teasers during interviews are an HR fad. Show the code.
Two employers offer you jobs. One asks you a teaser and expects a solution but lies, the other asks for experience and references and expects performance but tells the truth.
What do you ask the parrot that lies about the job the truthful parrot is offering?
When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
Google isn't giving brain teasers to find good programmers. They're giving brain teasers to find creative technical people who can come up with the next big ideas.
Bring the puzzles as an applicant to the interview, and ask the interviewer to take them. If the company puts someone who isn't even smart enough to do brain teasers in a position as important as interviewing and hiring, then the upper management probably isn't very intelligent either.
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
How should an interviewee best handle these questions?
Soul sucking corporate culture ahead. Run away before they hire you.
As long as the applicant's ego is substantially smaller than DHH's, then you've got a shot.
Every organisation has different needs, but even so "looking at real code they've written, talking through bigger picture issues" takes time, and an initial interview with more basic questions should probably be there to weed out the weakest candidates (unless the people in charge of recruitment interviews have nothing else to do and want to look busy, of course).
Needless to say unless you spend time puzzling over this specific type of problem you don't have the skills to answer them.
The impression I had was they were going through a dog and pony show of "trying to find a candidate" for their position. I am not sure what they were up to. Whatever it was, they weren't looking for a candidate for the advertised position.
There was an absolute reek of duplicity, insincerity and dishonesty about every single employee I met on that interview, starting with the prostitute-cum-receptionist who greeted me to the project manager who wouldn't look me in the eye to the interviewer who looked over my resume (which had only a distant physics class) and said "we're not going to ask you about programming, I can see you've got that down, we want you to solve some puzzles" and sprang on me some physics puzzles I could only solve if I were a physics major.
I couldn't wait to get out of there.
I saw that ad for a few more months online. I always wondered what they were up to.
Brain Teasers definitely test something.
If for example you are applying to a brain teaser solving/creation company then it would be ridiculous not to have to solve a few to get in.
If you are using one to test mental flexibility, well that can be as useful as being able to churn out well made and documented code, for the appropriate job.
Troll is not a replacement for I disagree.
The last time I was tasered it'd hit me in the ass and it didn't make me smarter...
The brainteasers are there to see if someone is smart. Could the employee escape from a paper bag if necessary? I'd say these puzzles are important for finding creative problem solving and would be just as useful if not more useful in a manufacturing/fabrication/maker type of job.
Think of that American drilling team that drilled the hole to free the Chilean miners. That engineer's rig wasn't meant to do what he did with it. Can't aim it? He aimed it with a hack. Hole's plugged? Fixed it with a hack? Don't have a 28" drill head for this rig? Let's hack one together in a week. If that guy with the big brain didn't pick up the phone and say "hey"...those 33 guys would probably have been entombed for half a year if not forever.
Dude did it in one month with a toolbox full of hacks. Fucking brilliant.
Anyone can write code, but not everyone has the ability to think outside of the box. Brain teasers are probably a great way to weed out those that aren't creative, provided you follow them up with questions showing they know how to do the job.
I serriously doubt Google doesn't follow up with relevant skill questions. Fail the brain teaser first; you save interviewers time, and leave no question to why you didn't get the job.
I've heard in several places that Microsoft used to ask such questions (during the 90s,) but stopped using them after it became obvious that they didn't identify more-qualified candidates.
Interesting that Google has allegedly picked up the practice. (Didn't they use to ask more programming/theory questions?)
I actually wrote a blog post on this very subject this morning (I pushed up the publishing when I saw this). The post
In short, I disagree. I find brain teasers invaluable. But not in determining skill, but in determining personality and how a candidate behaves when they are faced with a challenge that they aren't familiar with...
If a man isn't willing to take some risk for his opinions, either his opinions are no good or he's no good
Repace high school diploma with BA or CS BA (aka no tech schools) and you end up with discrimination
The real issues are with jobs that say need a 4 year BA for jobs like mail room.
As for IT jobs bypass people who went to a tech school (that are a better fit for people with learning disabilities) or who have done alot of learning on other jobs / on there own can be seen as violating the law.
Not only that I have seen job ads that ask for -Minimum ACT / SAT Scores and -Minimum GPA 3.0 Now that sounds like even more of away to passover people who have learning disability's who may not be good test takes but can do a job
community college is also a other place that for some people who have learning disability's Is a better fit then other colleges but most of them MAX out at 2 years.
http://www.businessinsurance.com/article/20111206/NEWS07/111209935
“Under the ADA, a qualification, standard, test or other selection criterion, such as a high school diploma requirement, that screens out an individual or class of individuals on the basis of disability must be job-related for the position in question and consistent with business necessityThus, if an employer adopts a high school diploma for a job, and that requirement ‘screens out' an individual who is unable to graduate because of a learning disability, that meets the ADA's definition of ‘disability.'”
The letter, written by EEOC attorney adviser Aaron Konopasky, goes on to say, “Even if the diploma requirement is job-related and consistent with business necessity, the employer may still have to determine whether a particular applicant whose learning disability prevents him from meeting it can perform the essential functions of the job with or without a reasonable accommodation.”
JIM COLLISON
"Employers should not fear the EEOC warning. In fact, employers should use it to focus their attention on identifying the actual essential qualifications needed to perform a job...and how to assess whether or not a candidate has these qualifications. Because education has been so dumb-downed in the last 50 years, a high school graduation diploma or a high school equivalency certification simply is not evidence that an individual possesses the essential qualifications to perform a job. The same is true for many if not most post high school degrees. Check out the new book "Academically Adrift: Limited Learning on College Campuses" by Richard Arum and Josipa Roksa. Also check out the new Skills Gap research report from A.C.T. showing that just having a diploma or certificate is no evidence an applicant possesses the foundational skills of reading for information, locating information, and applied math needed for almost every job today. Jim Collison, President, Employers of America, Inc."
Perhaps Google is one of the companies who invest a lot of money into their recruiting process?
Perhaps Google is famous for analysing all available data, including backtesting their hiring decisions, to design their hiring practice?
Perhaps Google is looking for someone who is able to solve problems, and demonstrate they are able to think logically, structure their steps, and reach conclusions?
Perhaps Google is not just looking for programmers who efficiently produce code, but architects who do well in situations that are new to them?
Tell the interviewer how elite you are, or the number of boxes you've p0wned
I've never liked Brain Teasers. Every time I try one I keep thinking about how I would write a program to solve it.
I guess we combine the two approaches: we send our candidates small coding problems to solve. So we see real code they create and have a standardized way of comparing it to what other candidates have provided.
It works really well at filtering out people we don't want to waste time talking to, and gives us a starting point for the technical interview. It isn't useful for deciding whether or not a candidate should be hired, since there are many other factors that come into play.
Can you last 5 minutes in the Octagon?
Where I work, we give candidates a quiz, but it contains questions that we really do expect a competent candidate to answer.
Stuff like "how many conductors are there in a Cat 5 UTP cable". We have had applicants for Network Engineer posts that cannot answer this question (not just fluff it, literally cannot answer it).
Even then, we take it as another data point: something to consider alongside how they performed in a face-to-face interview. Someone who scores highly *might* be good, but then again, they might know someone who has already answered the quiz; someone who scores low might be poor, but equally might have been suffering from interview nerves.
But the idea isn't to get an answer - and I am very up front that I don't care about the answer, and I already know it anyway. What I do want to see is how someone approaches a problem that they don't know how to solve. I had one candidate ask me the answer, I already know it after all - immediately top of my hiring list, and she was an awesome hire. Another asked if they could use google on their phone - again a pretty much perfect answer. The puzzle is completely irrelevant, the ability to question, put forward ideas and not just say 'I don't know' or, even worse, go completely silent and get embarrassed that you don't know, is pretty fucking critical. IMHO.
I also look at samples of previous work, and we make all candidates carry out real world tasks along side us.
The best is the enemy of the good
Most real-world problems are not brain teasers.
I've been at my job 10 years now, and if I interviewed for it or a similar one now would expect (and do well at) a detailed technical interview. But it is in a very different area than what I studied in school- when I was being interviewed, we all knew that what the interviewers needed to discover was whether I could learn what I needed quickly and then apply it to designing new things. They already knew I didn't know it yet. I didn't even know Verilog (I do the digital side of mixed signal chips).
The best question was a quick lesson in how one of the main building blocks of many of our systems works, followed by questions about implications and what would happen if various broad changes were made with the architecture.
But the puzzle questions (usually requiring broad math and science knowledge, no one asked me elephant in the fridge type questions) were a good way to get at whether I had a broad knowledge base and could apply it to new things.
So they have their place, probably more for people crossing fields than those doing something they are experienced with.
A few weeks ago in a tech iv (phone) with a "vice president of software engineering" I was asked "How would you find the middle of a linked list, using the most efficient method." It was for a web development position, for a small company who does medical instruments. I must admit from the beginning the guy was arrogant as hell, dropping his title on me no less then 3 times in the first minute, but i tried my best to stay focused and keep iv. My answer was "Why would i ever use a linked list? in 15 years of software development, hardware integration, mixed with web apps, etc. I have never used a linked list for anything" I have heard about linked list in university, many years ago, and did some stuff with them in a class once, but never in industry. I called the shop back and told them i wouldnt take the contract if they did offer me it, a IV is a screening process for the interviewee as well..sometime u just know it will be hell working for someone like the arrogant ass.
anyone else use linked lists on a regular basis?
#include bier;
When interview product support people, I've used some extremely hard puzzle questions as a test, but I'm never looking for a correct solution - I'm watching to for signs of freezing/panic when confronted with something unexpected, and I'm looking for the applicant to be able to: ask clarifying questions if needed, remain calm, and when presented with a portion of the answer, be able to apply it further. But that's partly because the job was support of a complex product with a large number of components that can interact in unexpected ways. Being blindsided by a customer question isn't uncommon, and being able to reason through the process and explain the steps as needed meant that the customer wouldn't have to call back a week later having gotten themselves into the same boat again.
Brain teasers are just another fad. In fact, if the interviewer is asking you about your resume/experience then there's another HR fail. The interview is the corporation's one chance to judge your character and personality, e.g., chemistry. Your resume and references show you're a hot shot programmer. Are you also a psychopath who will reek havoc and destruction through the organization? Most HR organizations and hiring managers don't get that and simply spend the interview rehashing your resume. When they do that, the candidate is in control.
Perhaps. The article says "looking at real code" is better. Again perhaps. For example the problem there is: did they really write the code, if so how long did it take? Did someone else suggest fixes etc? You don't know. I mean 300 lines of beautiful C is all fine and dandy but if it took you 3 months to write it and half of it is cut and pasted from the web how good is it really?
They recognized the beautiful code is a sea of crap that is the Internet, added to it, and made it beautiful. Would I hire someone who could spot that useful code, even if they didn't write it, and could add to it and still make it beautiful? You mean, would I hire Steve Jobs?
I8-D
I took a class in organizational psychology back in college. Once of the sections was on best hiring practices. From what I remember, the best correlations to job performance were:(in order from best to worst)
1. aptitude tests (can you learn the required skills)
2. work examples (do you know the required skills)
3. Structured interviews (same questions given to each candidate)
4. Unstructured interviews (on the fly questions)
5. Resume/ CV
6. Personality test
7. Drug test
8. Honesty test
The last two had very poor correlation to workplace performance. For the best results, use the top 3.
One of our competitors trademarked the term "hypothesis". From now on, we will call them "boneheaded ideas".
if administered properly. As someone who has done hiring, my process is as follows: "Meet and Greet" - sit down and chat for a bit, see if I (and the people I work with) can stand to be around the person and have a reasonable conversation with them. If you don't have basic communication skills and can't carry on a basic conversation, you won't last here. Even in software dev, you need to be able to interact with the outside world in a meaningful way. So we weed those out first. "Show me the code" - show me some code that you've written. Bring me some examples of what you're capable of. I'm going to ask you questions about the logic flow, reasons why you laid things out the way you did, how long it took to write, etc. "Write me some code" - I'm going to give you a task that is related to what we do at this company (not some meaningless, trivial code exercise, but something that you will experience working here) and a timeframe, then review what you write. I will again ask about logic flow, code layout. If you don't finish, that's ok. But give me a good reason why. If it's something you were unfamiliar with, show me the steps you took to get up to speed. I don't honestly care whether or not you know how to use every arcane little function of a language, I care about how well you can get the resources you need, learn on the fly and adapt to new situations. Lastly, "Brain Teasers" - yeah, I use them. Not cause I care about how creative you are or cause their trendy and cool, but because I want to watch you think. I want to see how you handle pressure. Best answer I ever got was a guy told me he didn't know much about the topic of the teaser (it was engineering related), but he had a friend who was a engineering professor that he would talk to about it, but the consult with his friend would cost $500 and could he bill that to me? I laughed my ass of and hired the guy on the spot. Because, on top of all the skills he had and the ability to think on his feet and learn new things, he had some balls. That's how I do it. And it works really well for me, but I know I'm not the norm. Most companies would never allow a process like this. It's to bad though, we've had a lot of success this way.
To quote Bob Fosse: I don't want people who want to dance; I want people who *need* to dance. That is what I look for during an interview, someone who clearly loves what they do and doesn't just sit around waiting for orders or just did whatever was told of them. I typically ask them about a project they were on, and if they get into the details, even if it's not exactly specific to programming but that they understand the "big picture", as well as their role in it, and look to see the eyes light up. It's especially Then I move on to the question that a lot of people don't expect, surprisingly, but is very telling: "What got you into programming?" Any flavor of "because it's really really cool" works; sadly a lot of responses are "it was either this or becoming a lawyer | dentist | whatever".
Stop repeating that myth. There were none in my interviews, and after being hired and attending interview training, we're explicitly asked not to use them.
If you're hiring a sales guy, your interview should test his persistence, resilience, likability, and perhaps ability to hold his liquor.
If you're hiring a customer service rep, your interview should test their patience, politeness, and thoroughness at collecting information.
If you're hiring an engineer, solving puzzles is part of the job, brain teasers are one quick way to gauge how a potential hire will respond to the kind of task the job requires.
As for hiring HR staff, I'm not really sure how to judge them, other than the fact that any good person I've ever encountered in HR didn't stay in the job for long.
There are many kinds of intelligence and any 'brain teaser' plays upon elements which may have no bearing on the job being sought. If any questions are asked during an interview, or brain teasers given, then those should have a direct relationship to the subject matter required for the position. Anything other than that is going to give a biased result and is probably closer to drawing straws than to real science. Having a greater amount of one kind of intelligence or creativity may give an edge to the person being interviewed but it is not a direct bearing on competency for most job positions. If you are hiring based on creativity you should test one way, and mathematical skills for another. No singular test is perfect, but it should best be tailored towards the qualities required to fill that specific role. Test for what is important, as someone with a 250 IQ may not stay in that secretarial position for very long, even though they got all the answers correct on the test.
He keeps submitting complete NONSENSE summaries about INANE webpages. We might be looking at 10 per week or so. And of course, the fool, Soulskill, always lets a few through.
Slashdot, Slashdot--what has become of you? Does anybody else in here feel the way I do?
Some sunny day...
If you're using brain teasers as pass-fail criteria, then you're stupid. And if you're using them in an interview process that lasts less than an hour or two, you're even more stupid. They can be good for understanding a person's thought process during problem solving, but that's it. It's not the answer, but how they come up with it. That being said, the "how many gas stations in such-and-such city" where you have to pull estimates out of your ass are not good for choosing programmers, so don't even go there.
I used to work for a big company that you've probably heard of. When we interviewed people, our group had a brain teaser that we liked to use, probably because the answer (and there was a fixed, definite answer) was not the obvious one. And we got to draw pictures of it on the white board while asking it. But it was the programming test that really mattered to our group.
First of all, we had them do it on a white board in front of the group. (This was after all the individual interviews were done, and we had warmed up the group part of the interview with a brain teaser or two.) We weren't looking for getting API parameters in the right order, just that you could do the algorithm on the fly, and in a less than quiet environment. (typical cubicle farm level noise from us chatting to each other during this) We didn't even care what language they wanted to write in, the point was getting the algorithm right. And if they got something wrong, we would tell them how the output would be wrong, and let them fix it. Again, the goal was to see how they write code, and show us how, not that they could spit out the right thing from memory.
First was to implement strcpy. Any C programmer (our stuff was mostly C++) should at least understand how strings and pointers work to build something around *p++ = *s++ with a loop. So you probably got an off-by-one error, so what, we point it out, you fix it, but you at least got the basic idea right if you got even that far. Second was to write code to copy a file, since you should also be able to understand how to get data in and out of files. Then we would ask how to make the file copy faster, since most people would try the one-byte-at-a-time approach, and you ought to know about buffering, too. Finally, reverse a singly-linked list. This is something that any CS student should learn in their second year Data Structures class. Not to memorize it (because it's kind of pointless to memorize such a function), but to figure out how to do it from scratch. If your degree says "Computer Science" on it, you should be familiar enough with how to walk down linked lists to at least make a decent start on this one.
Well, guess what. The fresh out of college CS grads generally failed horribly, especially the ones that had been weaned on Java, where you don't have to deal with pointers like you do with C and C++. It was really stunning and even sad to see people fail at this. (The EE grads did much better, FWIW.)
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
Ive been on both sides of the table plenty and have both faced and given brain teasers. To say they are inherently good or bad hiring criteria is thinking of it the wrong way. Its just one tool in the toolbox. A hammer isnt inherently good or bad, you use it when its appropriate and not otherwise.
I personally put little stock in them, except that I love to get a wiseass answer because it shows personality. For example, I got hit with the Google-ish "how many golfballs fit in a schoolbus?" question once. My answer, almost immediate, was: "Come on, thats just silly, everyone knows golfballs do not ride the bus when they go to school... they don't need to, they're balls, they just roll!" The interviewer absolutely loved that answer.
To me, there's far better ways to evaluate a candidate. For a programming job its actually easy: give them a real-world task typical of the position, tell them they have as much time as they need, set them up at a workstation, show them where the bathroom and snack machine is and give them some space. See what they produce in that situation.
If a pion (n-) collides with a proton in the woods & noone is there to hear it, does lamdba decay into the source pa
It's not an exercise in writing efficient code. It's an exercise in managing attention span (which 7 variables do you need to track in your mind at any one time) and in communication. The first CAN be tested with the right puzzles. The key is RIGHT puzzles. The second would be best tested by having programmers write an essay. Anyone who can clearly explain a topic in English probably won't write readable code either. And since readability of the code is 70% of its value... well, you do the math.
Any guest worker system is indistinguishable from indentured servitude.
I think if they were done over the phone it would be better. I find people either let you have at it, or interrupt/cut you off/presume you are thinking the wrong thing based on whether they "like" you. I've an IQ 150 but look kind of unintelligent. Quite often I am not able to get a word in edge wise with these sorts of interviews but I have gotten offers from phone based ones. I don't really have any interest in tap dancing for Amazon or Google anymore, I get my jobs through referrals or people I've worked with. I am too old to pretend to be patient with people I can tell are less intelligent than me interrupting me to say something inane like "--ah but you are forgetting prime numbers!"
The problem with puzzles and similar selection methods is they don't select for the job. Sure, we can rationalize they test for intelligence, creativity, and functioning under pressure, but the endless research in personnel selection indicates they are just not very good at predicting which candidates will be good employees and which will be bad employees.
The only two pragmatic indicators are:
1. Education - a relevant degree from a respectable institution.
2. Job samples - the candidate is evaluated on work similar to the one on the job s/he's being selected for.
I hire embedded programmers. If they graduate from a university and can't apply basic algebra principles, that is a pretty good indicator that they cut and paste and memorize. I don't ask them puzzles, but I do ask them to take a real-world phenomenon and turn it into an equation. If, knowing its (constant) speed, starting time, and starting position, you can't express the position of a train as a function of time, you can't program either.
Some people call those "puzzles"
We would use "puzzles" or situations in order to see a person's methodology. We wanted to see how they would troubleshoot and think through a problem. We weren't interested in their answer, just how they can to that answer. We even told them as much and would help guide the along with feedback. This was a critical test, because we worked as a team an had planning sessions like this all the time (just not putting one person on the spot). The types of "puzzles" we would set up were more real world issues.
Every piece of code I've written professionally is the property of the company I wrote it for and covered by NDA's (or worse). I could show some Turbo Pascal MSDOS programs I wrote in school but that's hardly representative of my current abilities.
Support Right To Repair Legislation.
Reason is, they can be practiced. I did logic problems as a hobby in high school (specifically cross hatch grids) and I've always loved working on brain teasers. Back when the GRE used logic problems instead of writing for testing analytical skills, I scored 790/800. When I applied for a few jobs that used the standard logic problem test, I shocked my interviewer by not only finishing the problem set, but getting almost all of them right.
Does this make me a good programmer? Of course not. I'm just a beginner. Does this make me a good employee? Well, I like to think I'm a good employee and I certainly hope my current boss agrees, but problem solving is only a small part of what I do (whether it's debugging random NTFS errors appearing on a backup server or debugging the company coffee pot.)
Occasionally living proof of the Ballmer peak.
I took some advice from Joel on Software and I ask a candidate to psuedo-code the Fibonacci sequence using recursion. It shows me if they can think through a problem. The recursion tells me about their background in programming and whether they can think abstractly while holding multiple levels of a problem in their head. And it shows me a little bit of programming skill.
And I don't care if they get everything 100% correct.
Is it a puzzle? Not sure if it falls under that category but it seems more pertinent than the usual puzzles.
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
When I hire (web developers, specifically), I want to see code they've done, sure. But I'd rather see them write code live (yes, write). Simple live coding exercises only demonstrate how they approach simple problems, but they can also reveal good/bad programming habits.
Apart from that, I don't really do brain teasers, but I do ask questions that give me insight into creative thinking, problem solving, and hindsight/foresight.
There are a lot of textbook programmers that are pretty much useless in the real world, and there are a lot of "creative" coders that are so "out of the box" that they're impractical cowboys. A combination of live coding and conversation gives me more insight than a full library of supposedly authored code.
Basically, I draw a lot of inspiration from this old blog post I ran across a while back.
Over 30 years of hiring programming staff, what I have found is the converse of your proposition.
People who are not good at solving brain teasers are poor at being good programmers. If they are good at solving brain teasers, that really doesn't say anything about whether or not they will be effective as programmers.
Also, I'd be remiss if I didn't add a few words about "being effective." It's not ability to produce code that counts, it's ability to produce code that can be maintained later by other, less effective programmers. A good solid foundation is absolutely necessary, as is sufficient commentary so that someone who is stone cold on the code can dig into it and fix things, or change parameters. The long term cost of code is not creation of code, it is maintenance of code.
Looking at code is an important hiring criterion, but it is also something that is simply and totally out of the ability of an HR person to achieve. Perhaps the idea of using brain teasers is simply because it is a screening process that can be carried out by HR.
Don't take life too seriously; it isn't permanent.
What I do when hiring is to give applicants a small homework problem, based on this essay.
The problem involves a small amount of 'design recursion'. I have them submit pseudo code so as to not obsess on syntax.
I find it gives me good insight as to their thought processes, design skills, attention to detail and craftsmanship.
Replace high school diploma with BA or CS BA (aka no tech schools) and you end up with discrimination
personality tests are determination as well.
The real issues are with jobs that say need a 4 year BA for jobs like mail room.
As for IT jobs bypass people who went to a tech school (that are a better fit for people with learning disabilities) or who have done alot of learning on other jobs / on there own can be seen as violating the law.
Not only that I have seen job ads that ask for -Minimum ACT / SAT Scores and -Minimum GPA 3.0 Now that sounds like even more of away to passover people who have learning disability's who may not be good test takes but can do a job
community college is also a other place that for some people who have learning disability's Is a better fit then other colleges but most of them MAX out at 2 years.
http://www.businessinsurance.com/article/20111206/NEWS07/111209935
“Under the ADA, a qualification, standard, test or other selection criterion, such as a high school diploma requirement, that screens out an individual or class of individuals on the basis of disability must be job-related for the position in question and consistent with business necessityThus, if an employer adopts a high school diploma for a job, and that requirement ‘screens out' an individual who is unable to graduate because of a learning disability, that meets the ADA's definition of ‘disability.'”
The letter, written by EEOC attorney adviser Aaron Konopasky, goes on to say, “Even if the diploma requirement is job-related and consistent with business necessity, the employer may still have to determine whether a particular applicant whose learning disability prevents him from meeting it can perform the essential functions of the job with or without a reasonable accommodation.”
JIM COLLISON
"Employers should not fear the EEOC warning. In fact, employers should use it to focus their attention on identifying the actual essential qualifications needed to perform a job...and how to assess whether or not a candidate has these qualifications. Because education has been so dumb-downed in the last 50 years, a high school graduation diploma or a high school equivalency certification simply is not evidence that an individual possesses the essential qualifications to perform a job. The same is true for many if not most post high school degrees. Check out the new book "Academically Adrift: Limited Learning on College Campuses" by Richard Arum and Josipa Roksa. Also check out the new Skills Gap research report from A.C.T. showing that just having a diploma or certificate is no evidence an applicant possesses the foundational skills of reading for information, locating information, and applied math needed for almost every job today. Jim Collison, President, Employers of America, Inc."
There are two problems with the use of "Brain Teasers" as interview questions;
1. Interviewers don't know what to look for when using them. A brain teaser is less about getting the correct answer than it is the reaction of the person you give it two. Do they think about the question or give up immediately? What is their thought process while trying to solve the problem? If they jump to an immediate conclusion and get the right answer, they've probably seen your brain teaser before. That doesn't really tell you anything about them.
2. If you do know what to look for, you're probably going to filter out most of your candidates. There isn't a shortage of workers right now, but there's still a shortage of the guys you want working at your company.
Funnily enough, how they use their brain teaser tells you almost as much about them as it tells them about you. If you can tell they're actually looking for a correct answer, you'll realize that the abilities of your co-workers is going to be as hit-or-miss as any other company. If they let anyone on the team you're trying to get on talk to you, hit them with some questions about how they feel about their company's execution, and watch their reaction closely. If you get anything resembling a resigned sigh, you might want to pass (Or at least bump your asking salary by 20K or so.)
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I interviewed online with Google in 2007. No puzzles, etc. -- they went right into watching me code. Another large company, local to my hometown, actually had a good-sized math and problem-solving test I had to take before I was interviewed. Again, though, no puzzles. It was straight math, followed by a face-to-face interview/conversation that focused primarily on programming. All seemed very well organized to me. The smaller companies are the only ones I've had ask strange questions, but the "in 5 years" questions was the strangeness threshold there. That was my first time being asked a question like that. Of course, my answer was, "I have no idea. Texas, maybe, herding cattle..." Yes, I got the job.
expletives welcomed
There is apparently a long history of the use of aptitude tests in the selection of programmers. From a 1965 article in Datamation on programmer recruitment: "Creativity is a major attribute of technically oriented people," suggested one representative profile. "Look for those who like intellectual challenge rather than interpersonal relations or managerial decision-making. Look for the chess player, the solver of mathematical puzzles." There is a little piece from the "computer boys" history site above has some funny images from this period.
I used to use one of the puzzles out of "Cuckoo's Egg" by Cliff Stohl. If they solved it (which almost none ever did) that meant they were really good, or had read the book (indicating, to me, that they might be more than just a fly-by-night security guy). Not an instant hire, but an interesting data point.
If they couldn't solve it, then it was a good gauge of how they dealt with pressure. The "Shit, I'm interviewing and I can't figure out the puzzle" made some freeze up and fall over. That is useful information when you plan to put these people in front of clients, who will ask hard questions.
People who didn't get it, spent some time trying to work through it, then could articulate what they tried and why that didn't work had the qualities I was looking for: intelligence, diligence, and humility.
They knew I didn't have the precise skill set they were looking for and I was brutally honest about it in the interview.
However the puzzles were fun, or I thought so, but I did my thinking aloud. I realised they probably wanted to know how I 'thought'.
One puzzle was; you had 9 balls, one lighter than the others, a set of balance scales. Using two measurements find the lightest ball.
I got offered the job. I did have to keep a lot of thoughts to myself though ;)
I was handed a stapler and a box of staples and asked to write detailed instructions that anyone could understand on how to load staples into the stapler at an interview. I imagine that they were looking for someone with good communication skills. That was my first job out of college desktop support.
You need to ask the hiring manager, of course people interviewing who could not answer a puzzle and did not get the job would say it was a bad idea. Although not answering correctly is not a reason to not hire someone its not coming up with a logical approach to getting to an answer.
I ask all sorts of questions all relevant to the job. I ask simple algebra problems and some people get offended (these people typically can not answer algebra questions.) I also give pseudo programming tasks.
Frankly for programmers I have found that lots of people can be good at the mechanics of programming but the juxtaposition of programming, creative problem solving, quick learning and other skills are what makes a great hire. For example if they can't do complex math they won't be a very good financial programmer. If they can't solve problems creatively you will always be micromanaging them.
Showing me code is not worth much, how do you know it is actually their code and without knowledge of the program all I can tell is that it's neatly written not good at doing what it needs to do. Even if it is theirs and good, perhaps it took them two months to write it and it should have only taken a day.
I have found people who ask these questions are not worth your time. Every job that asked me this stuff was never a job I wanted for long because it was always not what they made it out to be and the person who asked this is usually an arrogant jerk who hasn't written to much code.
What I have found is this; with rare exceptions people who can answer these puzzles right off without even thinking about it, are smart, but the code they write is awful stuff. Even now going for my masters, the one "helper" for the theory class is great at this sort of thing, but when it came to presenting code on a previous class we took together they were all but thrown out of the room for non-working exception throwing code.
When I have interviewed I never asked this sort of thing, means nothing and is insulting IMO.
If I cannot ask you pointed questions related to what I am intending to hire you for I should not be interviewing. I place far more value on what someone has done in the past, work and job history.
On one recent occasion I terminated a interview when asked such questions. Then immediately hired by a competitor the next day for twice the rate.
Got Code?
have you seen correlation between interview puzzle success and job competency?
The point of brain teasers is not so much whether the candidate gets the right answer, it is getting them to show you how they work a problem that is outside their standard frame of reference. The worthwhile insight you can glean is whether they ask you to guide them or come up with their own approach to solve the problem.
Depending on the position, you may want either candidate. Some positions call for independent problem solving, others are better having a person who detects problems and raises them to the team before proceeding.
Stop-Prism.org: Opt Out of Surveillance
I'm entirely with them on that.
Puzzles test one thing: Your ability to solve puzzles. The assumption that this will translate into or is representative of any real-world skills is just that - an assumption. It may turn out to be true or not, but AFAIK there is no conclusive evidence pointing either way so far.
But - the dark secret is that this is how the business world works. Almost the entire "wisdom" of business is this kind of anecdotal evidence, cute ideas, mantras spoken by authority figures and crap written in long-discredited books. It's a miracle the whole thing works at all.
And yes, I am speaking from personal experience. I have seen companies being reorganized because the CEO (and those around him) believe that there can only be x viable companies in this market, so they need to enlarge market share because only the first x will survive. No evidence whatsoever, it's an "everyone knows that..." point. I've seen a company dead-set on cutting costs not because they weren't profitable, but because someone somewhere said that their cost-per-employee ratio was worse than that of their competitors. I've seen downsizings because the employee-to-customer ratio was higher than some "benchmark" value.
In this, like in many cases, someone somewhere made up a number that may or may not have made sense in his or her specific context. Then someone else who didn't quite get it took it and ran with it.
Same for puzzles and assessment centers - they serve a specific purpose in a specific context, and if you know how to use them, they can be great tools. Like with all tools, just because you have a hammer doesn't mean you need to hit everyone who comes through the door with it.
Assorted stuff I do sometimes: Lemuria.org
I've been interviewing technical staff for over 10 years. I'm well known for using brain teasers during the interview. It's not the deciding factor, but it does contribute to the overall evaluation.
The important distinction is in the value of the candidate answering the question. Am I interested in whether or not the candidate gets the answer? Not really. It's nice if they do, but they often don't. I'm more interested in their approach to the problem. In IT Engineering, you will be presented with a task that may seem impossible that you need to engineer around. How will you approach this problem? Will you become quickly frustrated and abandon it? Will you choose an illogical and/or proven-failed solution and stick with it (i.e., continue to pound that square peg into that round hole despite the fact that it just does not fit)? Or will you recognize that the seemingly desirable solution simply does not suit the need to the scenario, and realize that there must be another way? It is THESE thinkers (the last one) that make very valuable staff for IT Engineering.
Simply giving someone a brain teaser (particularly a "trick question" style) to, say, humble the candidate is idiotic. But if used wisely, certain puzzles are very helpful in gaining insight to the thought patterns of the candidate. I agree that it is not relevant to the programming and/or administrative experience of the candidate. It's a window into how they think, and how they may face certain situations.
It's not unlike the point of the "Kobayashi Maru", except the tests I tend to use do have a definite solution (other than reprogramming the simulator).
To put another spin on this, ask yourself what other occupations hire this way.
My sister is an ER nurse going on 10 years. None of her interviews have ever included a technical question. Ponder that for a moment. The person working in the ER, in charge of someone's LIFE wasn't asked a technical question. Same with my brother in law, the anesthesiologist. Never asked about drug interactions for pain medications, etc.
Now, obviously there is some level of trust in the hiring system with passing boards and certs. and the like, but no technical questions during the interview? What does this tell us about tech hiring?
I recently sat through an hour+ interview with around 15 or so puzzles and technical / API / language minutia questions. My experience prior to this was with a recruiter for the same company who:
1) continually called my home number during work hours and commented on not being able to reach me
2) was late showing up to meet me in the lobby, couldn't figure out the badge / security rules so he signed me in under someone else's name.
3) forgot what room the interview was being held in, called up another employee from the lobby and asked them to look at his Outlook Calendar.
I wonder what this guy's screening process was like.
Back to the interview...I suppose I performed in the middle of the pack and maybe answered 70-80% right, working through the solution, etc. However, in general, I do terribly in that setting...always have for some reason. It's not a matter of ability or creativity, I have a patent and have led successful projects that brought high-end mathematical software to the market.
The thing that stunned me is that no questions were asked about either of these things on my resume. Every question centered on some technical aspect, never on project results, never on prior experience, never on revenue I've personally generated for prior companies.
As an interviewee, I've faced a variety of interview methods, most of which are just too much for code monkey roles, but never the less one still gets thrown through the routine. Probably because a company/dept/interviewer has used said method for years and either feels it works or has no idea what will.
Personally I feel that there is no 'right' way to interview developers, because writing/debugging effective code is very much like skinning a cat, in that, there are many approaches to it. Naturally there are folks who try hard but are not really cut out for it.
The interviews I have most enjoyed are really just 'having a chat', the interviewer tries to get the measure of me and I can try and get a good feel for company I'm looking to spend my daylight hours in. Then if our two bits of the jigsaw appear to join, then there is the handy concept of a probation period just in case.
When all else fails, you've won.
The argument seems to be that these kind of questions help to identify problem solvers and people capable of taking on problems that they're (presumably) unfamiliar with. The company I work for uses them and I just can't help thinking we've lost some great potential hires and gained some awful hires due to the reliance on them. I feel like if you're going to use them you need to make sure they're not the pivotal piece in the interview process - too often these questions are the make or break tool for a hire and I think that this is the actual problem. Interviewing is hard for both sides, engagement is key, there are no magic bullets.
After you've responded to the puzzle, there should be another, that begins with: A guy walks into a bar... The creative mind is forced to make up a funny situation, or at the very least, a guy with good memory will remember how this joke goes.
WARNING: Smartphones have side effects--most of them undocumented.
As someone who's been on a few interviews and had to interview people, I really dislike any test that puts the candidate "on the spot" in a high pressure environment (such as a live interview). Unless, of course, that's going to be a feature of the role you're hiring him or her for. That's not usually the case for software developers. I would extend this past brain teasers to also include "sketch out some code on the whiteboard" type tasks, unless the code being sketched is fairly simple. I agree with the 37signals guy that looking at actual code is way more useful than whiteboard and brain teaser type tasks, not least of which because some candidates try to "game" the brain teasers and memorize the common ones before the interview.
If I had my way, I would cook up a programming tasks that exercises a few "tricky" aspects of coding. Maybe some concurrency, some processing of arbitrary input, etc. I'd plunk the candidate down in a room, by himself, with a computer, network connection and whatever IDE I would expect him to use in the role. If there is no expectation with respect to IDE then provide all the common options and let him use the one he prefers. Express the problem in written form, albeit with some ambiguity that requires him to ask for clarification (or make a judgment call). Structure the task so that it's easy to get it working correctly for typical inputs, but tricky to get it working correctly for pathological inputs. Possibly choose a task that can be partially solved by using common off-the-shelf libraries, and see if he is savvy enough to use them. Stipulate in the problem description that it's okay to make use of free software as needed. Grade him as follows:
1. Correctness. Does his solution work correctly for the common cases? Does it work correctly for the pathological cases?
2. Completeness. Did he ask for clarification on areas where the problem description was ambiguous? If not, was he at least aware of the ambiguity and able to defend the judgment call he made?
3. Did he attempt to reinvent the wheel, or did he have the breadth of knowledge to make use of existing libraries *where it made sense*? I wouldn't count it as a huge strike against someone if they were ignorant of the fact that a given library existed, but able to reproduce the same functionality via custom code.
4. Is the code "clean"?
5. Is his code performant?
6. Can he dialogue intelligently about his solution, why he chose to do X over Y, etc.
Another task I might include in interviews is debugging. Come up with some toy applications that have contain fairly subtle (but not hard to correct) bugs. Have the candidate find and correct them.
There's a technical ladder? Anywhere I've ever worked, it's more like a stool - start a decent distance off the floor, then go nowhere.
That's a function of the size of the company and industry you are in. In general, the technical ladder (or stool) becomes steep very quickly. And as you climb it up, you start to see that you do more management than actual hands-on thingie-building. But do not delude yourself into thinking that this type of management is of a non-technical nature.
Software Architect. Enterprise Architect. Technical Lead. Principal Engineer. Technical Director. Chief Scientist. Let's call these upper-stool technical positions.
These types of positions require you to do less hands-on stuff, but the management you will have to do must be technical-oriented. How you assign technical tasks to people and teams will depend on whether tasks are technical feasible, on identifying the technical capabilities of your team, on understanding the resources required to complete technical tasks.
Granted that a lot of people who get into these positions let go of themselves, gradually detaching themselves from the technical realities on the ground, where the pedal hits the metal. And as a result, their decisions are no longer technical, with technical consequences that is beyond their grasp. But those are examples of doing a bad job in their positions. And that exists at all levels, from the uber-chief of technical reality down to the lowest code monkey.
These are the fabled paper tigers.
That is, being detached of technical realities is not an inevitability of working so high up the ladder/stool. Good technical people remain strategically and tactically technical always, regardless of their pecking order. A good above-the-clouds architect can drop back to code with only a few days to clear the mental cobwebs. A good technical foot soldier can extrapolate the reasons behind good high-level technical decisions, even if he/she does not have the management experience (which naturally they don't at their entry level of their careers.)
My suggestion to people who find themselves staring at the technical stool: put another stool over it, secure it with nails, crazy glue or some other good shit, and then climb it. That is, like a good engineer, you need to engineer and build your technical ladder.
This can only be done without realizing first that to climb it, you will have to gradually move away from hands-on work without losing your technical wits. You cannot allow yourself to become a paper-tiger.
This will also means that when you find yourself at a company where there is nowhere else to go but down (because the stool cannot go any higher for whatever reasons), then it is time to go somewhere else where there is a chance to nail/glue another stool over the one you have built so far.
That's because 37 Signals is looking for hipster programmers, not geeks.
Look - the brain teaser questions really only do a few things in an interview and they're just one tool in an arsenal to determine if, as an interviewer, you have a good candidate on your hands.
First, it throws a question at an interviewee that s/he cannot possibly prepare and rehearse an answer for. As a result, you get to see how they think through (or not) something spontaneous. Also, sensing how the interviewee approaches the questions and, ultimately, how they answer can give a great amount of insight into how they might handle stressful situations.
Second, it opens the door of conversation to (possibly) something beyond the traditional job description, salary requirements, and hours discussion. It helps the interviewer form a more complete picture of a candidate. This can work well if you, as an interviewer, need a candidate to work well with an established team or if you want a specific personality type working for/with you.
However, as an interviewer, you need to ask the right kinds of brain teaser question to get the results you're looking for. Unless you've got a specific reason for asking them, and you'll usually want the true motive to be obscured from the interviewee, they really only hinder the interview process. The interview process is a bit of a dance and asking random brain teaser questions for the sake of themselves is akin to throwing some slam-dancing moves when the music calls for a slow walz.
In other words, have a reason for asking them. And if you're relying on brain teasers alone to make a hiring decision, you're doing it wrong.
My sources are unreliable, but their information is fascinating. -- Ashleigh Brilliant
Brain teasers are not good hiring criteria. However, when you encounter them in a job interview, that does not mean they (or your answer) are meant as such criteria. Good simple criteria for hiring people simply do not exist; people aren't themselves during job interview, they have probably prepared it and thereby skew the outcome.
Therefore everything that's said and done during a job interview, whether it is a discussion about your previous projects, some brain teasers or talking through some code the applicant wrote (or did not write:)), is irrelevant. It's all just bullshit to get the conversation going. Once this conversation has taken long enough for the potential employer to get a good gut-feeling about the applicant, the interview is done.
While some facts may be checked beforehand (references, education etc.), there's really not much more to go on for the employer than that gut-feeling. When brain teasers help to develop that gut-feeling, they can be a great tool for judging applicants.
However, when hiring people, I never use brain teasers. What I do is let them discuss a piece of code. I don't care whether they wrote it or not (or whether it is crap code I wrote myself:-)), I don't even care if it is good code or not and I don't care in what language it is. What I do care about is what the applicant has to tell about it. What would they change, what do they like or dislike, why did they choose this piece of code, which decisions do they _not_ understand etc. etc.
While hiring new people is always a gamble, simply discussing a piece of code engineer to engineer is probably the closest you can get to objectively judging an applicant.
0x or or snor perron?!
"Are Brain Teasers Good Hiring Criteria?" will be precisely the brain teaser that I put forth to my next prospective hire.*
(*Note: I have never hired anyone.)
We know where leadership by an anti-intellectual "strongman" who scapegoats minorities and likes boisterous rallies goes
Yeah i got that question and it really surprised me to be perfectly honest. It threw me for a loop. I knew about the job(3-D visualization, opengl/c++) and i told them that that work interested me and thats why i wanted to work for them. but then they asked me again. What was i supposed to say? Why do you think i want to work for you? Because i need a freaking job! Did they expect me to know everything about the company?
I'm not strictly interested whether or not they can find the answers in 5 minutes. I've asked questions even I didn't know the answers to, and for even the simple ones, smart people can draw a blank because they're tired and nervous from interviewing. I'm looking to determine if the candidate can reason at all and has good background knowledge. If someone tries to BS their way through, I don't like them, because this suggests that they don't care that much about reasoning and they aren't willing to be honest about their ignorance. If someone gets mad at themselves because they can't think clearly enough to work it out, I like them, because it's a hint to me that if they had the time to work on it with less pressure, they'd figure it out. (And there's a huge difference between interview pressure and deadline pressure.) I've had a few who were determined to get it, whom I stopped early because I learned what I needed to know, and I could see they were on the right track anyhow. (I explained that being able to answer the question right then and there was not the factor that was going to affect my hiring recommendation.)
Very often, I'd get a vibe about someone right when I first meet them. I would go through the interview process just to verify objectively that my intuition was right. And I did generally turn out to be right. Mind you, I also listened carefully to the the impressions of other interviewers, and sometimes reinterpreted what I had observed. I could spot someone with high intelligence and/or really good engineering skills, but on a few occasions, characteristics of maturity, persistance, and initiative got by me.
"Respectful" - management gibberish for "accepts the technical judgement of a boss who wouldn't know a computer from his ass, mostly parroting his 5 year old son". Like Obama, doesn't know how many states the US has (you'd think he'd look it up after getting it wrong once ...), but very eloquently saying exactly what he heard somewhere. And if an employee gives a respectful answer ... they're lying. Which may actually be the right character for dreary "just fucking do this easy thing" jobs.
Resulting in the -fabulous- situation that we all know : management hiring morons who lie to them so managers feel superior.
It really depends though. If you want a CRUD developer (because you've read in the newspaper how "superior" ruby on rails is to access/excel or foxpro for your tiny little 2-user application) the dumb questions to see if your employee will lie to make you feel good might actually be a good technique (because the only real factor is how compliant a developer is - any idiot will do). You want a developer to actually DO something remotely complex ... probably the brain teaser interview is far superior in this case (which are then meant to evolve into a deep discussion of a -preferably difficult- algorithm). The vast majority of developers are of the CRUD variety, but you'll never hear anyone admit as much.
What people seem to neglect about brain teaser questions is that at google they were originally meant to evolve into a discussion about CS algorithms. A discussion that the vast majority of the candidates has never even attempted, doesn't master the nomenclature, never programmed anything remotely complex (seriously, asking these guys what breadth-first-search is is too much to ask in 95% of the programmers that get through the phone interviews, wtf ? I'll never hire anyone who can't write A* on a blackboard without needing the internet. You should also get it right. You don't have to get the followup "now double the speed of what you just wrote down" perfect, but you should know the principles, and you should be able to make a good faithed attempt to modify your program. It's not that hard. And if you miss that one, don't miss the "what determines the optimality of this algorithm ?" question. Again intelligent discussion is much more important than directly stating the correct answer. If you miss the "double the speed" thing, I'll tell you what to do. If you can then modify your program correctly while discussing the various reasons for what you're doing intelligently, you've done vastly better than someone who just writes down the correct answer never saying a word. Syntax errors, make as many as you like (but not so much to make it obvious you don't know the language. And frankly the interviewer lets you pick the language, please limit the mistakes to cosmetic ones ... you're not inspiring confidence in your abilities. Additionally things depend on the language. Getting a detail of compiler-linker-architecture interaction wrong on C ... not a big deal (unless you start screaming at the interviewer when he tells you you're wrong - sadly happens). Getting anything about java wrong ... java is a tiny language ! There are about 3 weird things in the entire syntax, please make sure you know them all.
Now there's the "evil brain teasers" discussion. Which is cute, but it won't take away the requirement that candidates can discuss CS algorithms in an intelligent manner. You want to go straight to the hard part without a cute "let's be friendly here" introduction ? Fine. Brain teasers were never meant as more than an ice breaker and don't determine success or failure in the interview. Today it's back to "hello, sit down. State your name". Clearly that's "better" (wtf ?)
I once applied as a programmer to work on a server infrastructure for a next-generation search engine. They were looking in particular for people with great expertise in the C++ language and in the Boost libraries, areas in which I was a very good candidate.
They asked me to perform a task and send the result by email before meeting me in person.
The task was to write a program that would take an integer n, and display the nth integer that satisfies a particular condition involving primes (I have forgotten what the exact condition was). I was told I would be judged on the performance on my program.
It was obvious that what they wanted was for me to know the mathematics about primes so that I would know the right formulae to compute the nth value quickly. As I didn't know them, it was irrelevant to the job I was applying for, and I didn't want to spend time researching it on the Internet, I chose to fit their requirements differently.
I computed all the values beforehand, and simply made the program return the nth value of a table. Technically, it fitted the specifications they had given me exactly, and was the fastest solution possible.
Yet they chose not to make me go to the next stage.
Looks like brain teasers don't like being beaten at their own game...
(Another funny thing about this event is that I sent the code to the person as a tarball, and he was unable to open it and asked me to send him a zip instead.)
... is that the human mind does not literally perceive the world, see here:
http://bit.ly/dYaWUc
I never give candidates puzzles to solve during an interview. Why? Because they're not 10 years old; they're adults.
A real brain teaser, or a trick question? Many unintelligent people who imagine themselves intelligent when tasked with developing such a test will devise a problem or question with more than one valid answer/solution, but only count their solution as the "right" solution. The real right solution is the candidate's approach to the problem. Throwing random guesses out there is a shit answer (even if they luckily guess it right), and getting so buried in the minutia that they can't come up with a timely guess/estimate is also shit. You want the guy who is able to understand the scope and complexity of the problem, break it down into logical atoms, and make reasonable inferences and guesstimates based on the available data. If the candidate gives the right answer but cannot articulate why it is the right answer and how he/she arrived at that answer, they're going to be a useless piece of shit employee. Of course this assumes that the interviewers are possessed of these same skills and not merely imagining themselves as such (Dunning-Kruger).
I've been on many interviews and only had to do something similar once and, frankly, it turned me off the job. I had to sit in front of nearly 30 people (at the same time) during my interview and then to have some techie hand me a visio diagram half way through and ask how I would best design this network and what IP ranges I would use, etc. was overwhelming to say the least. It was also some part of their network still in the planning stages so HOW I could have answered correctly is beyond me when they don't even have a solution. Interviews themselves actually don't bother me when they are run correctly. I didn't appreciate getting blindsided with a test (one of several asked of me that day) especially under those stressful circumstances with everyone waiting on me to complete this document before moving to some other interviewer in the bunch. The job did not pay that much and was NOT a design job so that just added to my annoyance at this request. I get the idea but give the interviewee a break in this circumstance. At least give them forewarning so they can prepare somewhat or get into the frame of mind of answering specific technical questions in test format or as brain teasers and such.
If you are hiring somebody as a *programmer*, the marginal value of intelligence beyond "rather bright" is nil. Looking at his code is an obvious thing to do if it's available, but based on my experience hiring and managing programmers these are the other kinds of things I'd look for (references are crucial!):
(1) Conscientious. Will he adhere to coding standards? Will he commit his work every day? Will he focus on what he's told to?
(2) Honest. Will he tell me if he's having a problem, or will he cover it up?
(3) Personailty: A little arrogance is a good thing, but can he work with others without alienating them or getting into stupid turf or status battles?
(4) Ambition: Is he interested enough in furthering his skills that he'll willingly learn new technologies? Will he stretch on his deliverables in a way that adds to the teams capabilities?
Now the puzzle test may well make sense for many jobs at Google *other* than programmer, such as an algorithm designer. Programming beautifully is always nice, but hiring conscientious craftsmen wouldn't have helped Google when it needed to devise something like BigTable, or how to handle the immense, geographically distributed work load its customers generate.
You can see this in how Google treats new technologies. Ever wonder why they keep coming up with new things like Wave then dropping them? It's because many new services it develops are a side effect of being able to solve really tough problems in its core business. It's not a *product* oriented company; it's a company that does algorithms and infrastructure really, really well but does new things opportunistically.
There are many technical positions other than programming, and they take different kinds of people. The best system architects, for example, aren't necessarily the best programmers. They need to have experience and understanding of the process, of course, but they need a whole bunch of interdisciplinary business, technical, and people skills because they're shaping the activities of the developing organization far into the future.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I had an interview at O'Reilly several years ago and was given a piece of paper with some Perl code on it and was asked to comment on the code. The only problem was that the font was so small that it was literally at the border of what could be seen. I felt it was more a test of my eyesight than my knowledge of Perl. Brain teasers are a two way street, they can say a much about the person giving the test as the person taking the test.
The best job I've ever had was in a small company (of around 100 people) in which the hiring process took around two months to complete. It was a position as a remote developer for iOS.
After the initial interview with the HR people, they instructed me to code a small app in my free time for no monetary reward at all. The software was just a few screens with data and some relatively easy logic. The problem for me was that since I work during the day, and study at night, it took me quite some time to complete.
After presenting the code (it was my first obj-c project), the iOS architect interviewed me. He criticized some of the code, praised some other parts, and asked technical questions. The interview was probably the hardest part of it all.
After getting the job I can tell you it's pretty clear to me that while the whole process was a real PITA, the end result was amazing. The average level of knowledge and expertise I was able to find there, is something I hadn't seen before in other jobs. In retrospective, I can see how the software project they asked was a very good way to see people's willingness to get the job, and their ability to commit and deliver.
diegoT
Having been through through the Google hiring process, I can tell you that no one asked me a single brain teaser.
What they _did_ ask were programming questions - they were way too code centric when interviewing for a non-coding position. I have no problem with using coding problems to explore analysis and critical thinking skills, but the desire of the interviewers to syntax check code on the white board seemed silly to me.
As an interviewer I've asked brain teasers before. I'm not looking for the right answer, what I want to see is whether the interviewee's brain seizes up when faced with an unexpected problem. As long as they try something ... break down the problem ... look at it in different ways ... whatever, and they can articulate what they are thinking, they succeed.
I do hire developers, and I see brain teasers as a waste of time, meant to feed the ego of the interviewer rather than sort out people who are good fits for the company.
Communication skills are paramount in determining who has the best chance of success. That includes the ability to understand information being communicated to them, digest it, and respond by exporting that information clearly and appropriately based on an audience. It therefore follows that programming is every bit a communication skill as written, verbal, social, and listening skills are, and they are indeed correlated.
In my years of hiring experience, those with superior skills in the above categories make the best programmers. Even though we're all enamored with the idea of the asperger's guy in the corner who is a coding wizard, I've never come across anyone with poor written, verbal, social, and listening skills that could produce anything but garbage code. That may just be the programming/business environment we have, but it is still my experience and observation.
But getting back to the original question- we give candidates an hour-long programming test that is representative of the kind of work we do, weeds out those without basic skills, evaluates their coding decisions, and tests their ability to understand a business scenario and turn some requirements into reality. Brain teasers tell me none of this.
On a lighter note, you may or may not be surprised that over 50% of candidates who put "Java" on their resumes are unable to get past the first instruction to extend a given class. Completing this instruction concludes the first half of our technical test. Simply astounding.
I concur wholeheartedly. I could tell you that manhole covers are round because the manhole is round. But this doesn't relate in any way to actually solving problems by developing algorithms, or to how you approach a large code base, or how rapidly you learn our particular problem domain, or how reliable you are, or how readable your code is, or if you're capable of working to specification.
I do a phone interview to make sure the candidate and I understand each other, and to filter out bullshitters. Then I give them a test which should take half an hour to complete. I make my decision to hire almost entirely on the solution they provide, although the manager-types always want to interview the candidate.
Google's hiring process stinks of shit and red herring to me. You might as well be hiring an accountant on the basis of how well they do at crosswords.
Today's weirdness is tomorrow's reason why. -- Hunter S. Thompson
Is it only programmers that are irate about these questions? I don't see any posts from scientists, engineers, etc...
I've had over 100 interviews over the past 25+ years and the worst trend I've seen is people asking questions just because they think that's what they're supposed to ask or they read somewhere that's what Microsoft/Google/Current Popular Kid asks. Bad interviewers are the ones who don't know WHY they're asking a question.
The other trend started when all the "I.T." people came in after the .com bust in 2000. The I.T. world is filled with certification about installation procedures, product integration techniques, etc. Unfortunately they applied this mentality to Software Engineering and it's lead to an industry replete with boneheads interviewing for skill set and certification rather than TALENT. They don't even recognize there's a difference, so how could they possibly be expected to interview for and recognize talent?
These are the same people who say "Take this programming test before we'll even talk to you on the phone." Sure, why not, MY time is FREE! And what's hilarious is the results of the programming test are NEVER the criteria for hiring -- it's always some hidden agenda they never reveal and never post in the job description. Why not save EVERYONE (including themselves) a crap-ton of time and JUST ASK the actual important thing up-front? Why is this rocket science?
I've been a hiring manager myself in multiple positions and I've always hired very good people -- because I know how to ask the right questions. I can determine any candidate's viability in a 10-15 minute conversation. Easy-peasy. But for some reason most interviewers in the industry just aren't qualified to do this job. It's an unfortunate plague we've brought upon ourselves and one I hope to see the end of in my lifetime...
Having spent a fair amount of time on both sides of the hiring table in the software development world, I have some observations:
1) Puzzles, tricks, etc
These are generally very poor tools. They usually involve finding some particular angle or technique to solve. if the person has seen that sort of problem before, they can usually solve it quickly. If not, the likelihood that someone can find their way to the solution in the time frame of an interview is random at best. I also find them to be poor tools to evaluate someone's ability to problem solve as the puzzles et al typically require that one simply stumble into the solution - there is very little in the way of progressive problem solving involved.
2) Asking "what do you want...", "where do you want to be..."
Again, typically useless as most everyone has some stock answer that they think you want to hear. It generally actually tells you nothing about the potential employee.
3) Descent to Details
This is a tactic that I find usually works very well. If a person claims they have done X in their past, start digging into what X was, why did X come about, what were the challenges in solving X, what solution did they come up with. What I look for is whether this discussion leads to the prospect discussing details of the problem, the solution and the problem solving process. If they can intelligently discuss the details, then I can have confidence that they really did X and they had an instrumental role in solving X.
4) Open Ended Questions with no obvious "right answer"
One question I often ask is "lets assume you've been doing this job for a few years - how would you define being 'successful' at this job?". There is no obvious right answer and in general the prospect will reveal something of their work ethic and what it is that they want to do.
5) Short tasks
This one is tricky. It is often difficult to find a task which one could reasonably do in the time frame of an interview which also has any real depth to it. During the last dot-com bubble, I did find that asking someone to write even a simple program was useful at the start of the interview to remove the 'posers' who really had no clue at all.
The real challenge that I've found - particularly having gone through some interviews myself recently, is that Engineers generally make very poor interviewers. Good Engineers are focused on solving problems - so they try to pose situations to see if I can, for example, abstract. But there is no meaningful way to assess that in the framework of an interview. Or they will ask the candidate to solve some problem that they just solved recently - but again, usually, the problem cannot really be reduced to something that is meaningful/feasible in the time frame of an interview - and often the Engineer is looking for a specific answer (generally, *their* answer) to the problem.
So my other piece of advice would be - have your Engineering team talk to the candidate to assess whether they could work with them or not - but leave the evaluation of the candidate's ability to perform to the managers. The rank-and-file engineers are generally not good at that assessment.
My answer:
I've been under a non-disclosure agreement for the last 15 years across many employers. I know that you wouldn't want me to violate a non-disclosure agreement if you hire me here, so I'm certain you'll understand that I will not violate a non-disclosure with any of my previous employers.
For this type of work and for my experience, I believe this salary range, $xxx,xxx to $yxx,xxx is reasonable.
As a manager, I've tried to hire someone with all the desired skills for a fair salary, but our HR guy "demanded" old pay stubs. My guy was hired for $5K more than our offer while our HR team was screwing around. I bumped into him at lunch a few weeks later and asked.
Asking for old pay stubs is the HR way to ensure a highly skilled person doesn't double their salary with a new job. I know that I did this a few times over the years. Basically, I was underpaid by 40% at those jobs, so getting a 100% salary bump at a new job was definitely earned. Each time, I had multiple job offers for the higher salary ranges.
A willing buyer and a willing seller. Isn't that the basis for market economies?
Well I for one can't stand those type of questions during a live interview situation with people staring at you and watching their stop watches.
That is about as far from the normal design-thinking and programming environment as I can imagine.
I've written a programming language, an O-O database, a distributed storage-sharing app, a complex terrain model and zoomable map app and an active computer vision program etc etc etc but when confronted with a stupid little recursive thing in an interview my brain freezes and I sit there in fight or flight mode like a caged cheetah.
Figuring out what you're capable of when you have a month or a year to think about, investigate, and execute something is much more important than whether you can solve a Rubik's cube in freefall from 10,000 feet. Unless they're interviewing for Bond. James Bond.
Where are we going and why are we in a handbasket?
but I was considered for a marketing communications where I had to do a series of puzzles. I don't see what relevance that has to marketing communications.
As someone who interviews and hires developers...
I think the brain teaser has a place, but mainly to gauge how someone approaches and thinks through a problem, not for any specific answer. I agree with the OP, though: real code and big picture thinking is the best indicator of longer-term success. On the other hand, you can be a good coder without big picture thinking too, especially in a larger organization with good engineering management.
Personally, I look for people who know how to code (ie: can answer intermediate questions), understand what the code is doing (eg: in C++, why you normally use virtual destructors, but in what conditions you may not want to), and can think through problems (ie: here's an arbitrary hypothetical problem, tell me how you solve it). If you can do those things, you can be a productive developer; the rest (eg: specific knowledge, familiarity with tools/paradigms, coding trivia, etc.) is gravy.
You assholes.
We just quiz them on a few basic algorithms and ask them to implement them on a chalk board.
You could not imagine just how many fail at something as simple as finding the greatest number in
a massively large file of just numbers.
I've seen thbis practice in action and you end up with a bunch of idiot savants as your workforce.
I think the tricky brain teasers are fun, but when you use them, you run the risk of missing out on someone who might be a good developer just because they can't guess the trick. You also might get a bad programmer who just gets lucky and guesses the trick. These questions use up using a lot of time and don't necessarily tell you much about a candidate.
I disagree with a lot of the comments that say whiteboard problems don't tell you anything about candidates. I think seeing a candidate write code can be very helpful, as long as your evaluating problem solving skills rather than esoteric API knowledge. A good programing problem that requires several steps to complete tells me a lot about a candidate when I watch them solve it, even if they don't get the correct solution. I've started doing computer-based whiteboard problems which let candidates take advantage of an IDE with code completion and real-time error finding. This is more interesting, but you need to use a simple, well-known IDE (like Eclipse for Java) that someone can use easily without too much learning.
I agree with the OP that you never know how a developer will turn out until they do some real work.
Are the ones that have past experience and want to work. It's all a crapshoot.
Amen to the communicating. I'm a technical manager, and in many meetings we'll get into technical discussions. When that happens, I can see that many of us in the room may not be understanding. (Ok, at least one of us is not be understanding.) I get up and start drawing on the whiteboard what I think the speaker is saying (but keeping my mouth shut so they keep talking). My perception is that we come out of those meetings with a much better understanding than if we had just relied on words.
BTW, one of our meeting rooms is called the Whiteboard Room, because virtually the entire wall surface is covered with whiteboard. It's our most popular meeting room for technical discussions.
i find it amusing when people ask out of the box questions -- and then find they struggle with in the box questions
I think objective type written exam is same as a quiz.
Slashdot = Sarcasm
The premis is that Google and other coder rich companies hire based on brain teasing questions. I have gone through Google' interview process. Coding ability is tested and is weighted the highest. One off questions or brain teasers are weighted lower. These one off questions are designed to see how you think and solve problems. Do you ask for further clarification? Do you follow the intended path or blaze a new one? They are weighted lower because it is not the right answer they are seeking but how you got there.
Brain teasers are perfect hiring criteria IF ...
You want to hiring people that enjoy spending all their time answering questions that make them look smart, but not are NOT smart.
Remember Bill Gates ?!??!?!?! THAT was HIS big innovation, brain teasers to hire the WORLDS SMARTEST PEOPLE !
When he was a kid his parents sent him to a psychiatrist for arguing too much, then he argued with the psychiatrixt until they sent him home -- he is fine we would NOT like to see him again, THANKYOU !!!
They're excellent hiring criteria if you want to hire people good at solving brain teasers.
no more, but no less.
bickerdyke
It shows commitment to prepare and I believe that practice wont significantly improve your scoring ability. It is "platform neutral" and as such a good way to filter out the unfit. Of course it applies only to freshers.
OK
For me, these questions say far more about the interviewer/company to me as a candidate than I believe my answer will tell the interviewer.
Many people here are suggesting that such brain-teaser questions, whilst not directly related to a programming job, test the candidates ability to think outside of the box and to handle a somewhat stressful situation by having left-field and oddball questions thrown at them.
As a candidate, though, whenever I've been in an interview and been given these kinds of questions, I certainly do my best to answer them, and whilst I'd love to give a smart-alec remark, I never do since it's always good to be polite and respectful in the interview itself, even if you hear things that would very well put you off the position that you're interviewing for.
And that's the point, I hear these questions and despite verbally giving a reasonable answer, I'll immediately think that the interviewer is asking such a question precisely to see how I handle a "stressful" situation. I'll then further suppose that this is because they expect me be be dealing with "stressful" situations in the job, perhaps on a fairly regular basis. My mind will wonder about what forms these "stressful" situations will take. Perhaps the interviewer would be my immediate boss. Perhaps he's a micro-manager who'll stand at my shoulder all day asking, "Is it done yet?" incessantly. Perhaps the coding team follows no formal development processes/practices and they're an ad-hoc, fly-by-the-seat-of-your-pants kind of ragtag ensemble that works in perpetual panic mode, continually "dropping everything" just because some irate customer screamed louder than the last one.
You see, it doesn't really matter what forms those "stressful" situations will take, if the interviewer is concerned enough to "test" me on how I handle them, there must be enough of them to warrant that. If I'm interviewing for a programming gig, I want to join a well-oiled team made up of smart people who know exactly what they're doing and can do it very well. I want management that understands what it takes to motivate and get the very best out of such a team (Hint: It's usually management that sees themselves as working for the programmers, removing obstacles to let the developers do their jobs, rather than the other way around). And all of this means that the team should not be dealing with "stressful" situations all that often.
Any interviewer who sees fit to "test" me, in an interview, on how I handle "stressful" situations, I'll naturally assume that the job will contain many "stressful" situations, and I'll further assume it's because something is very wrong with how your team/company is run. Either way, I'll not want the job, even if you subsequently offer it to me.
So-called "lateral thinking" puzzles have nothing to do with cleverness or intelligence. I'm thinking of questions such as the one from the article, "A man pushed his car into a hotel and lost his fortune -- what happened?" These questions are purportedly about cleverness and "thinking outside the box." But the problem is, that there are a practically infinite series of possible answers that fit those facts, and you have to simply guess which one the questioner is thinking of. Rather than thinking outside the box, you are supposed to guess the questioner's box, as it were. I would answer the above question by saying, "You just told me what happened: a man pushed his car into a hotel and lost his fortune. They were two unrelated events. Next question please." It's a guessing game, like 20 questions or "I'm thinking of a person", nothing more.
Brain teasers are useless, and here's why:
My GF had a job interview at a large well known company. On the day before the interview, we talked about all kinds of questions that could come up, and I told her about the "why are manholes round" question as an example of possible brain teasers.
The next day, they asked her exactly that question and of course she knew the answer right away. The interviewers were very impressed and she got the job.
What did the brain teaser earn them? They didn't learn anything about the person they interviewed, it just so happened that she knew the answer to a random trivia question.