Slashdot Mirror
How Should You Interview a Programmer?
phamlen asks: "Having hired several programmers who haven't worked out, I'm wondering if other people have better success with interviewing techniques. Usually we have a two 'technical interviews' and a final interview. The technical interviews tend to be a combination of specific technical questions ('Is friendship inherited? How would you find out?') and algorithmic ('Given the numbers from 1-10 missing one number, how do you find the missing number?'). In addition, we essentially try to interview for: intelligence/performance. technical skills (algorithmic, etc.), and team compatibility. Unfortunately, we've been burned a couple of times by people whose performance didn't measure up to what we expected from the interviews. So I'm wondering if other people wanted to share their interviewing tricks - how do you find out if someone is a good programmer?" Surprisingly enough, we've done a series of these, so if you are interested in similar questions for sysadmins,
network engineers, or the one who will follow in your footsteps, then we've got it covered. We've also covered core IT questions as well. What special ways do you have of evaluating potential coders? How well have they worked out?
Unfortunately, I don't have the answer... I tend to look for someone with general problem solving skills, intelligence, and a genuine passion for that they do.
if you are asking them actual questions that have definite single answers - what is to stop them from studying for it?
wouldn't you rather have someone that can think on their feet rather than those that heard from a friend what your interview was like and studied to do well for that interview?
There are some odd things afoot now, in the Villa Straylight.
When I applied for my current programming job, they gave me a barrage of tests and compiled an aptitude and personality profile of me.
It was really freaky how accurately it described me... the main point was to evaluate me with reference to the type of person that excels at my job (Programmer/Analyst with some support duties)
They also asked for source code I had written and numerous references.
THe problem with an interview is it's too easy to bullshit. You need to go beyond the interview, as my current employers did.
Robots are everywhere, and they eat old people's medicine for fuel.
...if they answer "42", then hire them.
"Only in their dreams can men truly be free 'twas always thus, and always thus will be."
--Tom Schulman
Ask them if they write code as a hobby
What Open Source projects have they contributed to?
Ask them to bring some samples of source code they've written, and then do a walk-through
Ask them to solve a simple exercise with pseudo-code, then explain which language they would choose to implement it and why
Get them to find a known bug in some code that matches your "house style" (describe the unintended behavior)
Talk to their previous associates and boss....
YMMV....
Paul Gillingwater
MBA, CISSP, CISM
Interviewer: Who won the superbowl last year?
Programmer:
Interviewer: What do you do for fun outside of work?
Programmer:
Interviewer: Hmm. What do you look for in a woman?
Programmer:
Interviewer: Great then, one last thing we need to check...
Programmer:
Interviewer: Ok then, see you Monday.
Casca
The original posters questions and theories are a little weak. Testing a programmer's skills in constructing algorithms for random scenarios is a great idea.. if they need to use lots of algorithms.
The key to interviewing is to scope out the person's general work ethic, overall personality, and how well the person can do the job they have applied for. That's it!
In previous Slashdot threads we have learned that it's not wise to sit programmers down with a pen and paper and get them to write C code on the fly! Yet... the interview techniques you are mentioning are a lot like that.
Getting people to 'think on their feet' is good, if you're just talking concepts and ideas, but don't expect people to get things 100% right sitting at an interview table. These guys are programmers, not TV evangelists with all of the answers at the tip of a hat.
From the sound of your post it seems like you have interviewed people, found them to be great at algorithms and answering your questions, but then have found their work ethic stinks or that they're not as ingenious as you thought they were. That's because you assume that someone who can answer questions quickly and proficiently is a good programmer. Wrong!
Instead, look out for programmers who list extra-cirrucular projects on their resume. Look for programmers who have worked on their own projects, and can demonstrate them for you. Would you rather employ someone who coded a great deal of Gecko, or some gimp who can answer your algorithm questions?
Look for people who don't need incentives to work, but those who will program whether they get paid or not! Those are the people who will stick with you, and aren't just learning new languages to make a quick buck.
mogorific carpentry experiments
It seems that we have a collection of these articles and comments in our little community. CmdrTaco, why not put together a new section with a theme of Technical Recruitment.
Perhaps this new section could include these helpful questions and resources following the current re-education and recruitment techniques of the industry.
Any thoughts?
If you're interviewing the programmer, you somehow got pushed up to management and are screwed already. :)
-JDF
..for the most part. Most programmers with some sort of qualification can get your jobs done, unless your jobs require some amazing degree of skill. I probably couldn't write you out a bug free Quicksort first try, but I could certainly implement it in a real project.
And to be honest, most projects don't require skills nearly that nebulous. How many projects today are: get the data off the screen, validate it, then create the invoice.
The bigger question is whether they'll actually work hard on their jobs, or just play on SlashDot all day. And I don't know how to interview for that (and obviously neither do my employers).
.
Let's not stir that bag of worms...
At my company, since we're small, we need to know that new developers will click quickly. We do a technical paper exam (one hour) with some standard programming/algorithm questions. We then do a few riddles and logic puzzles. These are the best way to test raw intelligence, IMHO, since you have to think abstractly and quickly. We then do a few more design questions at a white board to test their skills at high-level design and diagrams.
;-).
However, the one thing that is difficult to test but really seems to be the deciding factor of a new hire "working out" or not, is whether or not they have the "passion". One way we try to determine their take on programming (just a job vs. a fun hobby) is to ask them to describe one software project that they have developed on their own time (not on the job or necessarily part of schoolwork). It's amazing how few actually code for fun or just to continue the learning process.
We then ask them what their favorite joke is just to jolt them a bit and see if they have a sense of humor. Most people fail this question, unfortunately
--- witty signature
Some example questions would be.
Which compiler do you prefer?
Complete the sequence. 2, 4, 8, 16, 32, 64
Are the voices in your head loud enough to disturb your coworkers?
"Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
Well, 10 minutes later, the president came back in the room, and there was a web browser displaying his creation -- a single sentence, "Hi Tim, I wrote a web page" in bold and italics. Up on the screen were other web browsers containing internet searches about basic HTML, as well as the output of "view source" from one of our web pages.
Three years later, this guy is still with us, by far the best customer service manager we've ever had.
I guess the point is, give the person a puzzle that you know that they have no idea how to solve, and give them the resources to figure out how to solve it, and see what they do.
- In Capitalist America, law violates YOU!
I've been on both sides of the interview table, and from what I can tell, most interviewers fall into the same trap: focusing too much on detailed technical questions. In reality, your programmers are going to be involved in much more than writing eloquent solutions to programming problems. Your programmers will most likely be involved in project management, project design, project implementation, project testing, and project deployment. Be sure not to get wrapped up in asking too many questions like "how many bytes in a java int?" Instead, look for good all around problem solvers. Ask about their design experience and what tools or resources they have used in designing previous projects. Ask how they would handle testing when a project has been under-quoted. These are questions that good problem solvers will be able to answer quickly, and those who "studied" for an interview will not. It will give you a much better idea of how your potential employee would work out in your business. Be sure that your interviewee will not only be a good programmer now, but also in the future when your development tools change.
Another useful tool for an employee interview is to have a break for lunch with a group of your staff. This will give you and your staff a chance to meet the interviewee in a less structured environment. Many times, an interviewee will relax a bit during your lunch, and you get a much better idea of the person's attitude. Someone who answers technical questions very well may turn out to be a social dunce. Or you may find that the person doesn't share the goals of your company. It will also give your staff a chance to find out if they fit in with the group.
If you don't feel satisfied without asking some technical questions, be sure to ask questions which apply to your framework, and not necessarily the programming language you use to implement that framework. For instance, if you design using Object Oriented principles, ask about "has a" and "is a" relationships. The idea is to ask questions that are still relevant if you change languages from C++ to Java or to some other language.
Using some of these ideas, my company has been able to easily pick the good candidates from the poor ones. YMMV: good luck!
I usually ask if they contribute to open source. Then, if they answer affirmatively, I tell them they can telecommute, give them a design spec document, and give myself a bonus for saving 100 percent on salary!
I don't have anything against vegeterians either </in our "How to cover up a typo" series> ;))
[humor]
It is obvious that anyone with hiring expertise, such as human resource specialists, can most effectively hire potential candidates by insuring that they have MCSE (Microsoft) or Red Hat (Linux) certifications.
This removes the requirement for the interviewer to ask intelligent questions, and for the interviewee to provide intelligent answers, streamlining the entire interview process completely.
After all, how else is an interviewer going to be able to BS a potential candidate into believing they know what they are asking about, and how else is a potential candidate going to BS an interviewer that they know what they are talking about?
As Microsoft and Apple have been pushing for on the desktop for years now, it is time we removed the expertise and knowledge from the entire process altogether, thereby "enabling" and "facilitating" the hiring process.
[/humor]
The Future of Human Evolution: Autonomy
How many people have you rejected that would have turned out to be the best people in your company?
I'll bet the answer is well over 'one.'
You're looking for a magic bullet. A simple mechanical reduction of human issues. It doesn't exist.
The only sure fire way I've ever found of evaluating an employee is to give them something to do and see how it works out, bearing in mind that often times a person with mediocre skills turns out to be a very valuable employee and those with great 'creds' often turns out to be nearly worthless. That's why God invented the probationary period.
To get a better look at what I'm driving at here take a look at another flip side. *You* are asking this question because you are performing less than 'perfectly' at evaluating prospective employees. Why? Because you're humans. You yourself are too complex to easily reduce your performance to a repeatable, mechanical formula.
It is always, ultimately, no matter what interview and evaluation process you impliment, going to come down to what it has always going to come down to, an educated guess and a gut 'feel.'
And you'll make mistakes, you'll hire people you shouldn't have, and *you'll let go people you should have kept.*
Thus it has always been, thus it will always be, as long as it's people we're dealing with.
KFG
that you're interviewing programmers, aka code monkeys. They might dance, but they will never perform. You'd probably like an engineer or a scientist. How you interview for them is totally different.
I worked for a startup company back when it was the cool thing to do. The nerds with titles were debating how to interview for a new position, and the battle came down to this essential problem - which is the best question:
1. What is java.lang.Thread.join()?
or
2. Tell me about how you start and stop different execution paths in a multithreaded application.
If you ask (1), you get a code monkey. He or she will write good code when given proper instruction because he or she has a minimum set of skills. Code monkeys can handle hammers and screwdrivers because they've used them before. Ask them to use, say, a quarter sheet finishing sander and they will be confused.
Ask (2), and you get an engineer or a scientist. Knowing that you can wait for the termination of a thread in java with join() is nice, but understanding the implications and uses of join() is ten thousand times more important. Understanding the concept is more important than perfect syntax.
My suggestions for questions are these two, because I think you are less likely to pick a code monkey and more likely to pick an engineer:
1. Tell me about a project you are particularly proud of, and explain some of the technical issues you faced in finishing it. (This is a good question for several reasons. First, you get a good sense of interpersonal skills, because they have to tell a story. You also can gadge a candidate's general interests in the larger field of computer engineering/science, and a feel for their particular strengths. Lastly, you get to see whether this candidate is a finisher or a ship-it-when-it-compiles person because you asked about finishing a project, which is never the most glamorous, but frequently the most important part of being a software engineer.)
2. Tell me about a project you worked on with a team. What kinds of challenges did you face and how did you solve them? (Again, story telling, this time with a definite bend towards interpersonal skills. You also get to assess team work skills, etc., in a technical environment. When I was asked about this question I talked about how my junior design project team needed to be more organized to meet our project schedule, so we got stricter about version control, documentation, etc. If the candidate tells you story about this irritating person or that jerk, you should consider whether or not you're going to be the jerk he talks about in his next interview.)
Outside of a dog, a book is a man's best friend. Inside a dog, its too dark to read.
"I'd buy puppies for orphans"
You MONSTER! Do you know how much it costs to care for and feed a puppy? And you'd inflict this financial burden on poor orphans?
You're sick SICK SICK!!!
"Mind, as manifested by the capacity to make choices, is to some extent present in every electron." -Freeman Dyson
1] Describe the technical accomplishment you're most proud of.
2] How did it work?
3] What did you do on it?
4] At our company, we have this general problem X. What are your first thoughts on how to solve it?
5] How would you rate yourself in (language)?
6] A language specific question. For instance in C, what does "volatile" mean? For C++, write code whose meaning would change if you used the keyword "virtual" in front of a base class. (Note: passing the test is nowhere near as important as that it generally matches with the answer under #5).
7] (Note: several questions may be done in this area - again, it is more important that skills are accurate on the resume than everything is done exactly right.)
8] Say you have a technical disagreement with a fellow programmer, and you really think you're right. What are the steps you'd take to resolve it?
9] What sort of software tools are you familiar with? How to you coordinate development with other engineers?
10] What are the things you expect from the company for us to make you happy?
I have noticed in interviewing that engineers can easily spot other good engineers. If you can't, it's because you can't program yourself. So go get someone with the skills to do your interviewing for you.
I don't know what survey your employer used, but you spent some effort to complete the survey, expecting that a well-designed system would evaluate some qualitative aspects of you. When presented with results, you subconciously hoped to be:
- described accurately
- described favorably
This subconcious desire on your part made you willing to forgive minor points that didn't fit your desired outcome, and willing to magnify points which did fit the desired outcome.Again, I don't know what survey you used, and there certainly are valid personality tests out there, but don't get too freaked out when one seems to describe you to a T.
DO ask for demos of working apps from previous jobs/schools. If they don't have anything working to show, they can't take a project, even a simple one, cradle to grave. You want self-starters who don't need constant supervision.
DO NOT make them solve brain-teasers on the spot, regardless of what joelonsoftware.com might say. I love brain teasers personally, but trying to get all the members of U2 across a bridge two at a time doesn't exactly translate. Reread number 1, and if they gave you their stuff, you're safe.
DO ask them to review code from your shop and tell you what they'd do differently. Whitespace, comments, logic that should be pulled into functions or other objects -- these are the kinds of things a good programmer will notice. A good potential team member will even point them out, point blank.
DO NOT discriminate because they haven't programmed in your particular programming language, unless the work is very short term. They're all dialects of the same language. Good code is good code, even VB! (Note that I didn't say "working code" -- I *mean* good, commented, well laid out, non-repetitive code) The only exceptions are pointers and object oriented code. Some people just can't get it. Test them [by showing them code to review] if you use either.
DO look for someone who gets passionate about a topic during your interview.
DO NOT for one second think that someone who claims they have 10 years experience in C, VB, Java, and FORTRAN means it. Ask what they've done which each language. If they can't tell you in enough detail that you can envision it, that's a "no hire".
DO, for heaven's sake, call their references.
And most importantly (and this is something olde Joel gets right), "Maybe" means "Don't hire". If you can't strongly recommend the candidate after the interview, don't hire him/her. Mistakes at hiring time will cost you for months and maybe years. It's worth spending the extra month or two to find someone worth their salt. Oh, man, it's worth it.
It's all 0s and 1s. Or it's not.
There seem to be lots of good responses here, many of which I have used myself in the past. But no-one has mentioned my favorite; the 'Toilet Tank Test'.
The 'TTT' is designed to find out if the person thinks about programming off the job, if programming excites them and just doing it is enough to motivate them all by itself. It works like this:
(After technical, logic puzzle and attitude questions are dealt with)
-- First Interview --
INTERVIEWER "OK, so let's suppose I walk into your house and go into your bathroom right now. What magazines would I find on your toilet tank, or wherever else you keep magazines you read often?"
INTERVIEWEE 1 "Uh... Golf Digest, Sports Illustrated, People I guess." (Doesn't mention Penthouse.)
INTERVIEWER "Thank you for your time. Don't call us, we'll call you."
-- Second Interview --
INTERVIEWER (asks 'TTT' question)
INTERVIEWEE 2 "Uh... Linux Journal, Dr. Dobbs, Game Developer I guess." (Doesn't mention Penthouse.)
INTERVIEWER "When can you start?"
Jack William Bell
- -
Are you an SF Fan? Are you a Tru-Fan?
In a corporate development environment, you don't want someone who can only write code based on what they already know. You want someone who can accept a task requiring skills they don't already have - yet deliver quality anyway.
...) then you have a candidate that has a variety of tools in their skills toolbox.
The old adage about "if all you have is a hammer, everything looks like a nail" is what I'm going on about here.
If your candadate only knows one thing ( Java or Delphi, or C++) be wary.
If the candidate knows something useful about a wide variety of things (Java, Delphi, C, C++, shell scripting, XML, XSL, HTML, CSS, Perl, Python, Ruby, batch files, SQL, XQL, servlets, JSP, ASP, PHP
Before anyone chimes in with the old myth "you can only know one thing well" - I agree completely, you can only be an expert in one or two areas. But you CAN know a dozen (or two) things well enough to know which to use - one of the brightest developers I've ever met was a guy smart enought to say "I shouldn't do this - it needs X and I don't know it well enough. Give this to person A and I'll pick up what they are currently doing." This same guy scored 100% on the Java certification exam - he's that good.
Ask your candidate what tools they know - from what vendors. Don't settle for one or two - keep pushing for as many as they mention. Ask them to explain how they would choose between tools - if needed, give them a scenario or three.
One of the things you're trying to find with this approach is how well they might understand the principals that underly the languages - just as you wouldn't ask a fish about water, you can't ask someone who knows only one tool to critique that tool.
Another idea is to get your candidate to give a five minute off-the-cuff presentation on something interesting. Limit it to stuff relevant to the position you're interviewing for, but otherwise leave it open for the person to choose for themselves. They'll choose something they know well - look for how they speak, how well they explain, how well they teach. Also shows how they work under pressure.
I interviewed for a position as Sysadmin once. They asked me how I'd troubleshoot the Blue Screen of Death. My response was "I'd ask Clippy".
Didn't get the job, though.
"Derp de derp."
Why not take your top three candidates and pair with each of them for an hour or two? Pick a task with little domain knowledge needed. Let them drive or do the majority of leading and see what happens...
Then ask yourself (remembering that this is your first pairing with this person)..
Did I like this person?
Did he try to work with me or against me?
Was he technically capable?
Was his technique compatable with yours?
Could he adapt to your style?
Could I corroborate daily with this person?
Does he smell ok?
Did he offer to buy you lunch?
Was he enthusiastic?
If the answers were yes or mostly yes, then you've got a winner.
managers...why god invented purgatory
So what if I'm the kind of programmer who would have to whip out a book to do this? Does that make me a bad programmer? What if my methods and code are far cleaner and efficient than most, and I always produce what I say I'm going to produce on time.. are you going to discount my work as a programmer because I had to look something up? Because I didn't know it on the spot?
Seems far too many people think it's about hotshot know-everything-instantly type of work, where you work when inspired or work when you are 'in the mood'.
What about those who can methodically and cleanly produce code day after day?
Having hired several programmers who haven't worked out...
... or showered, or shaved...
1.
2. For myself, having hired several
bodybuilders who haven't programmed...
Considered harmful.
you would expect them to work offline?!?! come on. just give them everything they would have in a normal working environment. do you really want to not hire someone because they forgot to add one little thing? because they didn't have access to the documentation for whatever tools they are using? I am of course saying this all because I'm one of those people who doesn't memorize. I just remember the best places to look when I need a reference for any of the number of languages I've done work with.
also, 90% of work is on the job training... do you expect people to be a perfect match walking in the door? wouldn't you rather have someone intelligent, adaptable and dependable, than someone who really really knows css and frontpage?
I ate my sig.
... to simple questions.
People who have "training" but lack skill or experience are desperate to show you what they know. You can ask a very simple question, and they'll throw out names of tools they think might be relevant, and buzz words they've heard. They're unlikely to give you the simplest answer.
I once was asked in an interview for a DSL installation tech job, "if you installed a memory upgrade into a laptop, and upon boot up the new memory wasn't recognized, what would you do next?"
I felt kind of foolish saying "Well, I'd open up the laptop, reseat the memory, and try again." But the interviewer nearly wept... he'd been interviewing people with all kinds of "qualifications" all day, and I was the first person who had given this answer. He told me how everyone else had said "Well, I'd start up Tech Tool..." or "I'd get out a memory tester and..." without even checking that the installation had been done right in the first place.
That, of course, is not a comprehensive method for finding a good person for a job, but it might make your technical questions a little more effective.
Don't you wish your girlfriend was a geek like me?
I'll probably get crucified for this....
If you have to include a "technical interview", not to mention 2 of those, then your company is shit. If your hiring manager can't get by with interviewing the guy/gal with some of his/her folks doing an informal interview (that do have some probing tech questions), then your entire structure is fucked.
A bit ago I went through a tough time at my company, so I started out interviewing. I got 3 tech interviews that asked me questions like "what is the command to increase the frame relay delay timing", "what is the public number for public MIBs", and what are the arguments to display only workstations in a Tivoli system?"
Give me 2 minutes with my books, I would have no problem. Expecting me to memorize all this random shit is just beyond stupid. Go find a 5th grader who memorized the nation and state capitals on the 1st try if this is what you want. If you want someone who actually solve a problem, maybe you want to hire someone who can research the problem and come up with a better solution.
I'm still with my first company out of college. The hiring manager didn't even ask about my skills- he wanted to know if I wanted to learn, if I wanted to gain new skills, and if I was willing to put in the time to learn new stuff. Of course, I'm a DoD system and network consultant, so I need to learn and master new stuff all the time. The couple of corporate projects I've been on have so focused on one single aspect that they get a llama that can program in Java, and they still hire the llama. (Yes, that was mostly facitous).
So go interview the guy/gal about who they are, what they want to do in life and in thier job, how they like to do their work. Don't worry about what languages they can program in (unless thi is what you are looking for), any literate computer person can learn a new language in a few days.
In summary, look at the person, not thier certificatons and answers to crazy technical questions.
Vote monkeys into Congress. They are cheaper and more trustworthy.