Slashdot Mirror
How Should You Interview a Programmer?
phamlen asks: "Having hired several programmers who haven't worked out, I'm wondering if other people have better success with interviewing techniques. Usually we have a two 'technical interviews' and a final interview. The technical interviews tend to be a combination of specific technical questions ('Is friendship inherited? How would you find out?') and algorithmic ('Given the numbers from 1-10 missing one number, how do you find the missing number?'). In addition, we essentially try to interview for: intelligence/performance. technical skills (algorithmic, etc.), and team compatibility. Unfortunately, we've been burned a couple of times by people whose performance didn't measure up to what we expected from the interviews. So I'm wondering if other people wanted to share their interviewing tricks - how do you find out if someone is a good programmer?" Surprisingly enough, we've done a series of these, so if you are interested in similar questions for sysadmins,
network engineers, or the one who will follow in your footsteps, then we've got it covered. We've also covered core IT questions as well. What special ways do you have of evaluating potential coders? How well have they worked out?
Ask them what they consider a productive day.
... the swap function. It may be simple and about three lines long, but you'd be surprised how many people it weeds out who simply don't understand pointers.
And understanding pointers (even if you use non-pointer languages) seems to be a common trait of most "Good Programmers".
When I applied for my current programming job, they gave me a barrage of tests and compiled an aptitude and personality profile of me.
It was really freaky how accurately it described me... the main point was to evaluate me with reference to the type of person that excels at my job (Programmer/Analyst with some support duties)
They also asked for source code I had written and numerous references.
THe problem with an interview is it's too easy to bullshit. You need to go beyond the interview, as my current employers did.
Robots are everywhere, and they eat old people's medicine for fuel.
The original posters questions and theories are a little weak. Testing a programmer's skills in constructing algorithms for random scenarios is a great idea.. if they need to use lots of algorithms.
The key to interviewing is to scope out the person's general work ethic, overall personality, and how well the person can do the job they have applied for. That's it!
In previous Slashdot threads we have learned that it's not wise to sit programmers down with a pen and paper and get them to write C code on the fly! Yet... the interview techniques you are mentioning are a lot like that.
Getting people to 'think on their feet' is good, if you're just talking concepts and ideas, but don't expect people to get things 100% right sitting at an interview table. These guys are programmers, not TV evangelists with all of the answers at the tip of a hat.
From the sound of your post it seems like you have interviewed people, found them to be great at algorithms and answering your questions, but then have found their work ethic stinks or that they're not as ingenious as you thought they were. That's because you assume that someone who can answer questions quickly and proficiently is a good programmer. Wrong!
Instead, look out for programmers who list extra-cirrucular projects on their resume. Look for programmers who have worked on their own projects, and can demonstrate them for you. Would you rather employ someone who coded a great deal of Gecko, or some gimp who can answer your algorithm questions?
Look for people who don't need incentives to work, but those who will program whether they get paid or not! Those are the people who will stick with you, and aren't just learning new languages to make a quick buck.
mogorific carpentry experiments
At my company, since we're small, we need to know that new developers will click quickly. We do a technical paper exam (one hour) with some standard programming/algorithm questions. We then do a few riddles and logic puzzles. These are the best way to test raw intelligence, IMHO, since you have to think abstractly and quickly. We then do a few more design questions at a white board to test their skills at high-level design and diagrams.
;-).
However, the one thing that is difficult to test but really seems to be the deciding factor of a new hire "working out" or not, is whether or not they have the "passion". One way we try to determine their take on programming (just a job vs. a fun hobby) is to ask them to describe one software project that they have developed on their own time (not on the job or necessarily part of schoolwork). It's amazing how few actually code for fun or just to continue the learning process.
We then ask them what their favorite joke is just to jolt them a bit and see if they have a sense of humor. Most people fail this question, unfortunately
--- witty signature
I have to agree whole heartedly. One of the nice things about programmers is that you don't have to guess whether they are good or not. You can look at their code or ask them to interpret some that your team has already created and critic it to see if they code in similar ways etc. Since it sounds like you are employeeing multiple programmers. Have one of them sit in with the guy and talk shop, work on functions, etc. If your guys are good programmers, they should have a pretty good BS detector. Artists have portfolios, why shouldn't programmers (PDA's aside).
One of the reasons I contribute to open source projects is to learn something new that will perhaps be needed for a future job. In the interview for that job, I would think that being able to point to source code that is in production, so to speak, would be a tremendous advantage. Has this been the case for anyone? Has anyone gotten a job primarily due to related open source contributions?
...casually ask them where they post. Then ask them what their User Name is. Then go read what they have written.
Not only will a handle like LuvWarez or MyBossSux tell you a lot, but I suspect that over time a person's SlashDot messages (for example) are a good indicator of their real personality and attitudes. And intelligence. And sense of humor.
I am in the same position at a small company and we've had our share of hiring bombs (excluding me :) ). Here, I think, are some core issues to consider:
1) Decide what you want. Uber genius or uber implementer. Everyone falls in between but I beware of people who claim themselves to be "really into design" when we really need implementers.
2) Don't ask the same old questions. I think most everyone is aware that a book of interview questions can be found online. Sometimes learning the answers to these is as easy as memorization.
3) Find out what the interviewee knows and doesn't know, never just one. One beef I always have is that when I'm being interviewed the interviewer would assume I know some very specific about a certain technology I worked with. Why would I know the specific way threading works on web servers when I've done web applications?
4) Technology and experience must go hand in hand. I think every programmer knows that technology can sometimes be seen in terms of an API. While experience is useful, so is design experience (and vice-versa). One doesn't necessarily make up for the other.
5) Ask relevant stuff. While intelligent people may correlate to good programmers, intelligence alone might not make up for lack of experience and/or the ability to learn new technologies quickly. I've had my fair share of algorithms and trick questions asked to me at jobs where I never implemented any sorting/searching algorithm nor did a project on how to measure time by burning candles. One thing I would look for is initiative to learn new technologies. Did they learn how to design and implement a web application on a short 6 month project? Or did they just manually fix html bugs for 1 year?
Lastly, if you're unsure to hire than pick the hotter chick (or the only chick).
"Injustice anywhere is a threat to justice everywhere." - Martin Luther King, Jr.
The first reason is that the programmer's employer is (almost always) the copyright holder of the code. A lot of companies consider their programs to be both property and a protected secret -- asking the candidate to bring in code is like asking them to steal. To put things another way... how would you feel if one of your employees was taking your company's programs to other employers (and potential competitors)?
You could require that the candidate to have done something for the Open Source or shareware communities... but then you're really limiting your choices.
The second reason not to ask for code is because you have no reference for it. You don't know how well it works, how well it integrates into the system... you don't even know if the person you're interviewing wrote it! I have known a couple of programmers who have submitted "brilliant" code samples to the PHB, but couldn't function in a working environment.
On a related note, would you penalize the programmer for adhering to his/her company's standards for formatting code, because they don't conform to yours? I've had interviewers say they didn't like the way I formatted my code, despite the fact that it was required for the job I was currently doing.
My suggestion: set up a computer off the network. Allow the candidate access to manuals, and give them a small programming task (and about 30-45 minutes) to complete, and let them work.
Many moons ago I used to ask questions like "What is your favorite movie?" to try and loosen candidates up. I then got the smackdown from our corporate HR, which informed me that the question could be viewed as discriminatory. Same thing for any questions about "for fun projects" or anything of a personal nature. It's a thin line to walk.
So a whole herd of us manager types got shuffled off to Behavioral Interviewing training. The basic premise is that you construct questions to gather information on specific work related issues they have experienced in the past. Assuming past performance is indiciative of future value in this case, you can get a good feel for how a candidate will perform in a given situation. As for faking the answers, that can be tough to do with some of the questions unless you are dealing with a very gifted improv performer.
Behavioral Interviewing is the norm at several Fortune 500 companies I have worked with. Take a peek at it, and definitely run any ideas you read here past your HR folks. All it takes is one lawsuit and you will find yourself on the other side of the interviewing table.
One thing that seems to be skipped is a clear description of what you WANT in a programmer.
Most jobs I've had I've had great evaluations due to my attention to detail and care given to corner cases. I pay a lot of attention to the design phases and ask a lot of questions. I communicate about deadlines and the occasional slippage. Project managers tend to love me.
One job let me go after my 90-day eval because they said that I didn't take enough on, didn't invest myself fully, wasn't enough of a self-starter.
From their viewpoint I had a great interview and didn't live up to it. From mine I was left ticked off that they didn't communicate they wanted a shoot-first-ask-questions-later guy. Or a throw-myself-to-the-wolves guy. I'm neither.
Since then I've switched to freelancing and I love it. My clients love me and call me back for their next projects.
Other point - occasionally, I think too much emphasis is placed on the brainteaser questions. Assuming someone's resume doesn't lie, a failure is more likely going to be about work style and personality flaws than someone's brain being too slow. I've had five salaried positions since college. Four asked me technical questions relevant to my experience and a bunch of other questions about what kind of guy I was, and they went well. It was the other one that focused on brainteaser questions like dumping cubes in paint. There needs to be room left for gut feelings in those kinds of decisions.
We've had this problem where I work, and have been fooled by people (well, one person in particular) who had an impressive resume and could talk the talk. He turned out to be someone who couldn't code his way out of a wet paper bag.
Our solution, which has worked very well for the last four years or so of hiring, is to sit the person down in front of a laptop connected to a small and portable flatpanel. I sit on one side of the table, watching the flat panel, and the candidate sits on the other side, working on the laptop.
The entire interview consists of trying to fix all the bugs (and there are many, some obvious and some subtle) in a small command-line tool. During the interview, the candidate is requested to think out loud as much as possible, so that I get a feel for how they're thinking about the particular problems they've been presented with.
Fixing every bug is not necessary or expected -- tackling the problems and presenting good solutions to those problems is crucial. It really gives you a taste for how the person goes about problem solving and how they code, especially under pressure.
Several people have commented that this seems like a severe way to interview, but most of the candidates I've interviewed have demonstrated a fairly good ability to work under these kinds of conditions, and it has improved the quality of our department staff considerably.
The standard interview format doesn't work for programmers. The best interview I've been given was from Art & Logic, which is a coding test. It weeded out the weak pretty well, as well as the non-self motivated types (I made the cut, worked with them for a while, then got an offer I couldn't refuse elsewhere - but they're a great team of solid coders).
My only complaint with them is that I didn't get a peer code-review afterwards - I would have liked to have had more feedback on the code, even though they apparently liked it enough to hire me.
A lot of game companies do this as well - I know Acclaim's subsidiary Iguana software used to ask for you to write a Defender knockoff, and I've heard of others.
I write code.
Paul,
I've been in this biz only a little longer than you have and agree with your approach.
A couple things I've noticed about the better programmers I've worked with/hired are:
Most play a musical instrument of some kind.
Most enjoy and are good at strategy type games such as chess and go.
Left handedness seems to be higher than in the general population.
They do crossword puzzles.
John
If you're not living on the edge, you're taking up too much space.
We just turned down an applicant, partly due to poor spelling on her resume. This was a scientific position which requires careful attention to detail. If she couldn't be bothered to run a spellcheck on her resume, why should we assume that she wouldn't take the same attitude to her work with us?
Don't get me wrong, skills are very important, but you do not, under any circumstances want to forgo skills for personality.
To be a good programmer you need to be:
1) easy to work with.
2) No Prima Donna attitudes.
3) passion
4) smart
There are some people that I have worked with that were really good at bug fixing. One co-worker would spends hours and days until she found really really tough bugs so she was really useful. Because she was good at fixing bugs, she thought she was a great programmer. WRONG! Fixing bugs is easy, it's generating your own code from scratch that is hard. This bitch thought she was too good for code reviews so she never did it, and changed code whenever she wanted and wreaked havoc on our products. Whenever you even mentioned some of her code she would get defensive and on a couple of occassions started crying because she thought we were attacking her. No dumbass, we're just trying to fix a bug in the program. Anyway, believe me, you do NOT want to hire people like that.
Brain teasers are the stupidest things to ask during an interview. Figuring out problems is only half the solution when it comes to programming.... you also have to worry about maintainability and coding style because in the long run, this is what matters most.
When I start conducting interviews again, I will
1) ask some technical questions to see if they can talk the talk
2) go for lunch with them and just shoot the shit to see what their personality is like
3) give them an overnight somewhat challenging coding project and have them e-mail me their code the next day. This lets them code in their most comfortable location, gives them all the reference materials they need (who they hell knows all the syntaxes off the top of their head) and I can take a look at their coding style.
The other thing I have learned for myself is that the next job I take, I'm going to require them to show me a sample of their code for their products. I will not take a job where the previous coders left a piling of steaming shit and then I have to come in and either clean up their mess, or code through hoops because everyone is too afraid to touch it because it might break everything.
... to simple questions.
People who have "training" but lack skill or experience are desperate to show you what they know. You can ask a very simple question, and they'll throw out names of tools they think might be relevant, and buzz words they've heard. They're unlikely to give you the simplest answer.
I once was asked in an interview for a DSL installation tech job, "if you installed a memory upgrade into a laptop, and upon boot up the new memory wasn't recognized, what would you do next?"
I felt kind of foolish saying "Well, I'd open up the laptop, reseat the memory, and try again." But the interviewer nearly wept... he'd been interviewing people with all kinds of "qualifications" all day, and I was the first person who had given this answer. He told me how everyone else had said "Well, I'd start up Tech Tool..." or "I'd get out a memory tester and..." without even checking that the installation had been done right in the first place.
That, of course, is not a comprehensive method for finding a good person for a job, but it might make your technical questions a little more effective.
Don't you wish your girlfriend was a geek like me?
In one of my previous companies we actually tracked engineer recomendations for new hires (i.e.: how well did a person you recommend to work at the company is actually doing after a period of time X), and after about 18 months I came up on top of the list after 98% of the people I recommended ended up being described as "outstanding".
If you care how I can tell the good ones from the bad ones, read on.
It all boils down to someone being proactive in learning things, entusiastic about the field he/she is going in, being able to communicate effectively, and basically have the capability to look at things in more than one way.
In my experience the best thing during interviews is to let them talk as much as they want, you will be surprised how much you can learn from them in just a few minutes. Encourage more conversation by asking short questions along the form of "can you explain that part a little bit more for me?".
Also, to avoid the "bullshit talker", once in a while interrupt them and ask them to described how exactly (in pseudo-code) they solved a specific problem.
It's also a great idea to ask them to draw visual diagrams that explains how things work. This tells you a lot about the way they solve problems.
If you have the time, place them in front of a computer and hand then a piece of pen and paper and tell them to write a made-up documentation for a fictitious project. This will tell you how well they communicate and how well they express their ideas.
During all this, it is up to you to figure out what makes this person special from the others. Is he great at explaining things? is she great at understanding what you mean and putting it down into a design on a piece of paper? Does he come up with novel ideas to solve problems?
That's basically it. I usually refrain (unless it is a basic requirement) from asking language-specific questions (C, Java, VB, etc), since usually a smart programmer can pick just about any other language in a few weeks, and besides, usually newcomers don't start from scrach programming, there's usually an installed based of development tools and written code which can bring him/her up to speed.
Those are my 2cents of wisdom.
It hardly ever happens this way in real life. Many interviewers have no clue what they are looking for. Most questions are egoistic - just to prove that they know something that the candidate doesn't. Last year, a business software company asked me questions about Turing machines and the Halting problem. I answered him and further added that I had never thought about those since school and did not expect to use them at his company. So why did he ask me that? He said he just wanted to test me!
An interview is not a quiz. If you are looking for a software developer to write Servlets, don't ask him higher math and complex algorithmic questions. Try to find out his views on software engineering itself (good practices that we all know about) and technologies related to his job (HTTP protocol, for example).
Also, don't ask a programmer about any specific api or libraries (unless knowing that is specifically his job!).
Don't ask him about tools ("how comfortable are you with Visual Cafe?" is a stupid question").
And so on. Bottomline: Know what you need from him and see if he has that!
All your favorite sites in one place!
2 . Scan through the list of numbers and stick them in their respective slot.
3 . Scan through the array and return the element index that is empty (or invalid value, as the case may be)
This is simple, straight-forward, easy to debug and only O(n). Why complicate things beyond necessity? With n unsorted elements, you are going to anyway arrive at an O(n) algorithm (atleast).
Programmers don't write code to fix problems frozen in time. The requirements keep changing and the code must be easily readable and maintainable! These are more desirable features in programmers.
In real life, in 3 months, the problem statement could be changed as - "atleast 2 elements may be missing and you have to return the highest, unless the lower missing number is 3, in which case you need to return 7".
Will the complex algorithms proposed handle this change gracefully?
All your favorite sites in one place!
I found this a worthy enough read to bookmark it. The author has some good ideas about preparing for an interview and formulating a meaningful evaluation of that individual's skills beyond the basic text-book Q/A dialogue. http://www.joelonsoftware.com/articles/fog00000000 73.html
Fools ignore complexity; pragmatists suffer it; experts avoid it; geniuses remove it. ~A. Perlis
> Give them a question they don't know the answer to
> (this may take a few tries). If they try to feed
> you some BS they're probably not a good candidate.
> Its always good to know you've got people willing
> to give the honest answer, "I don't know".
Heh. I went to an interview once for an administrative/support position. From the questions the interviewer asked, I got the opinion that he had an overestimation of his own abilities.
Anyway, he asked me a question but first gave some background. He said that he had a SCSI problem with a server that had taken himself and 5 other engineers over 6 hours to diagnose. He then asked me the question. I asked him a question about the system and instantly knew I was on the right track by his reaction. He said to me that it was ok if I didn't answer it correctly in the the next few minutes. I then asked him the second question about the system (zeroing in on the solution) whereupon he began looking uncomfortable. I asked the third question and hit the nail on the head. He said he'd get back to me and that I had figured it out.
I never heard from him again. To this day, I guess I intimidated him or he thought I was a smart@ss.
I'm not trying to brag or anything... it's just that I like to know my stuff. I like solving problems and fixing things.
I guess the moral to this story is that sometimes you have to feign stupidity? Don't look better than the boss... I dunno. I just get the feeling that they are looking for YES men who make them look good rather than people who know what the fsck they are doing. Heh whatever.
Codifex Maximus ~ In search of... a shorter sig.
I did not mean the drunk girl at the party.
I have found my female colleagues naturally better system analysts, quality conscience, far better at working within a team, and more willing to forego "gold-plating" of the deliverables.
There are fewer assembly or OS level system programmers that I've meant, and I think it is because they are not obsessive about minuta that assembly programming requires. I also find they are not willing to participate in "death march" style of project management. Which should be considered a good thing, because "death march" is a not good software engineering practise.
In any case, I think companies should hire people to be software developers (or software engineers) not code monkeys if they want them to stay as long term employee .