Slashdot Mirror
Programming Interview Questions Are Too Hard and Too Short (triplebyte.com)
Programming interview questions can feel unnecessarily difficult. Sometimes they actually are, a new study has found. And this isn't just because they make interviews excessively stressful. The study shows that harder programming questions actually do a worse job of predicting final outcomes than easier ones. From the study: Programming under time pressure is difficult. This is especially true during interviews. A coding exercise that would seem simple under normal circumstances somehow becomes a formidable challenge under the bright lights of an interview room. Stress hormones cloud your thinking during interviews (even though, sadly, neither fight nor flight is an effective response to a menacing programming problem). And it can almost feel like the questions are designed to be perversely difficult. I actually think this is more than just a feeling.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
I shortlisted for an interview and got ambushed on my arrival. There were 19 other candidates, and we're all ushered into a small room with 20 desktop spots on a table that went around the entire room. We were handed one single sheet of paper with a coding problem we were to find a solution for. All 20 workstations were Mac Minis. None of us were told there was a programming exercise, none of us were told it was a Mac shop.
I walked out of the "interview" in disgust. Eleven others went with me. By the looks on the interviewers' face, this has happened before.
I don't mind being put to the test, but I don't like being ambushed.
-- Karma whore? You betcha. --
"Do this arbitrary coding problem" is usually a good sign that you don't want that job anyway. Such things have little relevant to the job or to producing good software most of the time.
The only exception is on embedded where you actually do need to know the ins and outs of the compiler and stuff like that. But for Javascript or C# devs they should use String.Reverse() because whatever they code on their own will be worse and a waste of time. And even for embedded I would only ask by way of some questions about some sample code the candidate was showing me.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Answers are never that important to me as an interviewer unless I am verifying they possess a certain skill. I am much more interested in how conscientious a candidate is in looking for the right answer. It is predictive of how they will solve real problems when they are unsupervised later after being hired.
Quoted in full for emphasis.
It's not always important to have a problem answered fully or correctly in the allotted time. It is far more important to see *how* the candidate tackles the issue, what steps they take, what mental tools they bring to bear, and most important of all, knowing when to ask for more information and/or assistance (and then paying attention to what they're asking for, and the reasons they state as to why.)
I used to do this back when I dealt with just sysadmins. One of the fun bits was to chattr +i a config file, but then have them alter it as part of the testing (but not give them the permissions to fully do that).
The point wasn't to torture anyone, but to see how they handle non-ordinary problems. To watch how they tackle it, how they determine what's going on, then know whether or not to ask for help - and to see if that request for help is specific ("you'll have to change the attributes back before I can update it"), or if it was vague.
That told me more about the skills of the candidate than whether or not they successfully knocked out a problem.
Quo usque tandem abutere, Nimbus, patientia nostra?
I think a big problem with these questions is not if they are truly hard or difficult, but the meta thinking involved. Many of them really come down to 'which trick did the person who designed this test have in mind?' or even worse 'which niche within the language is the person who wrote this obsessed about?'.
I have found it useful to give them questions that they may not know.
I had a test that seemed to be rather good at judging ones ability. And it took people an average of 4 hours to complete.
Question 1. I have an HTML file with a Picture of two boxes overlapping each other, and a box with two empty Div's asking them to make the other box look like the picture. (Non Web people will need to google to learn how to use style sheets, and position properitys)
Question 2. A simple Application form, that asks for the address, validates that the format is correct, and just pops up a text box giving the errors or showing the address in a proper US mailing address format. (The zip code with leading 0's gets them every time. )
Question 3. A SQL Stored procedure, that returns a table where the rows become the column. The code works correctly, however there is a null in the data, preventing it form working correctly. (So they either will need to do something with the null, or exclude it, extra bonus points if they state how insecure that code was.)
This seemed to have allowed the company to get some good employees, but still it isn't fool proof, a few people slipped threw the cracks, mostly because they actually had experienced those particular issues before, so they knew how to handle them, but turned out they would struggle on new problems.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
I've conducted dozens of interviews over the past 10 years, and the best predictor I've found is simply to talk to candidates.
I typically ask questions like:
What's your favorite *nix OS / distribution, and why?
What was your worst day, and how did you recover?
What was your greatest success?
What's your methodology when you're asked to start a new project from scratch?
Key is to go into detail on each of the points, and listen for confidence, honesty, bullshit, and specific words. Test individual responses. Get down to syntax, command arguments, file locations, protocols... but try to keep focus on what they've done.
So far out of the 8 people I've hired/recommended, only one didn't work out.
A government is a body of people notably ungoverned - AC
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
I just rebooted and forgot to login to /. so this is repeated AC post.
Mimetics Inc. Twitter
My biggest problem with your approach is I don't have any code in github. A lot of professional programmers don't. I program 50-60 hours a week for a living, the last thing I want to do when I get home is program. And the company owns all that I code, and they don't put it on github. Any programming I do at home is typically related to learning new languages and libraries, and to put it mildly, none of that is worth publishing anywhere.
It shouldn't be about whether or not a person can program a string reverse, it should be about the entire team watching the new interviewee's process of writing the code on a white board (after they have interviewed him personally two or three at a time) to get a feel for his bullshit to brilliant ratio. (or his brillant to brilliant ratio if you're a TDWTF fan).
Unfortunately when I was at a company that did this, it was during the dot-com crash of the early 2Ks, so we only had one hiring round while I was there. Our two programs were reverse a singly-linked list and copy a file. Watching recent CS grads who were weaned on a diet of Java trying to do file copy (it's important in C/C++ to know how to sling data around in buffers) was such a train wreck. In my case, I knew the basic file copy inside and out, including the performance implications of buffering. The CS grads were having trouble with the basic idiom of read a byte, while not EOF, do something with the byte. It was the EE grads that actually knew what the fuck they were doing.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
If you can't program at all when under stress, that's not going make you a very effective programmer in the real world.
In the real world programmers don't have stress.
Most of us don't live in the United Retarded States of America.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.