Slashdot Mirror
Programming Interview Questions Are Too Hard and Too Short (triplebyte.com)
Programming interview questions can feel unnecessarily difficult. Sometimes they actually are, a new study has found. And this isn't just because they make interviews excessively stressful. The study shows that harder programming questions actually do a worse job of predicting final outcomes than easier ones. From the study: Programming under time pressure is difficult. This is especially true during interviews. A coding exercise that would seem simple under normal circumstances somehow becomes a formidable challenge under the bright lights of an interview room. Stress hormones cloud your thinking during interviews (even though, sadly, neither fight nor flight is an effective response to a menacing programming problem). And it can almost feel like the questions are designed to be perversely difficult. I actually think this is more than just a feeling.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
take this string, now reverse it, now convert it to pig latin, now reverse that.. How many internal threads are used to perform writes in a ConcurrentHashMap, now bark like a seal..
Answers are never that important to me as an interviewer unless I am verifying they possess a certain skill. I am much more interested in how conscientious a candidate is in looking for the right answer. It is predictive of how they will solve real problems when they are unsupervised later after being hired.
I shortlisted for an interview and got ambushed on my arrival. There were 19 other candidates, and we're all ushered into a small room with 20 desktop spots on a table that went around the entire room. We were handed one single sheet of paper with a coding problem we were to find a solution for. All 20 workstations were Mac Minis. None of us were told there was a programming exercise, none of us were told it was a Mac shop.
I walked out of the "interview" in disgust. Eleven others went with me. By the looks on the interviewers' face, this has happened before.
I don't mind being put to the test, but I don't like being ambushed.
-- Karma whore? You betcha. --
Answers are never that important to me as an interviewer unless I am verifying they possess a certain skill. I am much more interested in how conscientious a candidate is in looking for the right answer. It is predictive of how they will solve real problems when they are unsupervised later after being hired.
Quoted in full for emphasis.
It's not always important to have a problem answered fully or correctly in the allotted time. It is far more important to see *how* the candidate tackles the issue, what steps they take, what mental tools they bring to bear, and most important of all, knowing when to ask for more information and/or assistance (and then paying attention to what they're asking for, and the reasons they state as to why.)
I used to do this back when I dealt with just sysadmins. One of the fun bits was to chattr +i a config file, but then have them alter it as part of the testing (but not give them the permissions to fully do that).
The point wasn't to torture anyone, but to see how they handle non-ordinary problems. To watch how they tackle it, how they determine what's going on, then know whether or not to ask for help - and to see if that request for help is specific ("you'll have to change the attributes back before I can update it"), or if it was vague.
That told me more about the skills of the candidate than whether or not they successfully knocked out a problem.
Quo usque tandem abutere, Nimbus, patientia nostra?
I think a big problem with these questions is not if they are truly hard or difficult, but the meta thinking involved. Many of them really come down to 'which trick did the person who designed this test have in mind?' or even worse 'which niche within the language is the person who wrote this obsessed about?'.
I've conducted dozens of interviews over the past 10 years, and the best predictor I've found is simply to talk to candidates.
I typically ask questions like:
What's your favorite *nix OS / distribution, and why?
What was your worst day, and how did you recover?
What was your greatest success?
What's your methodology when you're asked to start a new project from scratch?
Key is to go into detail on each of the points, and listen for confidence, honesty, bullshit, and specific words. Test individual responses. Get down to syntax, command arguments, file locations, protocols... but try to keep focus on what they've done.
So far out of the 8 people I've hired/recommended, only one didn't work out.
A government is a body of people notably ungoverned - AC
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
I just rebooted and forgot to login to /. so this is repeated AC post.
Mimetics Inc. Twitter
My biggest problem with your approach is I don't have any code in github. A lot of professional programmers don't. I program 50-60 hours a week for a living, the last thing I want to do when I get home is program. And the company owns all that I code, and they don't put it on github. Any programming I do at home is typically related to learning new languages and libraries, and to put it mildly, none of that is worth publishing anywhere.
I'm not surprised HR raised holy hell over that they were 100% right to do so. Only people in fairly narrow niches have time to do take finye coding tests. It skews heavily towards the young and especially those without dependents. Older candidates tend to have more in their lives outside work which preclude doing such tests.
And older, more experienced candidates know you don't do work for free.
They don't know you or your company. You're spending zero effort to ask them for a lot. They have no indication you'll put anything like a commensurate amount of effort into evaluating their work. We've all seen jobs where the winner was selected even before the advertisement. Who on earth but the naive or desperate wants to risk their work on that.
There's no guarantee that you'll get a fair shake if you have an on site, but you know that the company is spending about as much time as you are, so there's a good chance they're not simply wasting it.
SJW n. One who posts facts.