Slashdot Mirror
Programming Interview Questions Are Too Hard and Too Short (triplebyte.com)
Programming interview questions can feel unnecessarily difficult. Sometimes they actually are, a new study has found. And this isn't just because they make interviews excessively stressful. The study shows that harder programming questions actually do a worse job of predicting final outcomes than easier ones. From the study: Programming under time pressure is difficult. This is especially true during interviews. A coding exercise that would seem simple under normal circumstances somehow becomes a formidable challenge under the bright lights of an interview room. Stress hormones cloud your thinking during interviews (even though, sadly, neither fight nor flight is an effective response to a menacing programming problem). And it can almost feel like the questions are designed to be perversely difficult. I actually think this is more than just a feeling.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
Interview questions are designed to be hard. Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar. And for most companies that means asking hard questions. Intuitively this makes sense because harder questions seem like they should result in a more rigorous screening process. But intuition turns out to be a poor guide here. Our data shows that harder questions are actually less predictive than relatively easy ones. Further reading: Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process.
It sounds like what they really mean is more time consuming
What came first? the egg or the chicken?
take this string, now reverse it, now convert it to pig latin, now reverse that.. How many internal threads are used to perform writes in a ConcurrentHashMap, now bark like a seal..
I shortlisted for an interview and got ambushed on my arrival. There were 19 other candidates, and we're all ushered into a small room with 20 desktop spots on a table that went around the entire room. We were handed one single sheet of paper with a coding problem we were to find a solution for. All 20 workstations were Mac Minis. None of us were told there was a programming exercise, none of us were told it was a Mac shop.
I walked out of the "interview" in disgust. Eleven others went with me. By the looks on the interviewers' face, this has happened before.
I don't mind being put to the test, but I don't like being ambushed.
-- Karma whore? You betcha. --
who is too soft and too short
I went to an interview and was handed a programming test to implement a web based vending machine. The test also required implementing a service that mocked a credit card processor.
All of the requirements for the vending machine were on a single 8.5x11 sheet of paper 12 point single spaced.
After 4 hours of programming I went to the interviewer and said I'm not going to finish this today. I found out later they normally give this test to candidates a week before the interview and have you email in your solution.
Small hint: sometimes the answer is not what they are looking for...stress handling, bullshit-o-meter, implementation ideas, probable likability, etc...
Only once I had an interview where I didn't get the job (as a software dev - back when I was a junior, was totally not prepared). The key is to be prepared and actually tell them you don't know and/or you don't want to bullshit them if you don't know the answer.
It's pointless to ask anything other than a basic programming question during an interview. IF you insist on verifying that a candidate knows the language they claim, do that OUTSIDE the interview. You can present them with a "how do you fix this error" or give them a bit of code and ask them what it does, but any detailed questions like "Solve this problem using recursion" really doesn't tell you anything.
Personally, I ask what seems to be a programming question, but really isn't. I ask them how they'd go about writing some algorithm and then start throwing system integration tests that are failing at them, their coworkers not being available to fix something and a deadline they don't think they can meet. The point is to find out how open they are to ask for help, how readily they will report problems meeting schedule and how they choose to interact with uncooperative coworkers. To me THAT's the stuff that is more important than being able to implement a swap sort on a linked list using recursion.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
I do a lot of rapid fire, easier questions when I interview. A person who is full of BS will get tripped up because you have to have a working knowledge of the subject to keep talking through rapid fire answers,.
Laws are rules for the court, but merely a bottom bar to hit for life. Think beyond laws in your actions always.
Go figure a tech recruiter thinks interview questions are too hard.
TripleByte's interview questions are language syntax trivia questions, and don't tell me anything about a candidate. And of course their data show hard questions aren't helpful either. They run a website where you do take-home coding questions, and they check for the correct "answer". Too bad the entire point of whiteboard coding questions is to see what their collaboration style is and if they can handle pressure. Neither of which you can get from their question/answer format.
Why do people go to college to study information systems; analysis, design, project management, requirements, QA, etc. only to be stuck in a corner to code bits and pieces of a system they really have no control over?
What that makes you is pretty much the dude on the auto assembly line screwing on brake drums. You may be very good at it. You can probably put brake drums on any car ever made. You can do it blind folded and you can tell by feel if the drum is unbalanced.
But you are still just screwing on brake drums.
But hey, we need brake drums on every car, eh?
... "Talk is cheap. Show me the code."
This might not work for all developers in all situations, but one of the best ways to establish credibility as a developer is to find an Open Source project you like and contribute.
When it comes to the technical aspects of an interview, any technical interviewer worth their salt should be able to go to Github and verify that you are the person who committed the code you claim to own. Take along hard-copy, annotated if necessary, and be prepared to talk to it, to explain why it is something special, elegant, efficient, and flexible.
If you can get testimonials from project leads or other contributors, to demonstrate that you're a good team player, so much the better.
There are several advantages to this approach:-
1. The body of work that you build up stays yours and can be taken from job to job
2. You can share this well in advance of an interview, giving your interviewer time to review it and thus lead to a good discussion
Yes, someone is going to argue that this approach could result in cheating and could allow you to claim ownership of code that isn't yours. This is not a new problem for the interview process to factor - and a good technical interviewer should be able to ask piercing questions able to determine if you really did write what you claim is yours.
It's a completely [utterly] different example of credibility, but I am for some reason reminded of that scene in the original, Black-and-White movie version of the "Dam Busters" raid from the Second World War. Barnes Wallace, the inventor of the bouncing bomb, needed the use of an RAF aircraft from which to test drop his prototypes. It quickly became apparent that he needed an actual Avro Lancaster bomber, but of course, in the middle of a war, these were in short supply. His response?
"Do you suppose that if we explained to your man in the Ministry that I designed the Lancaster that he might be willing to lend me one?"
The whole reason we hold interviews in the recruitment process is to try and establish whether the candidate has the chops to actually do the job. Nothing says that better than hard, real-world evidence.
I've gotten to the point I just don't do this nonsense. There's plenty of jobs out there. If they want to give you some silly, test, just walk out. If enough people do this, these stupid questions will go the way of the dodo.
A similar situation is seen in final college exams, where students are asked to answer/solve a difficult programming question against the clock. A too frequent outcome is that few students fully answer the question, while many very poor answers have to be opaquely scaled to meet an external requirement. Not the best mechanism to separate the sheep from the goats.
I was a lot happier when I bid contracts for work and stopped interviewing for jobs.
Being an employee sucks. The tax system is set up to screw over the working schlob. If you're smart enough to program well, you're smart enough to hack the tax system too. Programming is the only profession that has anything like trivial tests in interviews. The flip side of this is without professional credentials, that's where things go.
Otherwise.. stand up and do tricks.
Unionize already. Doctors are unionized. Lawyers are unionized. They call it something else, but that's what it is..
..don't panic
I think a big problem with these questions is not if they are truly hard or difficult, but the meta thinking involved. Many of them really come down to 'which trick did the person who designed this test have in mind?' or even worse 'which niche within the language is the person who wrote this obsessed about?'.
A lecturer asks students to learn the phone book by heart.
The physics students ask: "Why?"
The medicine students ask: "Until when?"
There's a longer version, but you get the point. Apparently some of these "hard interviews" are just testing how fast you can cram irrelevant knowledge into your head every sane person would look up when they need it. That someone knows how to write down a quick sort algorithm in c with every semicolon in the right place doesn't mean they know when best to use it.
And now for something completely different:
https://www.youtube.com/watch?...
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
I would have thought that most employers might be interested in your ability to perform under stress as well or at least your ability to handle it? If you can't program at all when under stress, that's not going make you a very effective programmer in the real world.
Please prove P=NP.
I've conducted dozens of interviews over the past 10 years, and the best predictor I've found is simply to talk to candidates.
I typically ask questions like:
What's your favorite *nix OS / distribution, and why?
What was your worst day, and how did you recover?
What was your greatest success?
What's your methodology when you're asked to start a new project from scratch?
Key is to go into detail on each of the points, and listen for confidence, honesty, bullshit, and specific words. Test individual responses. Get down to syntax, command arguments, file locations, protocols... but try to keep focus on what they've done.
So far out of the 8 people I've hired/recommended, only one didn't work out.
A government is a body of people notably ungoverned - AC
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
A realistic test would be changing/updating/fixing something that you are familiar with, not being given something you haven't seen before.
That just doesn't sound like it's going to bring out the best in candidates.
The best approach that I've found to making programming questions part of the selection process was having qualified applicants do the test prior to the first interview. We selected a few standard programming algorithms (Quicksort, Dijkstra's Algorithm and Hash table create) that we either gave the candidate broken code or the prototype which needed the code. The candidates were given one week to submit the tests at which point they would be reviewed and we'd decide whether or not to grant the candidate an interview. The review was based on operational correctness (did the code produce what we expected from given inputs) and performance (generally speed).
HR (this was at RIM, about ten years ago when they would hire anybody with a CS degree and was apparently breathing), raised holy hell as it eliminated at least three quarters of the applicants without us even talking to them. Well over two thirds of the candidates never submitted the tests with most of them saying "I graduated from ... and where do you come off seeing if I can program?" and others saying, "I'm too busy in my current job to do this." with HR supporting those arguments. About 10% came back and asked if we had any other tests - these 10% always did the test we gave them successfully (and sometimes brilliantly) and these were generally hired and we never had performance issues with them.
The big problem we had was explaining to HR (sorry "Organizational Development", RIM speak) that just because you graduated with a CS degree, that did not mean you could program and that we wanted people who wanted to be challenged. When it came right down to it, they were getting hammered by their management because we were only interviewing 10-20% of their "qualified candidates" and felt that our approach made them look bad as other divisions within RIM wanted to follow our approach.
My response was "tough shit" and lead to some interesting meetings with senior VPs at RIM.
I just rebooted and forgot to login to /. so this is repeated AC post.
Mimetics Inc. Twitter
I've talked with several programmers here in Silicon Valley, who seem to state that HR and management wants you to code a whole project for nothing.
https://www.youtube.com/c/BrendaEM
I've seen it far too many times. "Engineers" calling themselves "full-stack" but they suck complete ass when it comes to writing production code. These are engineers are ex-Amazon, ex-Google, and one guy who went on to Facebook. I have ZERO trust in most engineer's ability to code.
In the interview process just determine if someone can talk to talk. Save the proof for the first 30-90 days so people can work under normal circumstances. Some people suck at interviews anyway. For myself, I always bomb the easy questions because they are usually nonsense the compiler will catch, minor syntax, minor implementation semantics.
Unless you are hiring entry level these things are all minor noise and what separates talent from drones is higher level thinking. You are going to end up with someone who fits the old stereotype of an Indian firm that can only implement what you tell them verbatim but not think independently otherwise. At that point you might as well be outsourcing.
Problem solving is important but having a good work ethic is at least as important and most interviews do little to identify that.
How about ask questions that don't have definite answers but need technical skill to answer. I find these very useful both for getting a feel of a candidates depth of knowledge as well as there ability to reason?
I'm thinking things like: So you are a C# programmer. What characteristics of C# attack you to the language AND which ones do you find most annoying? What type of problems do you feel C# is best at solving and which would you recommend at least considering another language.
( just a hint, if you can't answer those type of questions about any language you program in , or at least come up with a descent opinion that you support with valid technical reason, you can't really claim you understand anything about the language.)
it's a sign of how bad the job market is for programmers that they can pull this crap. Thing is, even if those 9 bail they can just go to Congress and get more H1-Bs ("But. But. But we can't find anyone qualified!").
Heck, the current administration just added another 20,000 PHD candidates to the H1-B cap (they sold it as a benefit to workers because they'd require more PHDs ignoring that those PHDs are in addition to the existing 65k cap. Got away with it too...).
I'm really disappointed. I didn't expect much from this current crop of politicians, but I thought at least they wouldn't increase the caps. Heck, the H2-B (low skill temp labor) caps are getting doubled.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
I check to see if they spend enough time to understand the problem, do they ask questions to make sure the specs are, I like it if they ask, "do you want a fast algo or minimum memory algo?". I ask them think out aloud, and guide them towards the solution. Then the fun starts, I change the spec, I ask them to reimplement it optimizing for speed instead of memory or vice versa. Change it from "filter out odd numbers" to "filter out numbers divisible by another set of numbers", etc etc.
Right at the beginning I tell them, the test is not on programming. The test is on "Am I able to communicate well with you, and are you able to communicate back with me?"
I hire only PhDs and Masters from US universities, coding is necessary but not sufficient for my positions.
So, yes, there are hiring managers who would happily go through the same hiring process themselves.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
During an interview I'd realised I didn't want the job, so when they sprang an unexpected programming test on me I asked them for the project plan and the business case behind it. They said it was to check how well I could program, I replied that I want to check how well they're organised and what their policies were. It went on like that for a few minutes until the interview ended.
... it's idiotic.
Asking obscenely hard questions during an interview only makes the candidate think you are unreasonable in your day to day demands. A difficult problem in program design requires a lot of thought time. An interview is not the place for that.
In an interview I want to know that a candidate can plan their approach to solving a problem, so I generally ask, "hey, what's your approach to problem solving?" A person's answer will betray a lack of actual experience and ability every time.
My preferred approach is to use a question that has many different variations, and to start with the easiest variations and then ramp up the difficulty/complexity based the candidate's performance. I only have the candidate write code for one variation, usually the second-hardest one that we talked about, and then ask them how they would go about testing what they implemented. This approach has many advantages.
First, it lets me see how mentally agile they are. Weaker candidates have a hard time adapting their approach as the problem changes, and have a tendency to get locked into early solutions. Better candidates can see how the later variations offer more/different options/tradeoffs and can adapt.
Second, it lets me see how they think through easier variations, to get that strong "process" signal TFA mentions. This gives me good insight into how they approach problem-solving tasks. In some cases it also gives me a clue as to their honesty/integrity, because it's pretty clear when someone has already seen a simple variation of the problem and doesn't admit to it (during my intro I point out that if the candidate has seen the question before, they should let me know).
Third, it gives all but the very weakest candidates some early success, which helps to lower their stress level. There's a limit to how much I can do in that regard, because interviews are inherently stressful situations, but I think it does help... and I think people perform better when they're less stressed.
Fourth, it lets me tune the difficulty of the interview to the candidate's ability level. This is for the candidate's benefit, not to give me any information. Since I'm just one in a slate of interviews, unless I'm the last one the allotted time is fixed and there's no practical way to just cut the interview short (unless they're so bad that I feel I need to end the day entirely to avoid wasting everyone's time; this is very rare). This means that if the candidate is weak and I give them a hard problem, they end up spending the whole hour struggling vainly. This makes for an unpleasant hour for them, an unpleasant hour for me, and it increases their stress level for the next interview. If the candidate finds the first variations of the problem challenging, I just stay at the easy level.
This difficulty-adaptation happens very naturally, because I don't move on to a harder variation until they've solved the previous, easier, one, with whatever hints/tips I need to provide. I watch the clock to decide when to have them change gears from discussing increasingly-harder variations to writing code. On occasion I underestimate the amount of time it will take them to write code and have to tell them to stop before they finish. This happens most often with PhDs, actually, who are often bright and good problem-solvers, but don't have much coding experience -- often even after years in industry. On those occasions, I apologize for not allowing enough time, though I know it's still pretty painful for them not to be able to finish. Whether or not they finish doesn't affect my evaluation much. I get most of what I want to know about their coding skills in the first few minutes of watching them.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Definitely agree about the easy questions. When candidates can't answer easy questions correctly, that provides a strong indication that the candidate is a bad hire.
When candidates can't answer hard questions, what does that tell you? They need extra time to solve difficult problems? Who doesn’t?
When I interview candidates, I usually give one problem that's designed to measure how much of their chosen language they know. While I don't test for knowing language trivia, I do look for signs if they prepared for the interview. I tell them upfront that if they can't remember the arguments to a function to please ask, but if a pattern of unpreparedness appears, that's a minus.
I've had candidates that have outright refused to code. They'll sometimes even write pseudo code that describes a solution, but when I try to get them to translate it into their programming language, they refuse.
Others try to show off their coding prowess and usually fuck it all up in the process by increasing the complexity. I tell the candidates upfront to KEEP IT SIMPLE.
I don't require a complete solution for a hire recommendation, but there are a couple of key milestones they need to make.
I want them to show they can program, and can put a little thought into different approaches. I try to give hints along the way if they get stuck, or head off into the weeds. Giving hints is important, because it doesn't do anyone any good if the candidate is just standing there. If the candidate can take a subtle hint and incorporate it, that's a plus, because I also want to see how they work in a group or team. I've actually had candidates who thought I was lying to them when I've tried to give hints.
Anyway, the point is, in today's world of global collaborators, you can't just test for the candidate's knowledge of programming trivia, you've got to know if they can work as a group and learn from others. Being able to write working code is important, but so is teamwork.
FWIW, I recommend Hire about 3% of the time.
- Use conflict and confrontation during the interview to test the candidate.
Joel on software's interviewing guide https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guide-to-interviewing-version-30/
It's unprofessional to argue with someone you are interviewing.
It's unprofessional to use your position as interviewer to try to get someone to adopt a known false idea
It's unprofessional to advocate others to do so
It's unprofessional to group interview someone, abuse them and disrespect/mock them during an interview. It's not a court and you are net the district attorny
It's unprofessional to ask language lawyer questions, follow up with "Wow, everyone knows that" comments to assert your superior knowledge of the interviewee during the process. One wouldn't ask an interviewee a question and then follow up with "Dang, you are stupid for not knowing that" for each missed question.
Funny is I've seen these from programmers in their mid 20s which know 1 platform and are at near expert knowlege of that single platform.
I used to love the idea of programming-intensive interviews, as a way to objectively assess my most-relevant-to-the-job skills. After doing quite a few them, I realised about my mistake. A theoretically excellent proceeding becomes really bad when being terribly mismanaged, what is a surprisingly common scenario in the IT/programming world.
I never had to solve a really difficult problem, but lots of them under unreasonably short time constraints (i.e., the holy grail of arbitrary, cheating-prone, useless-parrot filters). Now, I prefer to avoid these situations, unless when knowledgeable people and reasonable conditions are involved.
Custom Solvers 2.0 = Alvaro Carballo Garcia = varocarbas.
I'd like to see data on this statement:
"Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer..."
How is that even knowable?
never understood these pressure-question things - you don't have your computer, your browser, its history, your bookmarks, your printed books - basically you're taken out of your working environment where you code - imagine anyone else having to do that - a musician auditioning without sheet music, musical instruments, or anything - a doctor having to do surgery in a kitchen - no other professional I can think of is taken out of the normal working environment and asked to perform professional work
Its about quality and consistency, same as any other engineering profession. It is were seldom about solving a tricky problem. Its about producing high and even quality that is predictable and stable. The algorithms you get from a mathematician, they are the best at it anyway,
I've had my share of whiteboard interviews and in general I've found they fall into two categories.
1 - A basic scenario to show you know the broad outlines of what needs to be done.
2 - Some esoteric problem that requires knowledge of obscure functions and is a pet project of an interviewer.
The first kind is useful. It lets them know how you think and that you know what's going on. The second one is a dumpster fire you're never going to answer to their satisfaction so you might as well thank them for their consideration and walk out the door.
My personal favorite interview story about the second was actually a logic question. The problem itself was flawed in that they didn't give a critical piece of information necessary to solve it. There was an assumption that was quite literally wrong. When they smugly told me how I should have solved it I pointed out that error. Needless to say I didn't get the job. Also needless to say I told everyone I knew about their lack of proper requirements for problems and to avoid them as an employer.
As a new developer I find these coding challenges particularly frustrating as I feel they don't demonstrate my ability to ship working code. That being said there's a wide gap between doing a fizzbuzz and reversing a binary search tree. How do I become better?
Wish I had some mod points for you today, AC.
Hey, Windows users, there is no such thing as "forward" slash, there is only slash and backslash.
... if they are hired on the basis of their ability to memorize stuff that can be looked up in two seconds.
A strong team can teach newcomers the right skills. It is more important to find the right personality/fit for the team.
The fact that you made the interview, likely means you have the skill.
I can teach you to code. I can't teach you to not be an *sshole.
What we do is provide a list of about 6 (depends upon skill set we're hiring) problems that equate to first or second year CS studies. They are all straightforward and require no fancy manipulations. The candidate picks one. We set them up with an environment, they can access Google (keep search history intact, no Googling the actual problem).
They are given an hour to do the problem; anyone on the team can do it in fifteen minutes or less. If they're having trouble, we will give them another hour. There is a proctor present to help with non-programming issues (for example, for C++ folks we'll help them compile if they're rusty with command-line gcc).
These types of simpler exercises provide insight into some of the fundamentals: can you write code? What methods do you use to solve problems? Can someone else easily understand your code?
The number of C++ 'programmers' who can't compare two lists based upon criteria, or web programmers who (no joke) can't code a page that dynamically retrieves data using a (provided) web service, or database folks who can't join two tables is staggering. The exercise weeds out people who have spent years - decades - copy/paste/trial-error coding. Sorry kids, we have real work to do.
take this string, now reverse it, now convert it to pig latin, now reverse that.. How many internal threads are used to perform writes in a ConcurrentHashMap, now bark like a seal..
When I interviewed with them in the early '90s, they gave a "programming" test that wasn't too far off from that. It was the most ridiculous thing I have ever dealt with. Converting from one pseudo language to another and then look up things on a chart and then convert back to a pseudo language.
WTF?!
I got fucking migraine about 2/3rds the way through and had to stop before I puked. It was just nonsense busy work. I got an interview afterwards by what would have been my direct report and he didn't look like he slept in a week.
Get this, they were a goddamn Microsoft Visual Basic shop. And they later got hacked so bad that everything they had was taken. So don't tell me that those tests get you the best.
As someone who has led offshore teams and teams of junior developers I disagree with the assertion that "bad" engineers are more costly than losing a qualified applicant. I work with less than perfectly qualified developers all the time and they rarely fail to deliver when given proper leadership. If anything, the hidebound senior developers who are unwilling to learn new things are far less productive. The technical debt incurred by those who just want to skate by is much more than those who are misfits trying to prove themselves.
google has become a crutch, and you can't use it during the typical interview.
Instead of a "programming" question, I often want to determine how they think. I can hire a code monkey anyday. Those positions are generally my college students and recent graduates. After I hear how they solved a particular issue and whether the candidate can relate to biological life forms, I ask a logic question. First to see how they handle the pressure and second to see what they come up with. I rarely get a "correct" answer but the process the candidate goes through is always telling. Here's one - "You just bought a hammer and a nail. Together, the hammer and nail cost $1.10. The hammer is one dollar more than the nail. How much was the nail?"
Another, "why are manhole covers round?"
The Kai's Semi-Updated Website Thingy
The only time I've had to hire someone, I told the local college and universities that I was hiring. Got a boatload of applicants, and after the first couple of interviews it was obvious that these guys hadn't much programming experience at all. So in order to weed out candidates I made a test, of some terrible code. Basically pointer arithmetic and weirdness. Most recruits gave up after a minute. One guy worked through it. He got the wrong answer but I could tell by the way he was working on the problem, he understood how things worked, and was methodical.
He got the job of course, and turned out to be a fantastic employee.
So I would submit that the questions aren't being asked to show what you know, but how you work through the problem.
Maybe if your interview processes were sensible you wouldn't reject suitable candidates based on ridiculous non-essential criteria.
When I graduated years ago and was looking for a job, I would walk out of interviews that did this nonsense.
Sometimes stress can be overwhelming. I took a test for a non-programming job using Quark Xpress back in the day (remember that app?) which I had known very well, but hadn't used in about 6 months.
I sat down at the testing station to do some simple text/image insertion, flow, leading, etc.
I literally could not remember a damned thing. I just sat there helplessly for 10 minutes trying to remember how to use the program.
Eventually I gave up and told them I'd have to do some refresher study before I could meet the requirements. It was embarrassing as hell.
It's easy to marry, difficult and expensive to divorce. Maybe we're solving the wrong problem :-)
Hiring good people is a common problem that good answers are difficult to come by. I was trained once to (basically) ask "5 whys" whenever somebody said they could do something --- in order to determine how much THEY did vs they were present when somebody else did the work. Each level down was intended to determine whether this person what making crap up or had really worked on the task.
I also knew a guy who gave logic/thought questions - he wanted to see if the person could think. He later recanted his line of attack as being useless as judging skill level. He'd ask "how many quarters stacked does it take to be as tall as the Empire State building?" To which he expected questions such as "on their side, end to end, how wide is a quarter, how tall is the building?" Did they seek requirements clarification and how did they solve the puzzle?! (its a problem in estimation, there is no exact answer--the goal was to understand how they arrived at it). But many people could solve it and still not write working code.
But I've learned recently that asking somebody to write a program in the language they claim to know is important. Why? Because there are a lot of script people who cut/paste and ferry modifications/customization and have a cursory understanding of Coding, Logic, and Problem solving. Just because you can copy somebody else's code and make it compile does not mean you can code. You know Java? great, so do I. Write a simple program that solves something domain specific (don't worry - I'm not here to see if know J2E or ask trick questions). C#? even better, that's what we use here.. Write a simple program....
If you have the skills - we'll train you and help you grow beyond where you're at today. No trick questions. You know Java and we need C#? Fine - want to learn C# --- we'll train you.
What they don't know is that I'm interviewing them, too.
Does the manager know what he is asking when he reads a question and then has a conversation about it, or is she looking for me to parrot whatever is written down? (Heh! I don't know their gender)
Are they trying me to get me to do the job as part of an interview? I have literally had the question, "Set up a framework to automate the testing of http://www.xyz.com/".
Are they looking to show me how much they know and how insignificant I am? Usually, this takes the form of, "Can you answer this obscure comp sci question that you'll never actually ever use?" or "Do you know this goofy trick that uses an obscure language feature in a non-standard way to solve an even more obscure problem?"
If I get a hint of any of that, I walk out and tell the interviewer that their hiring process is stupid. I've been around enough to know that working there will suck anyway. The interview should be a discussion about what is on my resume, to confirm that I did what I claimed. I wrote the damn resume so that they could decide if my experience matched up with what they need. If my experience doesn't fit the job, why did they call me in to test my "test taking" capabilities? Because, they're numb-nuts that don't know what they're doing, and I don't want to work with numb-nuts.
The best interview question is the one you ask at the very beginning. "Which part of my resume makes you think that I am a good candidate for this job?"
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
It was a mac shop. You have been warned!
Idiot very much?
It was a programming job. There is no better OS/computer for programming than a Mac. Linux is miles behind and MS Windows lightyears. Only some obscure Unixes are somewhere clustered around Linux, like HP-UX and AIX ... and Solaris.
But I guess in your blinded mind you tried to make a lame joke ... you failed.
I code on Mac, Windows and Linux. Coding on Mac is a dumpsterfire for anything non-trivial. Even Cygwin on Windows is a better user experience the the Mac.
Hey Im going to say outloud a question about a database that doesnt exist, nothing else, and expect you to write, on a whiteboard, how to solve that question.
How about instead, you give us the actual DB, and say I need the result set to encompass X, Y, and Z. Then let us connect via whatever tool we know (first signal we might be clueless) and actually use the actual tool you want us to show competence with.
thank you for the captcha btw, the word was "idiotic". How ironic.
HR understands this technical stuff is worthless, nerd talk.
In their lofty awareness, they would like you to transcend hiring laws and just ask this single, simple question:
Are you 40 or older?
Elegant, free from leadership reproach, and so helpful.
CS students please don't let this deter you from pursing this career path because their job is already so very hard.
Next, HR would like to tell you about how force.com can make your job a lot easier.
We gave applicants for my team a request to create a "Hello World" short application that spit out the text one character at a time using C#. Every applicant failed. They over engineered what was needed or didn't actually produce the output we requested.
All we wanted to see was "Hello World" on the console printed one character at a time.
So, there's an old joke about how the trustees of a university want to find out if the professors know their stuff. They come up with the question "What's 2 plus 2?" as the test. First, they go to the math department and ask the question. The professors say, "Oh, that's easy. It's four." Then they go to the physics department. Those professors say, "Oh, it's 4.0000000 with an uncertainty of another decimal place." Then they go to the college of engineering and ask those professors. They say, "Just a minute, let me get my handbook."
Point being, that trying to program from memory is a waste of time. Trying to write code quickly is also a waste of time. One should be thinking about the approach and the logic rather than regurgitating the details of some function call. It's entirely possible that there are multiple solutions to the same problem.
The fourth part of the joke is when they go over to the business school's accounting professors and ask, "What's 2 plus 2?" The professor looks around to make sure nobody can hear and whispers, "What do you want it to be?"
Were you expecting to be hired as a programmer without demonstrating that you can program?
Ah, listen up PHB! (You've said MANY times that you're a manager at some Silly Valley dipshit company.)
Your test is blabbed all over the place in the Asian "network". They copy it and memorize it. And even if "they" don't, someone who's studied and MEMORIZED algorithms and patterns and OTHER'S code could solve an employer's test easily - including yours.
Wait, you say! You "change" it! Ahahahahaahah!
Programming problems can be memorized - aka "Patterns" - and it doesn't require much creative or even intelligent thinking.
How do I interview and get qualified people? That's my secret sauce. And it's amazing how many "rejects" I get (*cheap) that do quite well - thank you very much.
*Sorry about the cheap part. But we employers pretty much control the market. We say we need purple pants wearing people eaters as recruits and many jump to train to eat people and buy purple pants.
Are we right? Fuck no. Do we think we're right? Well, that ONE purple pants wearing people eater turned into a rock star developer and we made the logical error in thinking that purple pants wearing people eaters are the best!
tl,dr; employers are idiots.
Feel through an assessment task because I couldn't come up with a viable Fibonacci generator and some services built around it fast enough and the one I finally managed to tape together was too clunky - which I was perfectly aware of. Having programmed for 34 years I felt quite silly and discovered that these questions are just as stupid as they were 20 years ago.
The other interview was a sweepingly broad question about how I would build a Microservice architecture Webshop by some 28 year old nerdy IT lead douche. Of course my reply didn't satisfy him.
These questions get asked by three types of people:
1) frustrated nerds in lead positions trying to prove to their own camp how smart they are and how dumb all potential replacements are
2) people who are seeking a performant and obedient fall-guy for a project about to blow up
3) interviewers who have no idea what they are doing and probably have a general governance problem either already in place or slowly brewing
The truth is this: Anyone who thinks they can assess a senior or even medium position with a few questions or a two hour progging assessment is on crack. I might be able to do that after 4 weeks. And I know a thing or two about tech team management.
I'd walk out of an assessment should someone ask one of me again without warning.
We suffer more in our imagination than in reality. - Seneca
Where I work, (and I got the job at 40+, now in my 50's) we have an entirely different hiring policy and different techniques from almost every post above here. I read almost ALL of the above input, and I was not going to give my opinion in here, but after reading it all - I was wondering where all of that experience came from.
So this is what we do:
We don't care about your age, gender, preferences, race, religion, political beliefs, school background, grades etc. We just care about a few simple principles.
- Are you afraid of learning new things?
- Are you entirely honest? (Our hiring staff is VERY skilled at spotting someone who's not entirely honest).
- How do you fit in socially with others?
- Can you jump into something that seem impossible to solve, and admit you just can't do it, but will be humble enough to ask for help from your colleagues?
- Can you take responsibility and learn from your shortcomings and appreciate the talents of others? Admit your mistakes, and not be ashamed of it?
We don't care for the worlds fanciest resumé, we don't care about HR companies and hiring staff, we're the ones you're going to work next to every day of your life, we simply want to know WHOM we're sitting next to. You - as a person, not what you know or whom you knew.
We also use something called "mentoring technique", this means that someone in our team gets appointed to be your mentor during your hire here, our staff is you in the future, so we hope you adapt our mentality, not to point fingers at you and blame. We don't run the blame-culture here, we teach ourselves and learn from each other, very much self driven - and we're one of the biggest companies in the world.
This approach is immensely successful, because it doesn't matter to us what you knew when you came in, what matters to us is your willingness to adapt, drive yourself, your endless curiosity, your genuine interest in making life better for others (our end users), and see your colleagues as assets rather than the competition. Everyone work at their own pace, no one is whipped into reaching certain goals, they set their OWN goals.
This kind of freedom, makes people REALLY want to know what they're doing, because they take pride in having a team with this mentality, and no pressure from anyone encourages positivity and genuine interest in the craft they work with.
When I got the job, I was literally thinking, they must be joking - I did tell them the truth, I'm not that knowledgeable, I know SO many that I can't even compare myself to 1% of their skills, and I make mistakes ALL the time, but I OWN my work and my mistakes, and I'm never afraid to admit I was wrong, neither does my colleagues. We're open minded, totally ready for change - because our everyday consist basically of improvements and total change in some areas anyway, so we're used to that by now. We literally EAT re-organization for breakfast. But my colleagues have ONE thing in common, their passion for the company and their awesome colleagues.
I can't tell you where I work for obvious reasons, but right now - they're even expanding their staff of brand new coders, one of them is 55+ and have literally no coding experience. I came in too, knowing very little - years later, I've been told by my manager that I'm in the top 10 range, which I still find so hard to believe, but I take what I can get.
What I'm trying to tell you out there, with this wall of text above - is to just go for it if you dream of it, it's not impossible. Sure - it'll be DAMN hard, but challenges are fun, and that's probably why you would be interested in coding anyway, right? Age means nothing, attitude and passion is everything.
What this world is coming to - is for you and me to decide.
When I interview I start with a question that is relatively easy that anyone who can code could solve. Then follow that up with a question that is more difficult that builds on the previous answer.
For example, I would ask a candidate to write recursive Fibonacci function (I would explain what Fibonacci numbers are). If they are able to do that, then I would ask them what would happen if their function was called like this: "fib(10000);".
...richie - It is a good day to code.
"Because the cost of hiring a bad engineer is so much higher than the cost of rejecting a good engineer, companies are incentivized to set a high bar."
Bullshit!
Everyone in California is an at-will employee who can be terminated at any time for any reason or for no reason at all.
The entire system is set up to favor termination.
It is not the companies that are incentivized; it is the employees, and the management.
They are not incentivized by a desire for excellence; they are incentivized by a fear that someone better will be hired.
I give each of my candidates a simple test, something like:
- Write a command line program to accept a list of items as arguments.
- Divide the arguments into types (dates, words, numbers, etc.)
- Sort each list in an order appropriate for its type.
- Eliminate duplicates from each list.
- Display the results on the console.
I give them 2 days to complete and turn in their answers.
I'm looking for:
- Are all requirements met?
- Does it actually run?
- What approach was used?
- How simple is the code? (I'm looking for simple solutions, few lines of code.)
- How readable is the code?
- How well-structured is the code?
- Are there good unit tests included?
This eliminates about 75% of candidates. Those who pass, I bring them in and discuss how they chose their approach, among other things.
I've been shocked how few candidates can complete the test!
When I interview, I don't grill; I aim to recreate the normal working conditions as closely as possible because I want the candidate to acts normally.
Skills are never spot on, but can be taught, providing the candidate has a reasonable normal outlook.
I will ask an impossible question towards the end, because I want to see if the candidate is prepared to call out bullshit and how they do it, constructively.
The key to finding qualified people is asking simple questions about tasks developers do every day such as conditional statements, code check-in, and HTML/CSS. Who writes a sort every day? This has worked perfectly for me all but one time. Ironically, that person had a double Masters degree. They couldn't get the simplest tasks done.
But attitude is far more important than coding prowess. Flawed personalities kill teams.