Slashdot Mirror
Appropriate Interviewing For a Worldwide Search?
jellomizer writes 'I am a manager of a small Software Development department, looking to hire some more developers. By edict of the CEO, the search must be made globally, so we are dealing with different cultures and different ideas of truth and embellishment, etc. To try to counteract this, we give the potential employees tests where I watch what they do, to see if they actually know what they say they know. However, it seems a lot of applicants drop out when I mention that this test is mandatory. Is this a sign that we caught them in a lie, or are we weeding out good people where we shouldn't be? Would you be willing to take a test as part of an interview? If so, is there any type of heads up you would like to know beforehand to make the decision of whether to take the test easier?' What other difficulties have people seen while trying to hire from many different cultures?
would be to give them a real life problem, ask them to solve it, and tell them that they can ask you whatever they want to, because that's the way it works in real life. If they know the answer immediately, well ok, but really what you want to see is their problem-solving strategy. I firmly believe that it's not about what someone knows that makes them a valuable employee, it's how they figure it out. How they solve the problem. People who rely on the resources around them, generally speaking, are better to have around then people who think they have to have learned the answer in a textbook somewhere. If the nature of your job is such that answers are already known, then you don't need smart people. You just need workers. Such a test doesn't need to concern itself with being culturally sensitive.
I'm starting to think that our interviews here should literally be: give them a day's work and see how they do.
I know there are definitely people who refuse to take a test out of principle. I'm not really sure what principle this is; maybe it's that they do poorly on testing in general. But we have been burned too many times by people who know how to talk the talk but turn out to have very little real skill. Sometimes too, there are multiple similarly skilled candidates to choose from. Giving them a coding test; especially an open-ended one can give you some insight into the sort of developer they are. Some people will be a better fit for the team just out of the approaches they tend to take toward problem solving.
Also, tests must be taken in-person. We do not allow phone or otherwise remote test taking. There are a lot of really unscrupulous agencies and individuals out there. Some of them have you interview with a different person than who they claim to be, including the person who will take the tests if any. The guy completely aces your questions and really knocks your sock off with his knowledge. Then he shows up, and his voice sounds different. You put him in front of a keyboard, and he asks you which key is the "Any" key.
The thing is, it's no offense meant to the interviewee. Indeed, just as it protects our interests to be certain that we hire a qualified developer, it's in your interests too (if you are a qualified developer) - the fewer and more quickly we sort through the deadbeats, the faster we get a job to a person who deserves it. It's not that you'll necessarily lie to us, it's that there are plenty of people out there who will, and until we really get to know you, the only way to tell the difference is to require you to answer questions that only a qualified individual is able to.
Slay a dragon... over lunch!
Sorry, but when I did I saw them go screwy so many times. almost always of you didn't do it the predetermined way you were wrong, or if you answered with an answer someone didn't know, you were wrong..
Plus after 15 years I find it a tad insulting.
The Kruger Dunning explains most post on
The business owner was looking for a new receptionist, and couldn't decide which of they applicants to hire, so he decided to do a test. He accidentally "dropped" a $100 bill in front of each of them. The first just handed the money back to him. The second took the money, then came back later saying "I invested that $100 you dropped in oil futures. Here's your $100, plus $50 profit." The last slyly pocketed the money and didn't say anything about it.
Which one did he hire?
The one with the biggest tits, of course!
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Good luck with eating the department.
No, I wouldn't be willing to take a test, and I actually flat walked out on an interview in 2003 when I showed up and was told - by surprised - that I was going to be taking an exam. I was also then informed that the open position was for a junior position. When I expressed surprise at this, the HR flack's response was "Oh, didn't I mention that in my email?" She hadn't. Either of those would be sufficient for me to end the interview process, which I did.
Why would I refuse to take a test? Simple: if you're giving me a test, the usual reason is that I'm being interviewed by someone who does not possess the ability to discern whether I know what I'm doing/talking about or not. If that person is the hiring manager, then I certainly don't want to work there. Working for people who cannot identify competence or incompetence is not pleasant. If that person is not the hiring manager, I still don't want to work there: it shows they would waste my time by having me interview with such a person rather than with the hiring manager or any other person who can tell if I'm competent or not.
In our company, we work with offshore programmers.
Our selection process includes a mandatory test, during which we assess the candidate on several points, mostly: IT Skills, ability to understand requirements, motivation. In order to avoid cultural issues, we tend to focus on facts and we try to avoid questions which may lead to a culturally biased answer. For instance, we would ask: "please explain me how you will implement such feature" instead of "did you understand what I mean".
The test is a simple project, and the candidate can work on it at his/her own pace. They are followed by a project manager as in a real work environment. Its duration is normally one week as candidates usually have a day job. We renumerate the candidates for the test they take with us.
The recruitment process has been found to be effective in most cases, allowing to effectively select quality programmers. We found that there are enough programmers ready to go through our selection process for us not to worry about the one refusing to take a test.
I've never taken a job at a company that requires a second or third interview. If I have to make more than one trip to your campus I'll take another job before you get a chance to make an offer. Likewise, if you have any testing that requires more than a trivial amount of time on my part, I'll pass on the "opportunity" to waste my time at your company for free. If you want me to spend my time you'd better be paying me for it.
I have been put in such situations and I just politely end the interview process.
I don't work for free, and anyone that would expect me too is not going to be an employer I want to do business with.
If a serious offer has been made and you want me to write on a whiteboard, great. If you want me to email you a finished project that you get to keep, you are going to have to pay.
I knew a guy once who, when he got a large fungible job in, would put out a want ad and "interview" people exactly as you describe. He'd literally get hundreds of hours of free labor this way, and the bastard knew he'd never be called on it.
It is for exactly this reason that I don't work for free during interviews. If my prospective boss isn't sharp enough to know that I know my stuff after a brief conversation and a look at my credentials, then I'll happily work for his competition.
He put his boots up on the table and made a face. "The sig," he smirked. "You can waste your life in search of the sig."
One thing employers should do more often is talk about salary up-front, so prospective employees don't waste a lot of time with testing and interviewing just to get an insulting low-ball offer. This has happened to me several times, and it's annoying. This doesn't mean testing is bad; I had some simple tests in the interview for my current position, and they gave me a very generous offer unlike other firms in the area.
I've found too many companies are tight-lipped when it comes to salary; they want to waste a LOT of your time in interviewing and such (and their own employees' time too), but then when the hiring manager says "he's great! hire him!", the HR person wants to give a lowball offer, and refuses to bring it up to anything reasonable even when you have a much better offer in-hand. If these companies would just state up front what their salary range is, we wouldn't waste all this time and effort.
Meanwhile, these tests do NOT need to be difficult or long. I had to help phone-interview some contractors at one job, and I came up with some very simple questions to ask them. One was, "In C++, can you tell me what a 'class' is?". Several contractors who stated "expert-level C++ knowledge" on their resumes couldn't answer that basic question. So you don't need to give them a 2-hour test; just some simple questions, maybe some short code samples with errors in them, etc. to see if they're completely lying about their credentials or not. So many people lie on their resumes (and this is definitely cultural; I saw this mainly in people from India), it's really important that you screen out the liars from the people who can actually do the job.
You've got to be kidding me. If I have to come to a location multiple times for a job interview, I'm probably wasting my time. The only way I'd bother is if we start discussing salary right away, so I don't find out at the very end that this employer is a low-baller.
In my 11 years of experience and many, many job interviews, I've never had to come back to a place for a second interview. If the employer can't tell if I'm a good fit in one visit, they're doing something very wrong.
I've worked several places that had some sort of practical skills test for programmers.
I consider them a strong success.
There's a surprising number of people that can talk a really good game, but fall apart when asked the most basic actual questions. And, conversely, people who aren't extroverts or for whatever reason aren't stars in the verbal part of the interview, but clearly know their stuff when given a written test. It's a really good way to fairly judge technical skills across candidates, and weed out the fakers before you have to fire them later.
And as an employee, I like working with smart, competent people. I know how much, from experience, a bad hire wastes my time and ruins morale, so I'm happy to work for places that put out effort one way or another to not enter into this trap.
Anyone who'd refuse on principal, I'd worry is either a) faking it, or b) too arrogant to work well with others. A good candidate is happy to prove that they're a good candidate, and won't have to work with idiots.
A test isn't the _only_ way to do this. Any sort of nice, concrete technical grilling will do. But for a programmer, it _must_ involve actually writing code of a non-trivial nature. You can't believe resumes - even if people aren't lying, a Senior Programmer at one shop may only be barely competent at another, and not even realize that the bar is set differently.
Of course, the quality of a test can vary just like the quality of an interview question, but the goal is good for the company _and_ the employees, and in my experience it works better than most techniques.
Now, if your shop isn't terribly compelling based on product, and you're desperate to not turn people away... well, you probably should be looking for places that feel like they need a test to screen out unqualified candidates, so you can stop hating your job, and not worry about fixing this particular problem. :)
-- Kate
Totally, if you are offering less than current_income + 10%, why should the interviewee waste his time?
If someone doesn't interview (or worse, complete an interview) because of a test I don't care how smart they are. They're too much of a prima donna. I've been in situations where an interview had tests that were way beneath my skill level, and in those cases I've either known immediately that the job I was interviewing for wasn't for me, asked the interviewer if there were more high level jobs available, or helped them fix their test. (In one case the test had questions that helped answer previous questions, so I helped them fix it.) In all cases I impressed the interviewer enough to get the job.
I do like your solution though of actually watching them write the code though, because that does prevent them just copying and pasting other code and sending it to you.
Copying and pasting code amounts to 90% of even the best programmers' jobs. Other than some aspects of GUI design and input validation (which even then just means "tweak the conditionals of what you have", not "write an input handler from scratch"), if you sit down at a blank editor on anything but a toy project, you either work in academia or need a much tighter deadline.
And there, I have my biggest objection to "testing" applicant, particularly in a vacuum. In the real world, I program with access to huge libraries of functions and at least half the time, a web browser open to look random things up as I need them. Yeah, I could rewrite Windows from scratch if you give me 500 man-years to do it, but do you want to see if I can really do the job, or do you want to see if I happen to know the prototype for some particularly obscure network calls off the top of my head?
People pay me to solve problems efficiently, not because I've memorized the latest edition of K&R.
(and for the record - Yeah, I pretty much do have the prototypes for the entire standard ANSI C library memorized, but would prove just as valuable if you wanted me to program in any language, including one I've never used before)
I'm now retired, but all the way back to my first tech job interview in the '60s, the interviews have included tests of my ability to perform as needed.
If one cannot test an applicant one is seriously handicapped in making valid hiring decisions.
--
Tomas
University Place, WA
If more employers were able to test their employees adequately during screening, there would be a lot fewer bullshitters out there trying to claim they can do what they can't. Burn in hell you cert-chasers who don't know what you are doing!
Every employer who gave me such a test, I always got an offer -- usually a good offer. Some of us are actually good at working through IT related problems and solving them. Some of use are not and just want to coast through on bullshit.
There are very few people who feel that they are "above the test" and the eager ones actually say "bring it on!" Those are the ones you want.
As for setting up a "practical" part of a test? Absolutely! If you have the skills to build a nice one, do it.
Nothing gets under my skin more than HR people screening resumes of really good people because the right set of words didn't appear on a page. Every successful hire I have ever had was when another IT person was involved in the screening and interviewing.
Yes, this process does weed out the self-important people.
And then you spend several days or weeks sifting through the 100 responses, integrating them into the code base and running regression tests, until you find the one that actually works. If you had that much time to spare, you could have fixed the problem yourself.
Many moons ago when I was doing coding jobs I had a series of interviews that required me to code stuff to demonstrate what I knew. I was looking to move away from my current job into new areas and didn't know how they worked but they all seemed to ask the same thing
1) Code something
2) Take a coding test
What I learnt very quickly was that lots of people are really looking to hire people not quite as smart as they are. I knew this because I had three interviews where the following happened
Interview 1:
Set an "impossible" task to code (in C) an address book and calendar solution with a GUI (this is the mid-90s) in 6 hours. No internet connection and no reference books.
3 hours later I set off to find the interviewer in the pub.
Interview 2:
Set a series of questions around "write the most effective code to do X".
There were 10 questions and I answered them in 10 different programming languages (Ada, C, Prolog, C++, Eiffel, 68k assembler, LISP, SQL, COBOL, Fortran) and the chap interviewing me didn't have a clue if I was right or wrong.
Interview 3:
They had a major bug in a current release, my job was to find the bug and explain why it happened. I knew free work when I saw it. It was a big C programme and it took me 4 hours to find the bug (pointer referencing problem). I wandered into the office of the person setting the "test" and said...
"How much to tell you the answer"
I didn't go for any of these. What I went for, and what I have done since, is go for the company that set me an abstract test that asked me to design a system which had a number of constraints. They didn't want code, just to see the conceptual model that I would create. When I joined I asked why they did it this way and the answer was enlightening....
"Because we want to hire smarter people than we are. If you talk code then its just about optimising, but if you talk about the abstract then its about thinking. We want people who give us answers we think are wrong and then they explain why we are wrong."
The key point was to give people a limited time (15-30 minutes) to understand the problem and propose the solution. You want people who are agile and quick, not people who can sit on their arses for 6 hours doing a troll job.
If you want to hire people dumber than you, set complex long tests that "only you" know the answers to. If you want to hire smart thinking people set very short tests that challenge their abstract thinking.
An Eye for an Eye will make the whole world blind - Gandhi
One C++ shop I worked in we used to have a short sample C++ program (~~ 2 pages long) that was deliberately written to contain a large number of problems, many obvious and some quite subtle. It was an "Animals" style example program so manifestly not production code. We asked candidates to examine the program and find as many of these problems as they could then suggest better ways of achieving the same thing.
We weren't terribly interested in how many problems they actually found but were vitally interested in how they approached the analysis and how deeply they understood object orientated design and the C++ version of that paradigm.
It was great fun to write and we eliminated quite a few posers with this tool.
But what if they want you to meet multiple people? This happens a lot when scheduling conflicts make it impossible to do one all-day interview.
Personally I go out of my way for a potential company if I like what I hear throughout the interview process. I can come back more than once. In fact I did two phone interviews with the same company and I'll be flying down there Monday to do an on-site interview. I get paid a very good living and I understand that my employers expect the best so I'm not too put out by it. A great job can turn into a lifetime position so one might as well put as much effort as needed.
Not having an answer proves they don't know how to use Google. It took me 5 seconds to find a good overview of what a c++ class is. If you were talking to me, I'd tell you straight up; I'm not good at regurgitating formal definitions, but I would tell you where to find the same article I am looking at, then riff on various real world projects and problems.
If you can't tell me, in very general terms, what a C++ class is, then you're not qualified to claim you're an expert on C++. It's that simple.
Yes, I do a lot of Googling around when programming to learn more, to find other answers to problems, etc. I also don't have the C++ operator precedence memorized, and refer to a table for that. But some things you need to know or else you simply cannot claim that you're "proficient" in the language. When the job req says "needs to be proficient in C++", then certain basic things like that you need to know.
I'm sure I could Google around, read a book, or whatever and put together something in Python pretty quickly even though I've never programmed in it before. But I wouldn't tell an employer that I'm proficient in it.
Really, who do you want to hire? someone who can tell you what a class is, or someone who can tell you about problems they have solved.
Are you trying to claim that it's OK to lie on a resume and claim knowledge of programming languages you've never worked with?
Don't forget, this particular job was a contracting job. Long-term employment was not to be assumed here; the company wanted someone who could sit down and start contributing right away.
If you are in a smaller organization, it is very likely that there will be periods where other skills are needed - such as interacting with customers (pre-sales/post-sales), design, documentation, etc. etc. So be careful that your are not screening for one-trick pony (coders) if there is any possibility that you will need someone with other skills as well.
It seems to me that if the job entails those responsibilities, that should all be stated up front instead of assumed. I wouldn't take a job that required a lot of customer contact; my specialty is software engineering, not schmoozing clients. Time wasted with customers is time that could have been spent productively. Find a different employee to do the customer-contact stuff; this is why humans invented the concept of "specialization of labor".