Slashdot Mirror
Pre-Employment Skill Set and Aptitude Tests?
stumbler asks: "I just had a lengthy conversation with my boss and co-workers about the value of giving skill set tests (programming ability) and aptitude tests (like reasoning or logical ability) to technical employees before they are hired. (We currently have no such tests.) For those that work in companies that require pre-employment tests, have you seen an impact in the quality of technical employees hired?"
They hired me. Which means the skill set tests don't have value! Any standard test can be gamed.
If I remember, those who past generally were well liked by the others who interviewed them.
Then there's the grammar test....
-Tolerate my intolerance
and you run a math teachers office right?
to most of us that type of stuff isn't very useful.. sure alot of us have done it at some stage (high school) but what bearing does asking such questions actually have on how well i can do my totally unrelated work...
if perhaps you were asking simple algorithmic questions... maybe boolean algebra simplification then i could see some benefit..
groklaw, wired and slashdot. The holy trinity of work based time wasting.
we give the subject a relatively simple question with no real answer that they should be able to analyze and generate opinions of on their own. Then we check the browser history and see how they posed it in Ask Slashdot. If they didn't provide any insight or guide the conversation in any way, then that's generally bad :-)
I was given math and logic tests as part of the interviewing process. Where they relevant? Somewhat. Did they really separate the good interviewees from the not so good? I doubt it. I'm guessing that any distinction they reveal would have come up anyway in the interview process.
Perhaps the reason I got hired by the company was not necessarily my performance on the tests, but when I told them later on in the interview that (truthfully) I had enjoyed taking the tests, and liked solving logic puzzles in general.
our tax rate is 6%. anyone who cannot answer, what's the tax amount on a 100$ sale, we will never hire..
answers vary, one accounting major looking for a summer job answered "about 60$" and called the next day to apologize, needless to say, he wasn't hired...
a simple test can be the most effective....
every day http://en.wikipedia.org/wiki/Special:Random
I work advising on travel arrangements - a form of travel agent, except I'm not selling anything. My employer used to test numeracy and geography, which is clearly applicable to the job.
The geography test was basically a national map with dots on it and a list of place names, including several fairly obscure ones. If you scored less than 75% you didn't get the job.
My employer no longer tests these things. This has resulted in a lower grade of new-starter, but if they're intelligent they learn, and if they have numeracy problems they didn't disclose it quickly becomes apparent in training.
But in the mid to long term this has no impact. If they don't know how they learn on-the-job. If they won't learn they're gone.
I see no reason why this can't apply to IT too, as ANY job has some sort of learning curve, but in a small company I expect it's safer to test basic skills - you're more likely to get useful results out of them straight away.
--
Before we can hire you to write UI widgets in Java, please derive the quadratic theorem. NOT! For some jobs this makes sense. For others it's nothing more than a thinly veiled elitism.
I was taught how to derive the quadratic theorem in my freshman year in high school. Unfortunately, it was only a one hour lecture, and the knowledge was never, ever used again in the subsequent twenty five years. I don't think I could pass your test. But fortunately, I don't have any bosses stupid enough to make it a job requirement.
Don't blame me, I didn't vote for either of them!
....give them a series of printed c/c++ code (or relevant language used in the company) that has obvious and less obvious errors / bugs in it. Tell them to locate the bug and find a work-around. Measure the time that it takes them to do so. Have them fully explain the bug and tell you where they've found it before (past programming experiences). this is a very good measure.
Pre-employment tests are sometimes good for generic categorization of employees. Skills and aptitude tests are quite different from cognitive and behavioral tests. If you've ever taken either of the latter, they can be laughable. The problem is that hiring managers give those tests to a candidate, see "Inability to focus" and "Cannot develop strong relationships" on the results, and assume that's bad.
Give the same tests to your current employees and try to correlate the results. You'll undoubtably find a few patterns amongst your excellent employees that differ from your mediocre employees. The more results you collect from candidates that you know are horrible along with your employees, the better you'll be able to customize the feedback for your exact environment.
In a previous job, I found that all of our core software developers did fantastically on a general cognitive test (IQ-like), but that most did horribly on behavioral tests -- they were all "Likely to be insecure with their work". In fact, potential candidates that were also "Likely to be insecure" often matched the personality profile that worked in our group. So, test your own employees and see what happens if you highlight candidates that perform similarly to those employees that excel, rather than taking the simplistic approach of "a bad score must be bad!" with prospective employees.
You may find that if you give a variety of tests to 20 candidates, ranging from specific skillset assessment to leadership profiles, that you can at least take a harder look at the 3-5 candidates that score poorly across the board -- if they're barely above average (or worse) and they don't test well, that's not a great sign. Conversely, you could hire someone because they do a great job on all of the tests, but that would be equally horrible -- lots of people are good at taking tests and bad at producing actual work.
Assuming a handful of people with equal qualifications, why take the risk? Especially in this job market, there are too many people out there that not only have the right skills and behavior that can also do well on the corresponding tests.
Interview problem solving often has something in common with Adventure style games: Guessing what the author was thinking is more effective then solving the problem.
When was the last time you solved a real-world problem in a few minutes with someone looking over your shoulder who already knew the "correct" answer?
There is no reliable algorithm or heuristic for hiring the best people, but some companies are comforted by introducing pseudo-rigor into the process.
to prospective Software Engineers. I'd never seen that practice before, but it actually works out quite well. It quickly weeds out those who don't really have a clue about programming. Everybody pads their resume to some degree or another, so it helps to determine who truly knows their stuff from people who merely recant the latest buzzwords. We use three primary languages here: C, C++, and Java. Depending on the position/group a candidate is being interviewed for, the applicant is given a set of 5 simple problems to solve. They sit in a room with a pad of paper, pens and pencils, and a non-networked PC loaded with our dev toolchain. We're basically checking for the following: 1) Do they really know the language? Too many people claim fluency in a programming language under the belief that they can quickly come up to speed should they get hired. While I do believe that most languages share many common features and a competent geek can learn a new language in a weekend, true mastery takes more time. In this market, most companies hiring a software engineer for a given language expect them to have experience in that language. 2) Can they solve a problem? I'm truly amazed at the number of people out there who can't find a solution to simple problems. We hire software engineers, not programmers. We need thinkers. The tests we use aren't really difficult. Stuff like: given a text file, determine the number of characters, words, and occurences of the phrase "i like cheese". 3) On a more behavioural side of things, we note whether they're able to find their way around the system to perform the test. As I said, we set them at a PC with an xterm open, and a printed problem sheet. Can they find Vi/Emacs? Do they really understand the gnu toolchain? If not, do they ask for assistance? We'd much rather have an applicant ask for help when he needs it than waste an hour trying to muddle through. This process usually weeds out 95% of the applicants. Of course this is only for basic skill verification. When they pass this round, we then do the face-to-face interviews. Obviously, this doesn't quite translate to other job positions, but I'm sure there are many engineering type jobs that this sort of hands on test would be useful with. As far as personality tests, my belief is that only a face to face interview, and of course time, will tell.
I'm a sysadmin by trade, and I think a nice way to do it would be to give them a broken system to diagnose and fix. Hardware or software. Whatever you have that needs fixed. It shows you how they handle the type of stuff you need handled. And, if you have enough applicants, you never have to hire anyone! Just have them fix your stuff for free! ;)
Gosh.. I finished a physics master's degree less than 2 years ago and I had no memory of how to do it. Scary, but understandable. Solving the quadratic equation is one of those proofs that is simple but has a "trick" that you have to remember. Unless you use that trick in your day-to-day work (which, as a software engineer, I obviously don't), you're not going to remember it any other way than by having seen the solution recently.
Pretty stupid test for an interview. I'm top performer in my current team despite being the most junior. The project would probably have failed if I hadn't been there. I wouldn't have remembered the solution, though.
Daniel
Carpe Diem
I guess I have an abnormal hatred of dishonesty, but I think the best use of these tests is too weed out the flat out liars or those that 'bend the truth' a bit too much regarding their skills.
So, my point is simply that, yes, these types of tests are quite useful for verifying resumes, but not a whole lot else.
Over the past few months I have hear stories about the recruiting practices for a certain Bay Area company. They use multiple tests - what they have ended up with is a tech manager and a bunch of programmers who are comsumed by the minutae of the programming language they use and their own intelligence. The project that is the basis for their company has not made much progress and they are burning money.
Presumably, they will be smugly certain that it was something outside of the brain trust that caused the company to fail when it flames out. I know I wouldn't want to work "there".
Tests of syntax and paper-based "find the error" tests are irrelevant and unrelated to the mode of problem solving that people do 8-10-12 hours a day.
I contend that navigating corporate structure, squeezing turnip-like business people for the true requirements and encoding them into systems are the real tasks that many IT/software people do.
I use maybe 1/10th of perl to do 95% of the work I do with it, does testing my knowledge of the other 90% mean you can tell if I am an effective/productive team member? No way. But that is the problem, the process of recruiting, evaluating and hiring tech people is broken. And I haven't seen any HR departments, contracting agencies or hiring managers who can discern true talent from a resume, test or interview.
Check out the book Software Craftsmanship for more about what I think people should be hiring and rewarding. And although people might think its a joke, check out the Dilbert Principle if only for the section on OA5, dilbert's management model - one of the rules is something to the effect of "First, get rid of all the assholes"...
If you would do that to me I would not consider taking employment with you.
To me it would look like you are disorganized and frankly could not care to work with a company with such messy outlook.
IANAL but write like a drunk one.
"Give them a wrong address for the company. Nothing too misleading , such as a different town, just be a few streets out."
It cuts both ways though. If I was applying to a company that hired people incapable of giving the correct address, I'd think twice. Likewise, if a company deliberately misled me as part of the interview process, it would be harder to believe anything else they said.
And the most you've done is prescreened people who can use Mapquest. Whoop-DEE-doo.
"For fun, give them some paperwork to fill out at the end of the interview and say "I just have to duck out and check on something - back in a tick". Leave and time how long it takes for them to wander out of the office in search of someone... 15 minutes to half an hour's a pretty good baseline."
Most of the interviews I've been to have had a specified time limit (or have been happy to tell me when asked). A lot of people don't have time to waste on interviews: they're either taking time away from their current job, or have the day off. Why waste their time "for fun"? Wasting 15-30 minutes of interview time is stupid when you could be doing something productive (like talking to them).