Slashdot Mirror
Pre-Employment Skill Set and Aptitude Tests?
stumbler asks: "I just had a lengthy conversation with my boss and co-workers about the value of giving skill set tests (programming ability) and aptitude tests (like reasoning or logical ability) to technical employees before they are hired. (We currently have no such tests.) For those that work in companies that require pre-employment tests, have you seen an impact in the quality of technical employees hired?"
They hired me. Which means the skill set tests don't have value! Any standard test can be gamed.
Derive quadratic theorem (don't just spit it out!). Prove square root of two is irrational. If I remember, those who past generally were well liked by the others who interviewed them.
-I am an elective eunuch.
we give the subject a relatively simple question with no real answer that they should be able to analyze and generate opinions of on their own. Then we check the browser history and see how they posed it in Ask Slashdot. If they didn't provide any insight or guide the conversation in any way, then that's generally bad :-)
I was given math and logic tests as part of the interviewing process. Where they relevant? Somewhat. Did they really separate the good interviewees from the not so good? I doubt it. I'm guessing that any distinction they reveal would have come up anyway in the interview process.
Perhaps the reason I got hired by the company was not necessarily my performance on the tests, but when I told them later on in the interview that (truthfully) I had enjoyed taking the tests, and liked solving logic puzzles in general.
Were they relevant- not "where".. guess they should have tested me on spell checking too.
our tax rate is 6%. anyone who cannot answer, what's the tax amount on a 100$ sale, we will never hire..
answers vary, one accounting major looking for a summer job answered "about 60$" and called the next day to apologize, needless to say, he wasn't hired...
a simple test can be the most effective....
every day http://en.wikipedia.org/wiki/Special:Random
I work advising on travel arrangements - a form of travel agent, except I'm not selling anything. My employer used to test numeracy and geography, which is clearly applicable to the job.
The geography test was basically a national map with dots on it and a list of place names, including several fairly obscure ones. If you scored less than 75% you didn't get the job.
My employer no longer tests these things. This has resulted in a lower grade of new-starter, but if they're intelligent they learn, and if they have numeracy problems they didn't disclose it quickly becomes apparent in training.
But in the mid to long term this has no impact. If they don't know how they learn on-the-job. If they won't learn they're gone.
I see no reason why this can't apply to IT too, as ANY job has some sort of learning curve, but in a small company I expect it's safer to test basic skills - you're more likely to get useful results out of them straight away.
--
see subject
because the damn PHB hired the ones that failed anyway.
--jdp Maintainer of VisEmacs
I've done some applying for positions at various software houses throughout north america and while I am still in univeristy, alot of their responses have been constructive. The general idea is a test for an overall skill set like programming languages and development cycles, and a test or two in the logic and reasoning ability. Just to make sure that you are not a talking box, and you have a brain that can deal with a changing enviroment.
I am presently working a 4 - 8 strech in tech support as a co-op term. With the market the way it is, I took the first job I could get my hands on. So right now, I'm sitting in Toronto as a bilingual techsupportist not programming, but generating another skill set I hope I do not end up exploiting for the rest of my life. With the market, my university (Acadia University in Wolfville Nova Scotia Canada. Yes... I know you have not heard of it.:)) has only sent out 3 comp-sci 2nd year co-op students. This is normal, but honestly, I'm getting scared for when I start looking for a real job in the next 2 years.
ANYWAY, the general plan for an application procedure is to weed out what the management or HR people get at. I know for my company, I'm involved (for some unknown reason) in the hiring process for the september set of co-op people. And we have designed an application procedure that they enter in answers for 50 questions, and we search a database of answers for the proper skill set. That way we don't waste our time dealing with duds.
The big thing is that people are looking for a person who can back up their resume, and think for themselves when they are allowed the opportunity. If you get talking heads sitting there always doin exactly what you say, then even wrong paths get confirmed.
"Are you guys just a bunch of yes men??"
"Yes sir!"
(CM burns talking to his team of legal professionals.)
while(1) { fork(); };
You can have someone submit an impeccable resume, or be able to answer questions quickly and reliably on tests, but the only way you'll ever truly know the ability of someone is to see how well they work.
I've conducted a few interviews where I poked and prodded to determine ability, but that one hour is rarely enough. You can always find someone who can talk a great game, but in the end cannot produce (or cannot produce to expectations)
In the end, I think the best result is to offer said individual an opportunity to prove themselves:
Give them a few weeks on a lower pay scale to prove how well they work. If after a few weeks they prove to be worth it, up their salary to a decent wage and give them back pay (at the new rate) on all hours worked during the "trial period" so that their time spent was not wasted. If they are not worth it, cut them loose.
How could I say to men: "Speak louder, shout! For I am deaf!"? -Ludwig van Beethoven
....give them a series of printed c/c++ code (or relevant language used in the company) that has obvious and less obvious errors / bugs in it. Tell them to locate the bug and find a work-around. Measure the time that it takes them to do so. Have them fully explain the bug and tell you where they've found it before (past programming experiences). this is a very good measure.
Pre-employment tests are sometimes good for generic categorization of employees. Skills and aptitude tests are quite different from cognitive and behavioral tests. If you've ever taken either of the latter, they can be laughable. The problem is that hiring managers give those tests to a candidate, see "Inability to focus" and "Cannot develop strong relationships" on the results, and assume that's bad.
Give the same tests to your current employees and try to correlate the results. You'll undoubtably find a few patterns amongst your excellent employees that differ from your mediocre employees. The more results you collect from candidates that you know are horrible along with your employees, the better you'll be able to customize the feedback for your exact environment.
In a previous job, I found that all of our core software developers did fantastically on a general cognitive test (IQ-like), but that most did horribly on behavioral tests -- they were all "Likely to be insecure with their work". In fact, potential candidates that were also "Likely to be insecure" often matched the personality profile that worked in our group. So, test your own employees and see what happens if you highlight candidates that perform similarly to those employees that excel, rather than taking the simplistic approach of "a bad score must be bad!" with prospective employees.
You may find that if you give a variety of tests to 20 candidates, ranging from specific skillset assessment to leadership profiles, that you can at least take a harder look at the 3-5 candidates that score poorly across the board -- if they're barely above average (or worse) and they don't test well, that's not a great sign. Conversely, you could hire someone because they do a great job on all of the tests, but that would be equally horrible -- lots of people are good at taking tests and bad at producing actual work.
Assuming a handful of people with equal qualifications, why take the risk? Especially in this job market, there are too many people out there that not only have the right skills and behavior that can also do well on the corresponding tests.
You've got the guy's resume. You've got references if you want them. You've talked to him. If you throw a test at him, that's just saying that you don't trust his resume, references, or word.
You expect the prospective employee to believe you when you tell him about the benefits, his opportunities for advancement, what kind of work he will do, etc., don't you? How would you feel if he demanded that you prove all of those things before continuing with the interview?
Interview problem solving often has something in common with Adventure style games: Guessing what the author was thinking is more effective then solving the problem.
When was the last time you solved a real-world problem in a few minutes with someone looking over your shoulder who already knew the "correct" answer?
There is no reliable algorithm or heuristic for hiring the best people, but some companies are comforted by introducing pseudo-rigor into the process.
I am only 16 so I dont have much work experience, but it would seem to make sense that you bring a portfollio of past work (on a USB thumb drive, CD, etc) and they look at it to decide if they like your style, how well written it is, etc. I am sure you all know what a portfollio is so I wont elaborate much more, but it would seem like the best idea to me.
Scott Swezey
Frankly, it depends on the job.
For programming? It's great. In java, you should be able to explain polymorphism, for example, or how to prevent memory leaks even with garbage collection, etc.
This is opposed to an interview, where you can ask more off the wall questions. A test is really only a bar - do you have the basic knowledge or are you bluffing? The interview will select the good people.
"For those that work in companies that require pre-employment tests, have you seen an impact in the quality of technical employees hired?"
Well I'm happy to report that OSDN doesn't require any kind of tests.
What is the airspeed velocity of an unladen swallow?
A bit OT, but does $44 sound right for a 128 MB USB drive?
to prospective Software Engineers. I'd never seen that practice before, but it actually works out quite well. It quickly weeds out those who don't really have a clue about programming. Everybody pads their resume to some degree or another, so it helps to determine who truly knows their stuff from people who merely recant the latest buzzwords. We use three primary languages here: C, C++, and Java. Depending on the position/group a candidate is being interviewed for, the applicant is given a set of 5 simple problems to solve. They sit in a room with a pad of paper, pens and pencils, and a non-networked PC loaded with our dev toolchain. We're basically checking for the following: 1) Do they really know the language? Too many people claim fluency in a programming language under the belief that they can quickly come up to speed should they get hired. While I do believe that most languages share many common features and a competent geek can learn a new language in a weekend, true mastery takes more time. In this market, most companies hiring a software engineer for a given language expect them to have experience in that language. 2) Can they solve a problem? I'm truly amazed at the number of people out there who can't find a solution to simple problems. We hire software engineers, not programmers. We need thinkers. The tests we use aren't really difficult. Stuff like: given a text file, determine the number of characters, words, and occurences of the phrase "i like cheese". 3) On a more behavioural side of things, we note whether they're able to find their way around the system to perform the test. As I said, we set them at a PC with an xterm open, and a printed problem sheet. Can they find Vi/Emacs? Do they really understand the gnu toolchain? If not, do they ask for assistance? We'd much rather have an applicant ask for help when he needs it than waste an hour trying to muddle through. This process usually weeds out 95% of the applicants. Of course this is only for basic skill verification. When they pass this round, we then do the face-to-face interviews. Obviously, this doesn't quite translate to other job positions, but I'm sure there are many engineering type jobs that this sort of hands on test would be useful with. As far as personality tests, my belief is that only a face to face interview, and of course time, will tell.
I'm a sysadmin by trade, and I think a nice way to do it would be to give them a broken system to diagnose and fix. Hardware or software. Whatever you have that needs fixed. It shows you how they handle the type of stuff you need handled. And, if you have enough applicants, you never have to hire anyone! Just have them fix your stuff for free! ;)
this prooves exactly whats wrong with 95% of "tough interviewers" ... the quardratic equation mean exactly fuck all to an applicatoins developer
Give them a wrong address for the company.
Nothing too misleading , such as a different town, just be a few streets out.
People who then show up at your office are obviously good at solving simple technical problems.
People who make it to the interview on time get bonus points.
For fun, give them some paperwork to fill out at the end of the interview and say "I just have to duck out and check on something - back in a tick".
Leave and time how long it takes for them to wander out of the office in search of someone... 15 minutes to half an hour's a pretty good baseline.
You are in a twisty maze of processor lines, all alike.
There is a lot of hype here.
I guess I have an abnormal hatred of dishonesty, but I think the best use of these tests is too weed out the flat out liars or those that 'bend the truth' a bit too much regarding their skills.
So, my point is simply that, yes, these types of tests are quite useful for verifying resumes, but not a whole lot else.
Over the past few months I have hear stories about the recruiting practices for a certain Bay Area company. They use multiple tests - what they have ended up with is a tech manager and a bunch of programmers who are comsumed by the minutae of the programming language they use and their own intelligence. The project that is the basis for their company has not made much progress and they are burning money.
Presumably, they will be smugly certain that it was something outside of the brain trust that caused the company to fail when it flames out. I know I wouldn't want to work "there".
Tests of syntax and paper-based "find the error" tests are irrelevant and unrelated to the mode of problem solving that people do 8-10-12 hours a day.
I contend that navigating corporate structure, squeezing turnip-like business people for the true requirements and encoding them into systems are the real tasks that many IT/software people do.
I use maybe 1/10th of perl to do 95% of the work I do with it, does testing my knowledge of the other 90% mean you can tell if I am an effective/productive team member? No way. But that is the problem, the process of recruiting, evaluating and hiring tech people is broken. And I haven't seen any HR departments, contracting agencies or hiring managers who can discern true talent from a resume, test or interview.
Check out the book Software Craftsmanship for more about what I think people should be hiring and rewarding. And although people might think its a joke, check out the Dilbert Principle if only for the section on OA5, dilbert's management model - one of the rules is something to the effect of "First, get rid of all the assholes"...
I am a director for a small, niche ISV. I use a skillset test to guage how quickly someone new can be productive. What I find more important is the verbal questions that I give in the interview. I find out how long they have been in a certian technology then I ask questions to determine how deep they are in it. We pass on those that take a long time to learn a new technology.
If you would do that to me I would not consider taking employment with you.
To me it would look like you are disorganized and frankly could not care to work with a company with such messy outlook.
IANAL but write like a drunk one.
That is abuse of the interviewee's time.
IANAL but write like a drunk one.
Back in the day is was just the IPAT, now it is online! It is an Information Processing Aptitude Test. Different organizations in the company have differing opinions about the test. I think that doing well on it helped me get hired.
Lasers Controlled Games!
My company (a small consulting firm in the northeast) has applicants take Brainbench tests on topics relevant to the position for which they are applying. When they instituted the testing, they asked the current employees to take the tests to establish a baseline. Applicants are also given a single essay question to see how they can apply their knowledge against scenarios we have faced. We don't rely solely on the tests, though. If someone aces them but comes off as a total asshole in the person-to-person interview, they're out.
The company is now considering giving DISC personality tests as well, to better ensure that new people will mesh well with the existing team. We all took them already to see how accurate they were, and the results were pretty dead on.
I'd give them a bs list of tasks in excel (I work in finance) and the test would be if you use the keyboard for them or the mouse. People who use the keyboard in spreadsheets for things like cell navigation, copy paste and other repetative tasks. Perhaps you could do the same with a command line vs GUI, I don't know if this is quite the same, as excell (the productivity increases by an order of magnitude if you understand the ctrl seeks and 5 basic keyboard shortcuts (cut, copy, paste, fill down, and fill right). Another test might be to throw them in a new environment (vi vs emacs vs Visual Studio, or something else) (I'd see if they could quickly adjust to a different office suite/spreadsheet). Good candidates should be uncomfortable, but able to function.
Degaussing scares the bad magnetism out of the monitor and fills it with good karma.
I use the proof that the square root of 2 is irrational every day at work.
The ITA Software site has some interesting programming "challenges" for applicants:
"If you are interested in this position, please solve one of the following problems so that we can evaluate your programming ability. Please send your solution and resume to us via this link. Unless otherwise specified, you may use any language you like. Aim for clarity and efficiency. Please include your program's final answer in the body of your email, and please send code that actually compiles and runs so we can test it -- no pseudo-code please!"
http://www.itasoftware.com/careers/eng/job1.php
They look like some of the problems in the ACM collegiate competitions.
Got a 'programming skill' test from a headhunter once, supposedly covering C/C++...at least 2 or 3 questions contained code that was obviously not C or C++. Another had code that was either broken, or testing very obscure details of the C language. Quite a few were just generally poorly written(though still easy to guess)...
Where they relevant?
I guess they didn't give you a spelling test...
*wink* *wink*
Gabriel Ricard
I've been given this sort of test twice. The first time, I did brilliantly. The interviewer commented that I was the first person he'd talked to who had gotten all of the details of one of the questions. While I am proud of my skills, I am also aware that I did that well because they were using almost exactly the same development environment that I had been using for the previous couple of years.
The second time, I did rather poorly. The interviewer was looking for a very specific skill set rather than general ability. I couldn't answer a number of his questions. I have since done many of the things he asked about in the interview. I've done them fast and well. In several cases, I've chosen a different set of tools than the ones he said his team was using. His answers were right for the problems he was solving in the environment in which he was solving them. Those solutions wouldn't fly where I've confronted the problems because of some important differences.