Slashdot Mirror
Testing IT Professionals On Job Interviews?
An anonymous reader writes "After having my university degrees, a couple of IT certifications, and over ten years of work experience in the industry, with 2-4 years of verifiable employment with each employer, working with a wide range of technologies, is it reasonable to ask me to take some test on a job interview? The same companies don't ask other professionals (lawyer, accountant, sales, HR, etc.) to submit to any kind of in-house tests when they are hired. Why are IT professionals treated differently and in such a paternalistic way? More importantly, why do IT professionals accept being treated less favorably than members of other professions? Should IT professionals start to refuse to be treated as not real professionals?"
I won't take them.
I have turned down several jobs over it.
thus the whole question is futile.
Skill assessment is done in almost all kinds of professional employment situations . yet it depends mostly on the hiring policy of the department of that particular firm if there will be an assessment.
And quite franky, I think there is a good reason why this is done with IT jobs more often: analytic and associative thinking and problem solving are not skills you can learn.
Plus, IT jobbers tend to be more annoyed by moron colleagues than non-IT employees.
And lets not forget that there is a huge amount of moronness out there - I myself did Job interviews with certified whatevers, who applied for a sysadmin position and couldn't tell me what information a notation like "192.168.38.1/24" provides. And thats just the very basic for such a job, but it already weeded out two thirds of the applicants, *completely unrelated* to their educational history or other certified qualifications.
And last but not least, it always depends on the quality of the respective management if such an evaluation is done: and speaking for me and my experience, a company should do it in *all* sorts of positions, no matter how professional, experienced and well educated an applicant is.
I was recently involved in a series of gruelling and unfair interviews in which we destroyed the confidence of a series of IT professionals with extraordinary difficult questions. Having spent 10 years as an accountant, and 10 years as an IT Manager, I found myself asking the same thing. In order to qualify as an accountant, I had to take 17 exams over the period of 6 years, with each exam having a 30-50% pass rate. During the first 2 years, I could barely make a living wage. To become an IT Manager - I was just in the right place at the right time. I since gained OCP and MCSE, but nobody takes them seriously - in relative terms, they were both very easy to pass. It is still a fact that an accounting (and probably legal) qualification counts for more than an IT qualification.
Testing is a good thing. Years ago, I ran from interview to interview, but didn't get the jobs. My grades from school were fine, but being a geek, there was one thing I was (and still am) bad at: "Selling myself". When we got to the question "please tell us about yourself", I didn't know what to say. I've always hated that question.
But then at one place, rather than expecting me to talk about how great I am (hey, if I was that good at selling, I would be in the sales department, not IT), they sent me a small programming task. Towers of Hanoi, I know, text book stuff, except the second part required a bit of brain work.
Guess what... I got the job. At the interview itself I probably didn't do any better than all of the other interviews (of course we did talk a bit about my solution to the test, but apart from that). I have no doubt that what made the difference was that I got to show my programming skills, rather than my (lack of) salesperson skills.
At my current job, new people get a simple programming test too (small company, we talk about things at lunch). And according to the manager/programmer who does the interviews, he's had several people who "had lots of experience with exactly what we are doing", but when shown a simple programming problem had no idea how to solve it. Those are the people who excel at regular job interviews. Good salespeople, but not good programmers.
A CV can be read in many ways. I think that testing is a good way to see that the skills, CV and open position match. That being said, testing can be done in many ways.
Someone recommended me to Google once, and the Google HR department obviously read my CV looking for the skills they were after. While I had them to a degree, that was only part of the truth. A later phone interview with one of their engineers clarified the situation a lot: He tested my skill set with a bunch of oral test questions that made it obvious to both that my skills were of the right sort but at the wrong layer of abstraction. (Scripting vs. assembly-level knowledge.) That test saved both parties a lot of time.
But like I said, there is good testing and bad testing. Often tests test passive knowledge, but not problem solving skills. Unfortunately the hardest to quantify stuff is also the most essential in terms of actual productivity.
.: Max Romantschuk
It is true that it is difficult for an employer to tell a good employee from a bad employee. Sadly, this has lead to what I can only call "hiring voodoo" -- the irrational belief without evidence that a relatively untrained interviewer will mysteriously be able to find out more about "what a candidate is really like" in an hour than the candidate's university or co-workers (references) found out in several years. Even stranger beliefs have cropped up over the years -- eg that artificial toy questions like "why are manhole covers round?" or "... how would you identify the heavier ball in only two measurements?" say anything meaningful about how a candidate thinks, any more so than handing them the Times crossword to have a go at.
There is what's humorously called the oncologist test for the 'puzzle' questions in interviews. "If you had cancer, would you ask your oncologist this question before you let him mess with your body?" After all, your body is both more complex and more mission critical to you personally than whatever it is you're hiring the candidate to work on, so surely it matters much more how the oncologist thinks...
My experience is that the majority of employers and the majority of employees are equally stupid and deserve each other. If you're at an interview and they seem retarded then you probably want to move on.
Anyway, a person can pass the kind of stupid tests given at interviews and still be a retard. I wouldnt't give such stupid tests to people I hire and wouldn't submit to such a test.
The best thing an employer can look for is a portfolio. Look the work over, ask questions about the work, double check that it isn't just stolen from some open source project. If their work is good, even if unrelated to what you're doing, then they'll be good. If not, or if they lack a portfolio, then toss them.
If you're going to claim to know Java then write a program in Java and put it in your portfolio. If you're going to claim to know Linux then write some tools to make managing a Linux server easier and show you know common command-line programs and config files. Do that sort of thing and then employers can know what you know.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
Any idiot can spend 5 years saying "Thank you for calling Dell", fix their neighbours PC's for a while, read C++ in 24 hours and call themselves a IT consultant without any repercussions.
Wrong. It is only that incompetence in IT is much harder to cover up than in those professions. When IT systems fail, they can fail spectacularly and effect wide numbers of people. An incompetent IT persons mistake will cause an essential server or the like to fail. If they're not competent to fix it promptly, it will show.
Inversely, when a lawyer, accountant, sales, HR person, etc screws up, the screw up will not be noticed as much unless it reaches epic proportions. It's easier to mask a mistake in these fields, and with the softer ones, e.g. PR, their metrics are so fuzzy that the difference between competence and incompetence is blurry anyway. Plus they are trained in buzz speak which they blurt out like a frighted squid spurts out ink to mask their escape.
May the Maths Be with you!
1. Yep. Let me even give an example. It didn't happen in a team I was in, but I know several people from that team.
So they got a new guy who had some outstanding experience, according to his resume. He had worked on major enterprise projects, been an architect, ate Enterprise Java Beans for breakfast, etc.
Turns out he was utterly incompetent. He spent about a month just getting used to their architecture and IDE and everything, apparently everything they did or the way they did it was new to him, and he needed some time to accomodate. Fair enough. Then started working on something, but never was quite done with it. Eventually they started asking to see some results. He started randomly changing files and checking them back in. The first few times he even had a good excuse, like "oops, I hadn't worked with this particular versioning system before" or "oops, I forgot some other file that mine depends on." There go a few more weeks, before it's obvious that his changes can't possibly even compile, because they have elementary syntax errors.
Eventually they fire him, but by now he's got several months of "experience" there.
Then someone finds his updated resume online. The guy claimed he singlehandedly improved their architecture, increase performance X times, got project management back on track, etc.
2. 'Nother example, my ex-coworker Wally. Spent two years on a trivial module, whose core someone else rewrote from scratch in 6 hours. It took another two weeks or so, mostly of testing, to get it bug-for-bug compatible with his, since a couple of teams already had their own workarounds for them. (Trying to get him to fix it was a bit like negotiating with the terrorists.) The rewrite was also benchmarked as 40 times faster than Wally's on large data sets. Literally. Measured.
The thing everyone remembers fondly about him, is how he asked for 2 weeks just to estimate the effort to fix a trivial bug. He got it too. (His team leader was a bit a Mr Testicle: technically he was involved, but he kept out of it as much as possible;)
He also massively practiced obfuscation. Any of his modules contained half the techniques from How To Write Unmaintainable Java code (literally) and megabytes of files copied from unrelated stuff to pad the number of lines of code per day. Obviously, it worked on his team leader.
Then he got moved through the maintenance of two other programs (one at a time), and just managed to make them both worse.
There we go, that's his provable 2-4 years employment. Well, ok, 5 in his case.
3. Example number 3: Old Father Williams. I got to think of him that way after a particular fortune on my linux box:
Pretty much spent 6 years in a place complaining about everything that everyone else did. Coding style, IDE, OS, _everything_. His first choice of a whine was Windows, which might even have had a point, but when Linux was finally allowed and half the team switched to Linux, plus the servers actually went Linux... he proclaimed Linux to be sell-out crap for idiots, and switched to preaching BSD.
He also caused a reformat-and-commit war in which he was preaching _three_ space tabs, as spaces. And wasn't affraid to check out someone else's project and reformat it, to make his point.
He spent two years, just "modernizing" the build process. Nobody knows what he experimented with on his c
A polar bear is a cartesian bear after a coordinate transform.
BTW, I prefer self learners as programmers (read: coders) over people with degrees. My experience is that they are more dedicated and know more about real world problems. That's simply because a CS degree is not focused on making you a programmer but a more general problem solver.
So when you want a coder you have to check what he really knows, not what diplomas he's having (that only gives a hint). Story is completely different if you're looking for someone higher up the food chain, of course.
Sadly, a lot of the "IT Professionals" I encounter are plain idiots. Even in-depth interviews can't guarantee that you have someone capable in front of you, but it does filter out those idiots.
I work for a small (5 ppl) IT-only company and when we hire someone, while he will get some basic training, he is supposed to work pretty independantly. But once in such a position you can pretend doing a lot while doing almost nothing, and still make things appear to 'work'. You'd be amazed what an incompetent guy can pretend to be and produce results that on the first glance seem to be OK. And then when his software goes into production you suddenly notice that he didn't use an XML parser, but expected certain data on certain lines and filtered it out using regular expressions - and NO, not using the standard regular expression library - but doing something like this in C:
sprintf(cmdbuf, "/usr/bash /bin/sed -e \"s/%s//\" > /tmp/filename", inputbuffer); ...
system(cmdbuf);
fp = fopen("/tmp/filename", "r");
You get the picture. He btw didn't even write a function to do this, but copy-pasted stuff like this a few 100 times... Software worked in test, client changed 1 insignificant thing in their XML generation (added a tag we didn't use), and our entire system went down. I ended up rewriting this guy's stuff after he was fired.
And that's the main problem with IT jobs, you only notice they're incompetent when things start to go wrong. And then it's too late. So if I have to interview someone for an IT position, I want to be as sure as possible we don't end up in such a situation again. Masking incompetence in an IT position is just too easy.
Not quite,
at school you can guess quite well what the questions will be, so with a bit of 'educated guesswork' you can pass any exam without really knowing 'everything', let alone 'understanding' it. Heck, you spend over 10 years learning to 'work' the system, it's no surprise one gets good at it.
When we hire people we try to prune out those that either simply wrote the right words on their CV and/or those that worked their way through education purely based on the above way. Not because we think they 'cheated', but because we are looking for people to help us with a certain task that involves certain skills. (This is for development job, I'm not sure how the Sales department does it's selection =)
It's amazing how often people will write to be 'very good' at eg. SQL while all they know is that it stands for "Structured Query Language". When asked to write a query 'out of thin air' to get the most recent date from a simple agenda-like-table and they are unable to come up with ANYTHING, then we both know where are wasting each others time.
Before we tested people, we got burned once too often by people who bluffed themselves into the company but turned out to be more of a burden than a helping hand =( By introducing simple tests we now only waste time at the interview level, we don't have to put time into educating them something they claim to be expert in already. That said, we sometimes DO hire people who /fail/ the test, simply because they show potential and we ARE willing to put time & effort in them. You'll find though that this will is a lot less present when the candidate's CV turns out to be 90%+ 'vapoorware'.
If there is one thing to be learned on slashdot, it has to be sarcasm.
OK, google "fizzbuzz". A large number of people in the industry (especially "qualified" ones, who haven't been selected for skill) have no idea how to work with computers. People plagiarize at university, get friends to sit their exams, and lie on resumes. There is no better indicator than an on-site, in-person coding test. Some tests are better than others (some employers are not too competent themselves), but there is no other way to verify whether a potential hire is remotely competent. It's not the only indicator (other indicators can be used once the candidate has been pegged as potentially useful), but failing to use it is suicide for any business that can't afford to have worse than useless programmers.
A couple of real life experiences:
CV: Two Years Oracle Database Experience
Real life: I wrote some hibernate code that ran against an Oracle database.
CV: Experience of XML and XSLT
Real life: I configured tomcat, that's XML. XSLT? Isn't that the same thing?
CV: 5 years. Java, C and Python. Real life: I wrote some C five years ago and changed it again recently. (his Java experience was fine). I edited a python program once when the input format changed, no I really couldn't write anything from scratch.
One of these actually got the Job, because he apologised for his CV and then gave a real account of what he knew that matched our tests. He said the agency put all that rubbish in after he filled in a check-box questionnaire!
I went to a job interview in '99 for a contract doing Network Admin for a pretty major bank; I had no certs, no degree at the time, but I had been working off and on with Tek systems for several years and they knew I had extremely extensive experience.
The Bank didn't want to interview me, but the recruiter sort of insisted; they were asking for people qualified in NT, Solaris and OS/2, and I was really about the only person they had available at that time with the right mix.
It was a working lunch interview; They started asking questions, and I started answering. then came the question, "what command would you use to upgrade a NT workstation machine to NT server?"
I replied that you would probably be best off formatting the drive, then installing it, as there was no good way to upgrade; Microsoft said you couldn't do it at all, and the workarounds were more trouble than they were worth.
The interviewers sort of grinned, and told me that of the 20+ people they interviews, all of which had at least a MCSE or a comp sci degree, not a single one of them had answered the question correctly.
At the time I had problems believing it, but as time went on and I got in to situations where I was doing interviews it got more believable; in the late 90's if you worked on computers, it was probably because you were a computer enthusiast and actually more or less enjoyed working with them; after about '98, you started running into people that were just doing it because it paid well; they might be damned smart people, but you lose something when you don't actually enjoy working with computers.
I also saw a lot of people who just were not smart enough, but were somehow able to cheat or memorize well enough to get a degree; when you asked them something that wasn't in anything they had studied, they didn't have the core of hands-on knowledge that would enable them to make an educated guess at the answer.
So, yeah, I have to agree, interview everybody no matter what their credentials are.
Why, yes, I AM a Pagan Libertarian.
... to filter out people who think they are so good they don't need to test or even interview for a job. This way we keep our team full of humble people who actually want to work here. We got the test off the web, I'm not even sure which language it's in.... :-)
The problem lies in what I've been saying for 10 years: There's a much bigger range of expertise between managers and workers within the IT field than in any of the others you mention.
In other words, a "manager" in HR, accounting, or law can perform the tasks of the people they manage to a much greater degree than can an IT manager.
The HR manager knows enough about HR to spot someone who doesn't. The same applies to nearly every other profession - except IT.
IT managers often know very little about the technologies, tools, and languages used by the people they manage. Therefore they have to rely on tests to determine whether potential employees are "BS'ing" or not.
The IT industry grew into existence much faster and very differently than other industries, and because of that the management structure evolved very differently, and in a much more reactive way.
The only solution is to change the qualifications used for IT management. If IT management positions are filled by individuals who possess the same level of expertise in their field as as that of management in other fields, there will be no need for these tests.
"Why are IT professionals treated differently and in such a paternalistic way? More importantly, why do IT professionals accept being treated less favorably than members of other professions? Should IT professionals start to refuse to be treated as not real professionals?"
Others have already pointed out that lawyers and accountants (CPAs at least) submit to testing and are certified by professional organizations. You can't market yourself as a lawyer, CPA, or even an engineer in some places without having the backing of a professional guild.
What I'd like to know is why, in the face of offshoring and job losses, the IT industry hasn't coelesced around a professional society or guild. A professional guild, with some rigid certification testing, would be more effective than even unionizing since it produces a win-win for both employees and employers. Is it just that the need isn't perceived to be there yet?
With professional certification, employers would know they are getting skills without expensive testing and competent IT professionals can be assured that they won't be working with "IT Professionals" whose sole IT experience is that they took one Visual Basic course.
There are lots of vendor specific certs (MS, Novell, Oracle, IBM) but to me, that's more akin testing accountants for having skill in using QuickBooks.
The Open Group has IT Architect certification (http://www.opengroup.org/itac/) which looks to be a start, but it doesn't appear to have gained much momentum. IBM offers cross-certification of its internally certified architects but even within IBM, not all departments bother to pay the fee for TOG certification.
I also see that there's an Institute for Certification of Computer Professionals (http://www.iccp.org/iccpnew/index.html) that's been around since 1973, and a lot of people on its web page have important looking letters next to their names (CCP, CDMP) but outside of this web page, I've never run across an IT person with this on their business card nor a company that insists on this certification.
So I ask the exact opposite question that the poster asks, "Why don't we insist on recognized industry certification testing for IT professionals?"
JoAnn
funny enough I know more self-taught IT pros that know what the fuck they're doing than the certified "pros" who look lost when a real problem arises.
Most IT tests are jokes anyway, most dont reflect real world situations at all.
knowing what each OSI layer does in the OSI model and how to convert an IP address into binary arent used in the real world.
the other problem is these tests require one to cram and just learn for the test, same with the certification courses.
you learn how to take each quiz then learn how to take the final test.
They need to create a test (for networking professionals) that requires them to tackle a real world issue. intentionally fuck things up and then require the students to fix it with what they know.
I think the fundamental justification for a test is that many people in IT (programmers especially) are more artists/craftspeople than professionals. Sure, thinking is a large part of the job, but mental product isn't the sole output (as it is in law, HR, and other professions you mention that don't get tested). Just as you would expect to see work from a carpenter, plumber, or film maker before you hired them, an employer can very reasonably ask you to demonstrate your skills. Just because someone has degrees, certifications, and experience doesn't necessarily mean they have elegance, finesse, or artistry when it comes to writing code or debugging systems. A lot of what the industry values in its employees can't be taught, but it can certainly be tested. Giving a potential hire a code sample with a subtle bug you can judge their process in ways you can't with mere questions. Similarly, asking an interviewee to answer questions on the spot allows them to demonstrate how they code in an ad-hoc environment. This is especially useful for identifying people who don't have enough experience to develop independently.
Also, many professions, such as the law, have certifying authorities (such as the bar) where practitioners must pass a minimum standard in order to join the profession (and can be kicked out of). Because IT has no such standards body, and because so many people in IT are self taught or taught on the job, it's very difficult to know how much someone knows. There is no 'cannon' of IT training, no standard practice, nor even much agreement over what constitutes an adequate body of knowledge. The fact that people with no formal training regularly conceive and develop systems far superior to those with certifications, training and experience continues to speak to this fact.
That, and a code example. You send us a code example after the phone interview, and before the in person interview. Basically, we're looking to see that you "don't do anything blatantly jack-ass". Also helps when you comment your code.
Then, as part of the interview process, we have a few questions we ask you to write essays for. One is based on design of a product we already built, one is based on design of a product we're currently building, and third... Is actually a riddle. My project manager came up with this idea, his thinking is "Let's see what kind of inductive/deductive reasoning this guy will use". Sounds like a good idea.
Turns out? All this crap is worthless. Case in point... We have a guy fresh out of college (no experience in the trenches) who we interview, and later get hired. His code is quite beautiful at a glance (not breaking down every line to profile it or anything), plenty of comments, nice style, etc. His essays are OK, they're passable.
But, he sits down, and we've got -multitude- problems. Let's start with one example: We're web developers, and in the first couple weeks, he needed to modify UI. In this case, he needed to use a few images for something. What kind of images did he put into our repo for versioning? BITMAPS! No, no, not a proper raster file-types like a PNG or a JPG, a bitmap -- BMP. Just cause Professor Dinglethorpe requires you to comment and indent your code properly doesn't mean you have a clue what really happens when you get down to production.
But, the real problem? COMMUNICATION! The kid just can't freakin' communicate with us. If he were bad AND would take the time to talk to us about what he's working on, we could stand it. We'd know what was happening when he did jacked up stuff in the code, and we later have to maintain his mess.
However, he doesn't take time to communicate with anyone. He's too busy leaving important meetings to take phone calls from his ultra insecure live-in girlfriend who calls him 18 times a day (for such important things as "Should we make lentil soup tonight?" and "What are we going to name this cat?"), reading I can has cheezburger and the failblog. Meanwhile, he slips under the radar. Our company plays to your competency level. So while I have taken on huge projects, become a stand-in for our system administrator, and the liason between customer service and information services (a pretty important role, they buy the important bugs -first-) -- this kid is getting assigned tasks like "We need to put hyphens between these words per the marketing dept". Good thing we get paid the same. Nice to get paid the same as the guy who's mastered reading I can has cheezburger. He can has cheeseburger, and I'll be the one to shove it up his... nose.
So, don't believe a technical test is going to determine if the next guy you work with is legitimate, and competent. If he can't communicate, and he has no work ethic -- frankly, you're screwed.
I feel they are critical.
We used to give a very simple test. Write us an EJB application (simple calculator webapp) that will run on jboss. They had a few hours to do it.
We only had two people out of over 50 that recruiting companies sent us over the years who actually completed it in the time allotted.
Now I should say we were less interested in them actually solving the problem than in their approach to solving the problem. They had full access to the internet and could use any open source or free software/tools they felt necessary.
- We had people who spend their whole time downloading Eclipse and plugins to build the app for them.
- We had someone writes us several pages in english on paper how they might write the application.
- One ended up getting the computer stuck in an endless cycle of reboots.
- A number just gave up
- Most really didn't knwo what an ejb was or how to write one (as part of the review we checked browser history).
All of the candidates were sent to us as Advanced Java developers with years of experience writing enterprise apps.
We told people up front, completing the test was not a pass/fail thing. (Although if someome completed the test they were hired on the spot.) We were looking at their work to assess:
-creativity
-troubleshooting skills-
-focus on the task at hand
-coding style under a tight deadline
-research ability
-how tool dependent they were
In the end, the only two that actually wrote the damn thing (outside of the internal developers we timed) just used notepad.
I didn't get the job, but found out several weeks later that they implemented my exact solution, as the guy they hired for the job EMAILED ME WITH QUESTIONS and quoted the entire email.
I submitted it to their billing department at my standard consulting rate and minimum bill, with a note attached that since there was prima facie evidence that they were using my solution. . .it was pay or go to court.
The check arrived via FedEx the next morning . . .
I'm a systems architect with many years experience. No real programming skills which the coding bunch will find odd and look down upon. Simply I hire programmers when I need them and usually have several on staff. I design, they code, I deploy.
Working on a big deployment my company hired a third party contractor for programming. The guy they sent was ok but seemed to be struggling with the project. I stepped in and reviewed the software and requirements and discovered he was doing it all wrong. He was fired and someone else finished the project.
Time passed and I applied for a job at another company. Was given a brief programming test which half of it I couldnt complete. Not a programmer and I explained that to them. They didnt care too much. As for the test I didnt mind and think its a good idea especially if its a programming position.
Now for the punchline... I did not get the job. It went to a guy who completed the test. The "same" guy we fired for incompetence on our project.
Anybody can write code. Its what you do with it thats important :)
people on ludes should not drive