Ask Slashdot: Are Timed Coding Tests Valuable?

First time accepted submitter xkrebstarx writes "A buddy of mine recently applied to a large tech company. Before setting up a phone interview with him, the unnamed company issued a timed coding test to gauge his coding prowess. He was allotted 45 minutes to complete an undergraduate level coding assignment. I would like to ask the Slashdotters of the world if they find value in these speed-programming tests. Does coding quickly really indicate a better programmer? A better employee?"

Comments

I dunno...

[Joce640k]

I dunno...but it will weed out the pretenders/bullshitters.

Re:I dunno...

[beelsebob]

The other thing it'll do is near guarantee that they get some bugs from all applicants, they'll then have something to discuss in the phone interview, and gage the applicant's responses and proposed fixes.

Re:I dunno...

[Anonymous Coward]

Exactly. I don't believe that a speed programming will always identify the "better" programmers or employees as the OP appears to be making a correlation between.

I would suspect that the company is just trying to weed out those who actually have no experience, not those who are "better". It is just a method of screening which can be very effective when dealing with large numbers of applicants.

Re:I dunno...

[hackula]

I never believed the whole "95% of interviewees fail the FizzBuzz test" until I started interviewing candidates. People with 15 years of "experience" on their resume would regularly fail or give up. I also encouraged googling, including just searching for the exact problem, and I encouraged questions and told them that both behaviors were seen as a good thing. IDK how someone could possibly get through a CS program and still fail this test, but it happened regularly.

Re:I dunno...

[SJHillman]

Mod parent up

I took a couple of programming classes in college and we had some timed tests. However, failing to get the desired results within the allotted time was only enough to knock off a partial letter grade (IE: go from a B+ to a B). The professors mostly wanted to see how you approached a problem, whether you commented your code, if any horrible coding practices jumped out, etc. In the follow-up, they would also want to know how you would fix any issues you came across if given more time.

How much you got done was still somewhat important - there was a basic expectation that you would have some functionality working and other pieces at least partially completed. After all, if you have to spend 45 minutes looking up how to do a basic loop, then you're probably not experienced enough for anything beyond entry level (if that), so it helps weed out the people who are outright lying about skills.

Re:I dunno...

[Anonymous Coward]

wth is a fizzbuzz test?

Re:I dunno...

[Cryacin]

These tests are vital when interviewing. We recently put out an ad for a senior programmer, and got about 100 or so responses. After 25 responses hit the circular filing cabinet, as they were obvious resume spam with no cover letters, we declined around a further 50. Of the remainder, we asked a simple question. In JavaScript, without using the reverse() method, reverse an array of numbers containing 1,2,3,4,5 in the most efficient way possible.

We got 5 responses back, which we interviewed. The other 20 were out. These tests are meant to weed out the crap that would waste our time, and honestly, the 5 guys that responded, responded in under 5 minutes.

Re:I dunno...

[beelsebob]

Well, you just failed for not googling...

But, print all the numbers from 1 to 100, except rather than printing multiples of 3, print fizz, rather than printing multiples of 5, print buzz, and for multiples of both, print fizzbuzz.

Re:I dunno...

[Gr8Apes]

so this should take you what, about 3 minutes, given a normal typing speed, compile and run time?

Re:I dunno...

[beelsebob]

And yet, as stated above, surprisingly, 95% of job applicants fail, just like the AC above.

Re:I dunno...

[beelsebob]

Which is why this test is useful ;)

Re:I dunno...

[hackula]

Should take 3, I give 30. Still fail. Also, any language they want no matter what the job requires.

Re:I dunno...

[deniable]

It sounds like less of a speed comparison than an upper limit to stop people wasting both sides' time.

Re:I dunno...

[arth1]

To be fair, there are a lot of senior programmers who don't speak javascript. They may have experience in it to the point of setting up the algorithms that the regular programmers implement, or assisting with eyeballing for glaring errors, but may not be javascript programmers themselves.

An efficient solution would, depending on the language, be to either change the pointers of the array, or create a new index to replace the existing one, and thus avoid copying each element, but I can fully understand seniors not knowing the best way to do this in an arbitrary language without looking it up.

But the most efficient way possible (and that's what you asked for), given that it only has five elements, all known, would likely be something like (not that I speak JavaScript):

a[0]=5;a[3]=2;
a[a[1]=4]=1;

Re:I dunno...

[serviscope_minor]

I never believed the whole "95% of interviewees fail the FizzBuzz test" until I started interviewing candidates.

Well, I've never interviewed job candidates and I have a hard job believing this. It's not that I think you're lying (I don't), it's that I've never seen it for myself and my brain has a very hard job accepting that someone who does programming for their livlihood could not solve this in their sleep.

It doesn't matter that it's true, it's just really REALLY hard to imagine. And I don't WANT to believe it.

Anyway, here's my entry.

s/^$/c_0/
:mainloop

#Advance the number counter
tclear1
:clear1

s/9$/0/;tdigit1done
s/8$/9/
s/7$/8/
s/6$/7/
s/5$/6/
s/4$/5/
s/3$/4/
s/2$/3/
s/1$/2/
s/0$/1/
:digit1done

s/_90/_100/
s/_80$/_90/
s/_70$/_80/
s/_60$/_70/
s/_50$/_60/
s/_40$/_50/
s/_30$/_40/
s/_20$/_30/
s/_10$/_20/
s/_0$/_10/

#Advance the Fizz counter
y/abc/bca/

#Clear the hold space
x
s/.*//
x

#Divisibility by 3 /c/{
        x
        s/$/Fizz/
        x
}

#Divisibility by 5 /[05]$/{
        x
        s/$/Buzz/
        x
}

x /^$/g
s/[abc]_//
p
x /100/!bmainloop
D
q

Silly slashdot. Why do you keep reformatting my code and why do you make me type junk here just to get my post accepted? Blah blah blah blah blah blah and you don't like too much repition wither this makes it very hard to paste code into the forum now please let this post in please pretty please with a cherry on top and sprinkles and is that even your bucket in the first place?

Re:I dunno...

[Anonymous Coward]

-- pseudo-code
for i in 1 to 100 loop
      if mod(i,15) == 0 then print 'fizzbuzz' ;
      elsif mod(i,5) == 0 then print 'buzz';
      elsif mod(i,3) == 0 then print 'fizz';
      else
              print i;
      fi
end loop

-- 4 minutes to write pseudo-code, maybe not elegant, but not googling
-- Depending on the language, I'd have to to look up the print, loop, and modulus syntax

Re:I dunno...

[beelsebob]

You know... Every time fizzbuzz is explained somewhere, there are always 100 attempts to solve it right beneath...

I think this is the first time I've ever seen the first response (or in fact, most of the responses) actually be correct!

Re:I dunno...

[samkass]

Would "I'd delegate it to a Java Programmer" be a valid response?

No, it would weed you out because it shows you don't know the difference between Java and JavaScript.

Re:I dunno...

[IICV]

Honestly, you didn't even need the "without using the reverse() method" stipulation.

Re:I dunno...

[smpoole7]

It's not only a great idea to weed out the wannabes, watch how they react when you challenge them.

The ones whose eyes light up and are eager to prove themselves? That's your guy or your gal. :)

Re:I dunno...

[Anonymous Coward]

That's easy; I'm a fast typer!

#include <stdio.h>
int main()
{
puts("1"); puts("2"); puts("fizz"); puts("4"); puts("buzz"); puts("fizz");
puts("7"); puts("8"); puts("fizz"); puts("buzz"); puts("11"); puts("fizz");
puts("13"); puts("14"); puts("fizzbuzz"); puts("16"); puts("17"); puts("fizz");
puts("19"); puts("buzz"); puts("fizz"); puts("22"); puts("23"); puts("fizz");
puts("buzz"); puts("26"); puts("fizz"); puts("28"); puts("29"); puts("fizzbuzz");
puts("31"); puts("32"); puts("fizz"); puts("34"); puts("buzz"); puts("fizz");
puts("37"); puts("38"); puts("fizz"); puts("buzz"); puts("41"); puts("fizz");
puts("43"); puts("44"); puts("fizzbuzz"); puts("46"); puts("47"); puts("fizz");
puts("49"); puts("buzz"); puts("fizz"); puts("52"); puts("53"); puts("fizz");
puts("buzz"); puts("56"); puts("fizz"); puts("58"); puts("59"); puts("fizzbuzz");
puts("61"); puts("62"); puts("fizz"); puts("64"); puts("buzz"); puts("fizz");
puts("67"); puts("68"); puts("fizz"); puts("buzz"); puts("71"); puts("fizz");
puts("73"); puts("74"); puts("fizzbuzz"); puts("76"); puts("77");
puts("fizz"); puts("79"); puts("buzz"); puts("fizz"); puts("82"); puts("83");
puts("fizz"); puts("buzz"); puts("86"); puts("fizz"); puts("88"); puts("89");
puts("fizzbuzz"); puts("91"); puts("92"); puts("fizz"); puts("94");
puts("buzz"); puts("fizz"); puts("97"); puts("98"); puts("fizz"); puts("buzz");
return 0;
}

Re:I dunno...

[ShanghaiBill]

And yet, as stated above, surprisingly, 95% of job applicants fail, just like the AC above.

I used to give people this test: Write a program, in any language, to read a list of number from stdin, sort them, and write them to stdout. For instance if the input is:

23
14
375
42

Your program should print

14
23
42
375

By far the most common "solution" was this:

main()
{
      printf("14\n")
      printf("23\n");
      printf("42\n");
      printf("375\n");
}

Most of these people had degrees in CS.

Anyway, I hired the girl that wrote this:

%!/bin/sh
sort -n

Re:I dunno...

[hackula]

Nope, but you might try a Javascript programmer. I would be willing to be a Java programmer would implement this using the ReverseFactoryAbstractFactoryFactory pattern though. They might be able to get it to you in a year, but you would be obligated to sign a 5 year 200k/yr support contract.

Re:I dunno...

[mcmonkey]

In JavaScript, without using the reverse() method, reverse an array of numbers containing 1,2,3,4,5 in the most efficient way possible.

Semi-off topic question: really? I mean, are we (programming job seekers) as a group that dumb?

I recently went through searching for a programming job after 5 years in a position that mainly dealt with configuring off-the-shelf apps. Every single interview at some point had a question that required reversing an array without using Reverse(). (Only about half mentioned efficiency.)

I'm big into puzzles and logic games, and as long as I can use pseudocode and not stress syntax, technical interviews are truly fun. I enjoy coming up with algorithms to solve interesting problems. But reversing an array is not an interesting problem; it's trivial. It got the point where I felt like I should walk in to an interview, introduce myself to everyone in the room, and go right for the white board and put up an array-reversing function, just to get it out of the way.

That a significant number of applicants for a programming position couldn't (or wouldn't) solve this quickly is sad commentary. (It's even worse if you consider it's likely yours would not be the first position these folks have applied for. Based on my experience, they would have been asked the same question previously, and not only couldn't come up with a solution, but also didn't think, maybe I should figure this out before I send out any more resumes.)

Re:I dunno...

[Anonymous Coward]

Should start with:
#!/bin/sh

But anyway I'd fail most of these programming tests :).

Re:I dunno...

[MrBandersnatch]

It took me 6 because I spent 5 minutes trying to find an elegant solution to the problem - do all older programmers suffer from the problem of premature optimization? ;)

Re:I dunno...

[funwithBSD]

It does not say any of the things you just said, it says:
without using the reverse() method, reverse an array of numbers containing 1,2,3,4,5 in the most efficient way possible.

You are implying that they were "supposed to reverse the array in-play" the way "reverse() does".

Unless you are implying that is the most efficient way to do it, period.

One could also presume that the way reverse() does it is wrong, inefficient or "broken" in some way for this use case, or why the fuck are you wasting my time refactoring reverse()? Use reverse()!

Personally, the whole test stinks of: If you were too stupid to look up reverse(), how would you hack around the problem?

 

Re:I dunno...

[seebs]

Exactly. That is the only person who considered the spec and reacted intelligently.

I'm pretty good at shell, and one of the things I note is that a TON of stuff that people often write elaborate code for can be done in a couple of lines of shell and then we're done.

Re:I dunno...

[funwithBSD]

And I probably would hire him.

IF he is good enough to take my stupid test and shove it up my ass for wasting his time but without refusing the request, he is the kinda of guy I want one or two of my team. Or more. My boss is this guy, I am this guy, half the rest of the team is like this guy.

Those guys are the kind of guys you can give a "Letter to Garcia" to and know it is going to get there without much intervention on your part.

And those are the sort of guys that make complex datacenter migrations/refits successful, even if you have to provide air cover because some PM tried the same stupid trick and he shoved the test up their ass and they did not like it.

Re:I dunno...

[SCHecklerX]

It's just that most bash 'functions' are other programs. I see no problem with her answer.

If you chose perl instead, it wouldn't be much more:

#!/usr/bin/perl -w

@list = sort {$a <=> $b} @ARGV;
foreach(@list){
      print "$_\n";
}

But I wouldn't have remembered how to make sort sort things numerically, and would have just done what she did with shell :-)

Re:I dunno...

[Anonymous Coward]

I don't understand why employers don't just ask to see some code the developer has already written. Some people (nerds) don't do well in social situations and it takes a few days to get into the swing of things.

I really don't get the "write a merge sort" routine. When was the last time someone did this IRL? At least people can use google, etc. When I was outta college, we had to do this crap on a whiteboard with 5 guys staring at the back of your head with no references. Talk about nerve racking.

I would think the best possible thing to do would be to ask the candidate to write something before the interview and give them a few days. That way they are in their comfortable environment and can do their best work, proofread, comment properly, debug properly, optimize properly, etc. That's what you're really after, not someone who can write buggy ass, un-commented, un-optimized code really fast.

Re:I dunno...

[narcc]

No, most older programmers suffer from "do it right" syndrome.

I wrote two different solutions and selected the most readable. It took 8 minutes, but I had to chase some damn kids off my lawn.

Re:I dunno...

[Goaway]

On the one hand it seems kind of lazy.

Programming is pretty much Applied Laziness, so that is probably a good thing too.

Re:I dunno...

[FictionPimp]

Yes and no.

I have taken programming tests for jobs. I tend to do really well and have gotten quite a few jobs and job offers from my tests. I excel at tests that give me a few days to complete some minor 'real world' type project. These jobs typically had more knowledgable staff and they used my 'test answer' application as a launching point for a discussion on my ability. They did this by challenging my design decisions, asking questions about areas in the code to get my insight, and asking me how I would improve the program given more time, or improve the test.

I have also bombed tests that were timed asking programming questions. Many of these tests asked for simple things (print out a multiplication table to look exactly like this example) and I simply brain farted because I don't typically write applications that need console formatted output. I end up looking like a idiot to the test givers. I found no value in these tests asking me to write a function that factors, or write a loop that prints out hello 10 times. Interviews after these tests tended (when they happened at all) to be lower quality and the employees seemed less knowledgable. There was much less meaningful discussion about my abilities, even when I 'aced' the test.

When I do my job in real life, I have time to plan, research, and I can keep open documentation for reference. I don't need to memorize every obscure function in the language my job requires, instead I can pull up documentation on how to use that object, function, or method. Many tests I have taken do not allow for this and it has cost me a few job interviews. While you have stated you encourage googling, most tests I have taken have explicitly denied the ability to search. I'd much rather have a week to build a working application then 45 minutes to prove I know how to write boilerplate code and memorized a whole bunch of console formatting functions.

Re:I dunno...

[TheSpoom]

Lazy, in a programmer, is a virtue.

Re:I dunno...

[maroberts]

There's the straight version...

#/usr/bin/perl
for my $i (1..100) {
      my $s = '';
      if ($i % 3 == 0) { $s .= 'Fizz'; }
      if ($i % 5 == 0) { $s .= 'Buzz'; }
      print $s ? $s : $i, "\n";
}

And for Real Men

perl -E 'say map{((Fizz)[$_%3].(Buzz)[$_%5]||$_)."\n"} 1..100'

Re:I dunno...

[RobertLTux]

"
@
%!/bin/sh
sort -n

That isn't portable, should fire her."

OBJECTION: feature not specified/requested therefore was considered optional

Re:I dunno...

[coder111]

1 year to write an array reversal? And I get 200k/year for 5 years? Sign me up!

Java must be the BEST programming languatge (for programmers).

On a serious note, bad code is very often management's fault for

a) Rushing things
b) Never giving any time nor resources for refactoring and maintenance
c) Never caring about software quality in general

I'm dealing with such a system right now. It's crap, but management won't allow me to refactor it much. Adding new features is the only thing they see, even though we are at a point when adding anything costs at least a month because of all the interdependencies and spaghetti and extensive regression testing required.

--Coder

Re:I dunno...

[jonadab]

Depends on the language. 3 minutes sounds about right for C.

Any Perl programmer who doesn't have a working solution in under a minute is deadwood. Given three minutes, he should revise the solution until it's elegant, easy to read, and maintainable (so that e.g. the numbers 3 and 5 can be replaced). In five minutes, it should take command-line arguments for all the different possible numbers to handle and what to say for them. And then he should look at you and say, "You know, I bet I just duplicated a CPAN module."

Re:I dunno...

[greg1104]

An in-place copy is the most efficient way to do it, that part of the question was a hint.

Many algorithms have a space/time trade-off in them. If you're willing to allocate more memory, you can run faster. That makes memory size part of efficiency. Any implementation that uses more memory should see some performance gain in return, before it can be considered more efficient. Creating an array that's the same size of the input is therefore less efficient unless there's some compelling runtime gain possible by doing it. And here that's not the case. An in-place implementation takes trunc(n / 2) array element swap steps, the two array version n array element copy steps. Even if a swap takes longer to do than a copy, the fact that you're touching more memory means slower, including things like CPU data cache disruption.

If this were a C compiler and using a second array allowed using memcpy rather than coding your own loop, you could make a case that there's a more efficient way to do this with more storage. That's different from this problem in two ways though. The target is not that low level of a language, and there's normally no reversing version of a memory copy available.

When interview tests suggest re-implementing a standard library feature, the goal is normally to match it as closely as possible. You shouldn't presume anything about the reasons for that, beyond that it makes a useful test question. If I gave this test and someone arrogant programmer told me I was stupid for asking, because it's already there, I'd point out that you're unlikely to improve on a standard function without bugs if you don't start with cloning it perfectly. Perhaps this is step one toward an improved version, and the next step needs something like it as a regression test harness to validate faster versions. Stopping to complain that you don't understand why you're being asked to do this is also less efficient, relative to just doing it.

Re:I dunno...

[narcc]

Why copy the array? Just reverse it in place:

var i = 0;
ar len = input.length-1;
while (ilen) { input[i] ^= input[len]; input[len] = input[len] ^ input[i]; input[i] ^= input[len]; i++; len--; } return input;

Illegible, but more efficient.

What a stupid test.

Re:I dunno...

[fzammett]

Wish I had the mod points, but this is spot on.

I've frankly never been particularly good solving these "simple" algorithmic problems on the spot either, even with my 20+ years of experience in the field, even with being a published tech author, even with being very highly regarded by all I've ever worked with. So, although I haven't had the need to interview for a job in quite a few years, I'm sure I wouldn't do any better than average if these sorts of things were thrown at me.

But, compare the code I write for a living with the code most others do and there's no question which one of us you'd want to hire.

I think this is true of a lot of good developers... there's MUCH more to this job than being able to solve a specific single programming task because all of us that are worth hiring can do that, given enough research and time... sure, I agree, it's nice if you can throw algorithm X, Y or Z up on a whiteboard from memory, that'll make you marginally faster than me... but speed, while not irrelevant, is FAR from the most important characteristic of a good developer. Code that is clean, elegant, logical, as simple as possible and maintainable five years down the road are FAR, FAR, FAR more important, and you're NOT going to gauge those things with a fizzbuzz-type test (I do A LOT of interviews now, believe me, I know this from experience).

Re:I dunno...

[ShanghaiBill]

So how can one pass the HR screen and get an interview with "programming knowledge", and even a CompEng degree, but with otherwise no experience?

Obvious answer: apply to companies that pre-screen with on-line tests. At my company, if you do well on the on-line test, you will get an interview. At the interview, you will spend about 30 minutes chatting, and about 2 hours writing code. Some of the coding will be on a white board in a conference room with your potential co-workers, and some of it will be alone in a quiet cubicle.

We added the "cubicle coding" because some introverted people don't do well on the white board, but can shine when allowed to focus in a quiet cubicle. I learned this when I declined to hire an applicant because he tended to mumble and stare at his shoes. That evening he emailed me an extremely concise and well coded solution to the problem, way better than I would have done myself. I brought him back for a second interview, hired him, and he turned out to be one of the best programmers I have ever worked with. Eventually he became less introverted, and when he talked to me he would stare at my shoes instead of his own.

Re:I dunno...

[Anne Thwacks]

elegant, easy to read, and maintainable

You mean rewrite it in another language?

Re:I dunno...

[drolli]

You know, switching programming languages a lot can sometimes mess up your memory about the specific syntax. However, i think i would get a simple loop done in at least 5-6 languages....

Re:I dunno...

[Ihlosi]

People with 15 years of "experience" on their resume would regularly fail or give up.

I'm curious ... can you elaborate on how exactly they fail?

Do their attempts just have stupid bugs or is the whole approach wrong?

I'd probably fail at "printing". Only writing code for devices without any human-targetted outputs does that to you. :P

Re:I dunno...

[dward90]

While you're absolutely correct that a fizzbuzz test is not a good way to determine if you *want* to hire someone, I think you're missing the point of these kind of exercises. It's purpose is to weed out candidates who are a complete waste of time. Trivially simple programming tests don't tell you if a person is a good developer. A passed test means basically nothing. However, a failed test means that to spend any time interviewing that candidate is a complete waste.

Note that I'm talking about really simple stuff, with no real time limit and not caring about bullshit syntax (You forgot a semi-colon! Go home!). I would never expect an experienced developer to code out complex sorting or search algorithms from memory. Those tests, for sure, don't tell you anything.

NOPE.

[TheBlackMan]

If asked for simple answer, here it is.

Of course

[Bogtha]

They are extremely valuable - they let you know which potential employers don't have a clue about programmer productivity / expertise.

Re:Of course

[buddyglass]

I took one of these during an interview. They put me in a room w/ a computer my IDE of choice and had me do a project. The general gist was to build a program that read instructions from a file, parsed them, carried them out, then output the result at the end. The instructions all had to do with string manipulation. After I finished, they brought my code up on a projector and had four or five developers do a sort of code review, asking me why I chose to do various things in certain ways. So, not only did they test whether I could finish the project successfully, they got to see whether I can speak intelligently about design decisions and handle constructive criticism from other developers. Seemed like an extremely valuable exercise.

Re:Of course

[jittles]

Yep. I've done lots of programmer hiring and I can tell you that one of the worst employees I ever dealt with got a near perfect on a C test, finishing 45 questions in about 6 minutes. Some of the questions required coding. The guy was a terrible programmer, but he really knew the language. I think he missed 1 question and that was because the person who wrote the test had accidentally introduced a typo that made the "correct" answer wrong.

The way you just described it makes it sounds an awful lot like you were doing "language trivia" question which are well-known to be beyond totally useless, and not related to the timed coding test as implied in the question. So please, stop considering yourself a good interviewer now, and read up on what works.

As I said originally, yes there was code trivia. But yes there was a timed coding portion of the test as well. And no, that test was not my choice. I am not a big fan of riddles and tests in interviews. I personally know a good developer who is terrible when put on the spot like that. You sit him at his desk, where he feels comfortable and in control and he does great. If he's put on the spot, he is a total mess. Does that mean that he isn't worth having around? No. Is it something he's working on improving? Yes. But he makes solid contributions and hopefully some day he will be able to handle that kind of stress.

No undergraduate level stuff for me

[loufoque]

I personally discard any company that tests undergraduate-level stuff.
This kind of thing is completely irrelevant to the skills of a senior engineer.

Re:No undergraduate level stuff for me

[beelsebob]

I personally discard any applicant that thinks like this, it means they can't actually think about algorithms, data structures, and how to design something well, and in doing so will produce sloppy assed code.

Re:No undergraduate level stuff for me

[nedlohs]

You really think that would be for a senior engineer position?

Unless it's an ungraduate assignment from a very specialized course it's going to be simple to do in less than 45 minutes. The only assignment I have done or set at an undergraduate level that wouldn't have been so was the "write a ray tracer" from a computer graphics course - and only because I don't remember the math off the top my head (it almost writes itself once you have that), I'd need to reread some stuff first.

Is it going to "indicate a better programmer", No. Is it going to indicate a "better employee", No. It probably will weed a large number of the really bad programmers though. And sure it'll cull a few potentially good employees , that might be worth culling those really bad ones though.

Re:No undergraduate level stuff for me

[buddyglass]

You would be surprised at the (low) quality of some candidates whose resumes suggest they're qualified for a senior engineer position.

Re:No undergraduate level stuff for me

[tibit]

I don't care how senior you are. It's like saying that an experienced transplant surgeon doesn't need to know how to make an injection. Yeah, in their job probably they'll never inject anyone, but still, you need to know it just in case. Shit sometimes goes wrong. It's a wrong analogy! As for a senior software engineer: how on earth can you pretend to be able to design, you know, software, if you don't demonstrably have a clue how to, you know, make software for a very simple task? Such "senior engineers" will proceed to design a 100kloc framework for a fizzbuzz. Because, you know, it needs to fulfill whatever buzzword quota they came up with.

Re:No undergraduate level stuff for me

[hackula]

Irrelevant? A senior engineer should be able to knock this stuff out no problem. If they cannot, then there is no way they would be able to do the job. Of course, no need to pick the obscure stuff that no one remembers, but a few basic problems just to make sure they can write some code is essential. After that, then you can discuss the intricacies of whatever specialty they happen to have. Conversation seems to draw out the advanced skills much better, however, there are loads of "expert" coders out there who could not code their way out of wet paper bag.

Re:No undergraduate level stuff for me

[beelsebob]

The thing is, you may not be hiring a worldwide known linux kernel expert. You may instead be thinking about hiring someone who's CV says he's done several impressive looking things. You need to verify what his level of involvement in those was, and a good first step is often "are you so retarded that you can't possibly be telling the truth on your CV".

Re:No undergraduate level stuff for me

[0xdeadbeef]

Basic competence is irrelevant?

Paging Dunning-Kruger. Precious snowflake on line one.

Re:No undergraduate level stuff for me

[ShanghaiBill]

I personally discard any company that tests undergraduate-level stuff.

This is one of the reasons for the test. It not only filters out the incompetent, but it also filters out the arrogant prima donnas, who will probably not be good at teamwork.

Re:No undergraduate level stuff for me

[qwijibo]

I use the programming exercise to determine if the person's computer skills match their claimed skills. From the resume and interview, I should have a good impression of what the person has done and could do. If I describe the problem to them and they say they can easily do it in 10-20 minutes, that's a good opportunity to nut up or shut up. The better they do, the more confidence I have in everything else they told me. If they claim lots of experience and are uncomfortable being put in front of a computer, they're not the kind of person I want to hire.

Re:No undergraduate level stuff for me

[fatphil]

> If you want to evaluate his skill, ask him to do something that's relevant to the added value that's making you hire him in the first place.

This is good.

At one interview, I got asked to review a page of real code from their current development tree. Asked for three to five things, I spotted half a dozen major things, the same number of minor things, and a few cosmetic ones too.

I was on my first company social later that evening.

Re:No undergraduate level stuff for me

[qwijibo]

When I'm hiring for positions that are paying high salaries, I want to know the person is just as good (or better) in front of a computer as they are at interviewing and sounding smart. A simple programming exercise can confirm that the person is good with computers, not just as BS'ing their way through interviews.

It's disappointing how many applicants will fail such a simple demonstration of their claimed skills. Likewise, if someone thinks a short demonstration is beneath them, I don't want to work with them. Every job has some crap that just needs to be done, even if its a waste of everyone's time. If they haven't learned the "do it and move on" philosophy, they're going to be hard to work with. I've worked with people who will spend days arguing against doing 20 minutes of work - I'm definitely not going to hire people like that.

Re:No undergraduate level stuff for me

[beelsebob]

And that will cost £1000, while asking the guy to do a 45 minute test before a phone interview will cost about 5 minutes of my time, which is no where close to £1000.

Re:No undergraduate level stuff for me

[gregmac]

You've obviously never been involved in hiring developers.

There are a *lot* of bad developers out there. So many it's sickening. Bad developers that have resumes that look like they can do stuff. They may even call themselves senior. They've worked on a team that has successfully produced a product (or at least at a company that has).

One of the memorable interviews I did with was via a referral, and it worked out that I went straight to an in-person interview (skipping my usual weeding-out process). This person had a decent CV. They worked on a project designing a military helicopter training simulator, which basically involved wiring a game (written by another team) up to a 4-person helicopter mock-up that included pieces of real equipment (radios, navigation, etc) so the actual equipment displayed and could interact with the game. I've always had a personal fascination with interfacing real-world hardware with software (and have done lots of industrial control integration), so I had a ton of questions.

Well, despite trying for ~15 minutes, I could not figure out what this person actually had specifically accomplished. The team had successfully built this thing from what I could tell, but this person could not explain to me what *they* actually did. I asked in many different ways, including very bluntly like "What was some piece of functionality/code you wrote yourself?" and the person "could not remember" (they worked there for over a year, and had left less than a year prior). The most technical thing they could say about the integration was that it "involved UDP".. Seriously.

Re:No undergraduate level stuff for me

[n7ytd]

You really think that would be for a senior engineer position?

Unless it's an ungraduate assignment from a very specialized course it's going to be simple to do in less than 45 minutes.

But the fizzbuzz test isn't the only question to ask, and for anyone who's interviewing as a potential programmer, it should only take 5 minutes of interview time. The purpose is to avoid wasting anyone's time with the follow-up questions.

Yes, the fizzbuzz test is trivial for a senior engineer, but it is still valuable because a large portion of "senior engineers" will still fail it. There is a difference between engineers who have mastered the trivial and moved on to bigger things and those who have scraped by the trivial, never quite understanding it.

The last time I was involved in a round of hiring, they asked me to come up with 3 technical questions that we e-mailed to the candidates the week before the on-site, asking them to bring their solution with them to the interview. I had mixed feelings about the idea, since I usually don't code for companies that don't pay me, but we wanted to reduce the intimidation that some people have in interviews and allow them time to do their best work on their home turf. I wrote three questions to be solved with C: reverse a string given at the command line, print the prime numbers between 1 to 100, print an array of 10 integers in sorted order--that kind of stuff. I included my e-mail address and invited them to contact me with any questions. Fizzbuzz was given as the opening technical question at the on-site. While they pounded out their answer to fizzbuzz, we used that time to review their solutions to the other questions.

From just these 4 questions, it was amazing what we could learn. Some of the candidates didn't bring anything with them to the interview. Some asked for more time, as one week wasn't long enough to solve such complicated problems. Some solved the wrong problem, or "didn't understand the question", even though they were invited to contact us with any questions. Any of these was a no-hire vote from me.

We had one student who was very nervous and kept apologizing that this was his very first job interview, ever. That one we took a little more time to coach and encourage, inviting him to come see us again once he'd finished some sophomore-level coursework. How his resume got that far is yet another testament to the absurdity of requiring HR review of technical resumes.

This was for an opening that was advertised as having at least 5 years experience. So while not exactly a senior level position, it was not entry-level either.

Those who finished the fizzbuzz test in under 20 minutes (I kid you not) and had completed passable solutions to the other 3 questions moved on to more interesting questions. Those who didn't, the interview basically ended right there. We continued to talk with them as long as they wanted, but often some of the people on our side of the table excused themselves for other things.

The programming tests did their job; they weeded out the resume padders, the people who couldn't program or use Google, and those who couldn't follow instructions or ask for help when stuck. Did we miss out on some qualified people? Maybe; we might have missed some sharp people, but we were in a business, not a deep-thinkers' club. We didn't need more smart people who can't do simple things.

Re:No undergraduate level stuff for me

[0xdeadbeef]

And I understand the pompous arrogance that motivates his feigned ignorance at the purpose of fizzbuzz tests. For every apocryphal story of "zomg they asked me to write quicksort from memory!" there are hundreds of actual examples of people with impressive credentials who can't even handle modulo arithmetic or linked lists or whatever trivial concept most competent programmers know reflexively. I know, early in my career I graded the damn things, and flunked several people with Masters degrees and many people with years of experience. Never failed a PhD's test, but I did witness one guy who stormed out of the interview when asked to take one. Buh-bye, cupcake!

No one is above this. Anyone who believes they are too special to take it is either so completely ignorant of the industry and the caliber of applicants that they won't perform well in a business setting, or, more likely, they are aware of their own shortcomings and are putting on a show to maintain face and self-esteem. Competent people use them as an opportunity to relax and maybe show off a little.

Seriously, refusing a fizzbuzz test for a programming job is like applying at the DEA and refusing the mandatory drug test. I don't care how you feel about it, you're a fucking a moron to think you could ever get around it.

just like speed writing

[jehan60188]

they are as valuable as speed writing tests
Took the GRE a year ago, and got a 0 on the writing portion since I decided to write an essay about how useless speed writing is.
Real writing, is a very useful skill- that's when you have access to a piece of writing software that can do simple things like spell check!
Similarly, knowing how to spit out code is not nearly as useful of a skill as knowing how to break a problem down, and decide what tools would be best for it.

Re:just like speed writing

[SurfMan]

Funny how you misspelled "Gammar Nazi"

Useful for weeding out non-programmers

[Ckwop]

We use Fizzbuzz and a short SQL test that take a total of 30 minutes for the first part of the test. If they fail this, we can them and don't give them an interview.

A surprising number of people fail this test.

We then have a larger problem with much more time allotted. Here we're looking for style and quality of construction.

That said, even with this longer test, the people we hire tend to get the same distance through the test. They're at least within the same half of an order of magnitude.

At the end of the day, in a paid position you can and do have a deadline to work to. You can't take forever building something. You have to produce the goods!

Re:Useful for weeding out non-programmers

[davidbrit2]

Yeah, give them a simple task. Something that any reasonable programmer should be able to do in 15 minutes. Then give them a solid hour to work on it. If they can't produce something working in that time, that's a pretty informative result. The time limit isn't a speed challenge; it's meant to be very generous, and act as, "Look, we need to move on..."

Personally, I like to give a few different options from which they can choose freely. Something procedural, something OO, and maybe something in SQL or a functional language. Perhaps a couple different choices for each - around 6 to 8 total. That way you don't run the risk of excluding a worthy candidate because you happened to design some problem they aren't really specialized in, and if they can't handle any of them, that's a nice big red flag.

Re:Useful for weeding out non-programmers

[Qzukk]

Mmm, yes, SQL peeves. Trying to do graphs/trees in SQL is one thing, but nesting aggregates is another aggravation (I admit that the proliferation of window functions through the ecosystem has made this easier to write in SQL than it used to be).

Consider if you have something like

product (id, name, price)
invoice (id, customer, date)
customer (id, name)
line_item (id, invoice, product, discount_to_price)

I'd suggest asking the person to write queries that will tell you (in no particular order of difficulty):
1. What is the average invoice total and average number of items sold?
2. How much is the total discount on the invoice with the highest total?
3. Who are your top 10% of customers by invoice total?
4. Who are your top 10% of customers by discounts received?

I'd personally be tempted to give bonus points to people whose first answer is

alter table invoice add column total ...

Sometimes you need to think on your feet

[fatphil]

So yes, they are valid.

I heard of one respected hi-tech company giving a 3 hour C programming task. I scribbled my solution on the back of a beermat in about 3 minutes. Were I to have been able to do it in a functional language, it would have been a 1-liner.

I can only imagine that a 45 minute test would be "write a program which says 'hello' to the world".

Aren't all tests timed to some degree?

[khakipuce]

A lot of companies use coding tests as part of the interview process and pretty much there will be some time limit, whether stated or not. They are not going to let you sit there for two days to answer 20 questions or complete a 10 line routine.

As to the value of rigid timing, then that is a bit dubious.Do you want fast and sloppy or slow and accurate? Does this tell you something about the organisation and whether or not you want to work there? I feel it really depends on how they treat the results WRT the timing.

Yes

[Splab]

If the people performing the test are any good.

First of all it will weed out the anti authority programmers. (There is a lot of people who will refuse to do this - the door is right there...)

Next, it will test if you can handle stress - quality of the code (should be) is irrelevant in these kind of tests. But you learn a lot about how people act under stress.

Personal experience, during an interview I was asked to implement a hash map on a whiteboard. What they where looking for was not an actual shiny working example (hands up, those who don't need to go look in a book to find a proper hashing function (or the interwebs)) - they wanted to see how I handled myself in a stressful situation.

Re:Yes

[SirGarlon]

If the people performing the test are any good.

This is a key point. I "failed" one of these interviews because my implementation updated the left-hand side of a loop condition, and the kid who was interviewing me didn't have the listening skills or mastery of basic algebra to understand that can be equivalent to updating the right-hand side.

The take-away for me was, thank God I'm not working there with him as a colleague or, worse, a supervisor.

This was at Google, where everyone thinks they're hot shit.

Re:Yes

[dcollins]

Similarly: I was once asked to convert a text number string to an integer value, and just jotted down the well-known textbook case example. The problem is, the efficient algorithm just goes left-to-right multiplying by another 10 each step, and the interviewer completely refused to believe that this worked (and more efficiently than his suggestion), even after I stepped him through an example. Still to this day I'm dumbfounded by that.

Re:The answer is...

[0xdeadbeef]

Paula Bean, is that you?

45 minutes is plenty of time!

[Anonymous Coward]

To find completed code for that assignment online.

Sure it does

[beinsvein]

If a job requires a skill that is easy to test, it should be obvious that you want to test it. Programming is such a skill. Sure there are tasks within programming that can't be tested in 45 minutes, but there are also tasks that can. I'd feel I knew more about a programmer's skills having seen a couple dozen lines of code she's written than for instance hearing her last employer's opinion, which may be biased by all sorts of interests, or reading the list of projects she'd worked on, where you don't know how she contributed. College grades in programming courses might provide the same kind of information, but courses may not be standardized and the candidate might have developed her skills since college.

Heck yes

[dkleinsc]

The basic rule of programming interviews is that you should demand that they actually program. It doesn't necessarily have to be a difficult problem: I've handed somebody a standard Fizzbuzz in an interview, and the competent candidates will solve it in 2-5 minutes, while the incompetent candidates won't solve it in 15 minutes.

The reason this is necessary is that on paper, the incompetent candidates can look identical to the competent candidates.

Re:Heck yes

[deniable]

Hah, the incompetent ones usually look better. They've had more practice.

We found that broken code was a better test

[mykepredko]

When I was at RIM, we used a broken quick sort method that the candidate was asked to fix. We didn't time how long it took the candidate to implement the fix, but it generally required the candidate to do some research as to what was wrong. One of the team leads created a simple app that tested the performance (ie speed) of the fix.

What was really interesting to me was the number of candidates who refused to do the test (50-60%) because they said it was "beneath them". The big problem was, RIM's HR (OD) that insisted we interview the candidatest that refused to do the test because we were losing potentially half the candidates that were responding to the job applications (this was when RIM was The Place To Be).

The best candidates were the ones that did the test and asked if we had any more. These candidates also tended to produce code that ran sort the fastest.

myke

Re:We found that broken code was a better test

[jittles]

Shoot I've been in the industry for a while and I'll be your beyotch and run to taco bell and pick your lunch up as long as I am getting paid my normal wage for it. There are certainly tasks that are more cost efficient to have a more junior employee to do, but I won't tell you no. Now if you asked me to do something illegal or unethical, I can see myself to the door!

They're very good...

[clickety6]

But they should shorten the limit to 10 minutes or less.

.

It shouldn't take any competent programmer more than a few minutes to google some fitting example code, rename a few variables and update the comments :)

Re:not really

[fuzzyfuzzyfungus]

coding quickly does NOT indicate a better programmer - for an undergraduate level coding assignment, it only indicates that you've been programming for a while and that it does not take you long to complete simple logic tasks.

It certainly isn't the only skill or(if the applicant pool is any good, even a distinguishing skill); but demonstrating familiarity and facility with your tools is a good thing...

Quick is not best.

[GhigoRenzulli]

Ask my wife.

It Doesn't Matter If Tests Help Employers

[assertation]

I've been programming for about 14 years. I have seen this conversation come up plenty of times. Many people come up with good reasons why such programming tests and other programming job interview techniques unfairly weed people out. I've been a victim of it myself at some interviews and have gone on to do STELLAR work at OTHER places. The bottom line is that employers feel that they need SOMETHING, even something crappy, so... these interview tests are a fact of life.

However misguided, if you are a programmer and you want a new job, you have to deal with this reality.

Fortunately, in 2013 there are a plethora of programming interview prep books out there so at least you have a chance if somebody throws a pop quiz at you about the tower of hanoi, asks you to write a script to print out only prime numbers or asks you to write a script to reverse a string using no speciality functions though you have not thought about such mishigas in years.

Re:It Doesn't Matter If Tests Help Employers

[tibit]

I don't know what tests you're alluding to, but man, if you can't do fizzbuzz-style exercise without breaking a sweat, you're useless, okay? I don't even think there's any argument to that. When you're designing large systems, you must have demonstrable working knowledge of the basics. Otherwise your designs will be bloated monsters where anyone who knows their basics will look at and say: it took you HOW LONG and HOW MUCH MAN-YEARS?!

Fact of life: if you don't have a decent grasp of basics of computer science, including algorithms and algorithmic complexity, you'll be useless at designing anything big. You'll make it too big, it'll unnecessarily underperform, and people who know their shit will laugh all the way to the bank as they overtake your employer's bloated product. It's like claiming to be a great architect without having working knowledge of the basics of structural engineering. Just as you need a feel for strength of structures and their scaling behavior in architecture, you need to know your algorithms and data structures in designing large systems. Software design and architecture are both engineering disciplines -- there's always a bit of art to them, of course.

Deadlines

[wisnoskij]

Are you telling us that you have never heard of job deadlines before?

Rules of life

[ledow]

1) No recruitment process mirrors what is required in the job. Don't expect it to.

2) Most recruitment processes generate only a single metric in a particular way.

3) That metric may not be what you think ("What did he do in the last ten minutes of the impossible task we set? Panic? Make stuff up and waffle? Or state that the problem required further time and an interesting avenue for development seemed to be X?" - you'll probably find that your code ends up in a bin within minutes of the test after a brief "Yeah, looks reasonable" check)

4) That metric, if it IS the sole basis of the recruitment process, will result in candidates being hired who are good at that metric, not the job (e.g. managers who are good at bullshitting other managers rather than managing, coders good at churning out stuff that looks right but is horrendous to use or wrong, etc.). If that's the case, you won't get the job, and wouldn't want to work there if you did (those people would be your co-workers, and your managers would think they'd done a good job of hiring them in the first place, and wonder why you have a problem with them).

5) The bulk of the recruitment process otherwise is about weeding out the chaff so they have more time to talk to YOU and find out whether you're actually suitable. Large companies get idiots who can't turn a computer on apply for datacentre systems administrator posts, and the test is there to save them time. Cut out the chaff, get a handful of candidates worthy of interview from THOUSANDS who applied (CV's written in crayon or with spelling mistakes = first stage bin, people who don't have relevant experience / qualifications = second stage bin, people who don't turn up to interview, or turn up late or scruffy = third stage bin, people who can't pass the test = fourth stage bin, the rest are interviewed properly - it's quite easy to get to the interview stage, or to manage the applications up to the interview stage, knowing NOTHING about the job at all).

6) I would hire the guy who approaches me on the day of the test and quietly says "I don't think that's enough time, but I'll try my best, okay?" if he has one ounce of relevant experience / qualification / skill.

Yes - tests more than just coding.

[jnelson4765]

I've done one recently - it also tests memory and grace under pressure. Some people just can't perform well under the gun, and in a high-pressure workplace where you may be dealing with outages that are hitting the tech press within minutes, and the global press within an hour, being ale to not fold under pressure is a critical job skill.

Plus, as my old business computing teacher in high school said, "You will be doing tasks that make no sense on obsolete technology for inscrutable reasons. If you have a problem with any of this, you should probably drop out of this class, since you do not have what it takes to be a programmer in the business world." Dealing with arbitrary requirements is part of working for any large company, and seeing if an applicant will go through with it, or if their ego is going to get in the way, is a useful test.

They do weed out those who just talk a good game

[Rogerborg]

But the curious thing is that I've never, ever, seen them given to contractors, only permies.

Given that contracts are (IME) composed of the best and the worst, but bill you the same either way, this would be the group to which I'd mostly like to say "You want money? Show me."

a sensitive subject with me

[TheGratefulNet]

ob disc: I'm an older guy (50's) and have been writing C code for almost all my jobs in my life since my early 20's. I write code at home, I develop firmware (and hardware, also at home) and I'm extremely technical.

but I fail a lot on 'programming timed tests' and I blame my older slower mind, mostly. the last time I HAD to get inside a tree data structure and totally rewrite it: never in my life. the last time I had to implement a sort routine from scratch: never in my life. its always been a matter of consulting the standard ref models, adapting them and using them. its simply not real-world to test memory recall.

younger guys (I was one once, lol) have an advantage. its fresh in their minds, and they don't have 35+ years of 'noise' in their brains for those sort/search/traverse routines to compete with. I remember when it was trivially easy to derive those algs on the spot or recall them from memory. I passed almost every coding test back in the 80's, 90's and even 00's. but now, I'm finding I have to sit back, think, maybe search a bit and then I can get the answer.

interviewers who are in their 20's and 30's have no patience for people like me. I bet they even laugh behind my back. and yes, speed DOES definitely change as you get older.

but put me in a real life situation where I DO have net access, emacs, gcc and gdb and I'll get your answer pretty quickly and demo it to you, too, if you want.

in summary, real life is an open-book work style. testing candidates closed-book style really reflects badly on YOU, the interviewer, for not getting this detail about how young vs experienced people think and solve problems. for the new guys, you can only ask them the datastruct101 level questions. but its wrong and even insulting to ask older guys the same questions and in the same way, expecting the same speed of reply.

when I interview with older guys, they 'get it' and they aren't such hard-asses. they know what matters, there is respect and it shows. I like interviewing at places like that. they have human beings there.

but 'compare dick sizes' interviews, where the company guys are there mostly to show off and try to knock you down (I'm looking at you, google...) are a waste of everyone's time and fairly insulting, at that.

I would almost go so far as to say that its improper to have a 20something interview a 40 or 50something. they have no idea what to ask, how to gauge the reply or value it and it often comes off as a strange young/old challenge.

anyway, I will not pass any coding speed tests. but I can always solve the problem, I can learn 'on the fly' as I research the problem and I know enough to avoid the bad algs and target the efficient ones. I just don't -care- about deriving the fundamental building blocks anymore. I USE the blocks, I'm kind of tired of looking inside them. (I build electronics quite a bit but I'd never be able to answer exactly how an NPN is built, nor do I need to care to be able to effectively use it).

companies that apply speed tests have missed the point. it shows poorly on them, in fact, that they think this actually makes sense.

Lets put it this way

[TheSkepticalOptimist]

Do you want to work for a company that values hiring its employees on whether they can write code under pressure? No properly managed tech company should ever have a culture of stressful, high pressure, time based code development.

I know a lot of people feel that in an interview they are the ones under scrutiny, but realize that there is a shortage of quality developers (but an abundance of wannabees) out there and you really should also be scrutinizing the company you are applying for. Making the mistake of only trying to be an attractive candidate without also attempting to "interview" the interviewer is the biggest rookie mistake any person makes when looking for employment.

I have been to several interviews where I basically realized within 5 minutes that the company had their heads up their asses when it comes to finding talent. If a company is unable to adequately interview for talent then you can only imagine how poor the state of the quality of their management and leadership is in the company. I know a lot of people blame HR, but poor HR only results when managers and directors are unable to adequately convey the talent and skill requirements they are looking for, so HR staff have to make up the job description and interview process.

Also, with over 12 years experience, I get really frustrated when being interviewed for a senior level position only to have to answer a bunch of junior level questions about coding. I know how to code, and it should be assumed I know how to code from my work history, how about asking me high level architecture questions, how to apply software patterns, or how to create efficient software process. I've moved away from being a code monkey and have been responsible for designing, architecting and overseeing development of entire applications, don't ask me what passing a value by reference means or how to write a recursive method, that just means you don't deserve my experience..