Slashdot Mirror
Are Contests the Best Way To Find Programmers?
Nerval's Lobster writes "Tech firms are engaging in several non-traditional hiring methods, from programming contests to finding the right people via algorithm. One of the more popular methods: set up a coding challenge or programming contest to bring out interested parties, with the top prize being a trip to the sponsoring company's headquarters to interview for a job. Look at what Facebook is doing in this area, sponsoring several Kaggle.com programming contests to find the best programmers; it also makes use of the site InterviewStreet to screen potential applicants. In theory, any company can build and run a contest online. But is it really the best way to go about hiring a programmer (or any other tech-minded employee, for that matter)?"
to find programmers who like contests.
It's honest (that you are competing with other, perhaps qualified, candidates); and a bit more objective, in that you don't have HR weeding people by the wrong metrics.
It's also a blantant attempt to get people to work for free.
No experienced competent software engineer would ever enter one of these code contest. If you need good engineers try offering the best compensation and the best working conditions.
Reminded me of this reality show called Hell's Kitchen where the winner gets to become new head chef
I believe that this is a good way to bring in young talent, people in college or kids who dont have a formal education but are self taught and need a shot.
I dont expect this to be a good method for bringing in top level experienced workers however.
have you seen my sig? there are many others like it but none that are the same
Maybe I misunderstand the nature of these contests, but what I produce in 4 weeks is different than what I produce in 4 days. I have to make serious trade-offs that will impact the software design significantly and will not reflect what my vision would be for the "big picture" goals like clarity, maintainability, modularity, safety, error handling and all manner of best practices.
I wouldn't want a prospective employer to judge me based on the stuff I can churn out in a flash, unless that's the nature of the work they have in mind for me.
"Now, I doubt any of you would prefer a rolled up newspaper as a weapon against a dictator or a criminal intruder."
I'm not going to go out on a limb and claim to be one of the best programmers.
However, I don't have time to do programming constests with my day job being rather busy. I generally give a lot to that job so I'm not going to spend time coding when I get home on the weekend.
I'd imagine that I'm not particularly unique in this regard.
On the other hand, you wil find good programmers who have no time commitments other tha coding, so I guess it does work out well.
SJW n. One who posts facts.
Let's see if I've got this right: there is such a shortage of programmers in the U.S. that we have to raise H-1B visa limits in order to supply them, and yet companies have to create hiring contests in order to screen the overwhelming number of applicants?
Proverbs 21:19
What is the 'industry standard' anyway? If we're going to have one how about 0.0001. One error in 10 million lines. I mean, if we've got 0.66 for the linux kernel and its say 7 million lines thats 4620 errors. Now do the same math with x zillion LOC for (add you favourite: Windows, Oracle, ...) and its rather frightening.
Interview question #1: Explain why you hate C++ Interview question #2: Justify why you repeated soundbites about C++ and not formulate your own explanations Interview question #3: Explain why you like Java or C# better Interview question #4: Justify why you didn't know that C++ can also do those things
Those who do not learn from commit history are doomed to regress it.
Its the best way to find alpha douchebags.
Seriously, I'm putting the blame on Hollywood for this one. Contests are just like the SAT - you have to train for it. It's a specific type of programming, and I don't think it necessarily brings the best coders. The snacks are always a great plus though.
It kind of depends upon what you need. Contests will find puzzle solvers, those that would work reasonably well doing research. Contests however, will not be very effective in finding the "big picture" folks that are needed to develop clean, robust architectures. I suspect they may well select against them. You will also almost certainly limit your applicants to college kids, and/or recent graduates looking to establish themselves not those that have already proven their abilities.
Two of my imaginary friends reproduced once
....all those competitions do is find the fastest programmers.
Fast usually means hacky. it definitely does not equal good, but unfortunately many managers dont understand that concept because they all subscribe to the "make it fit in my microsoft project plan" mindset.
Sadly, In this culture, poorly engineered and buggy software and the corresponding very costly rework have just become accepted as unavoidable even though its actually not.
Its actually much cheaper in real (but unfortunately largely hidden) costs to take the time to get it right before you deliver to the customer.
I'll take the programmer who loses these competitions because they took the time to do a robust job thanks.
Contests only gather a weird, smallish subset of programmers who are good enough to win a contest, but who have the spare time[1] and will[2] to enter a contest. [1] This means they have little internal motivation because they're not otherwise working on something that inspires them. Good employees have internal motivation, bad ones need Management to whip them, which is what you'll get here. Instead find those who contribue to open source projects or who spend free time giving stellar information to the programming community. (Blog posts, Stack Exchange, etc.) [2] Again, motivation. What are they really trying to get by competing in the contest? Can you, as an employer, provide that same motivation on a regular basis? Probably not. So once again you'll get a pretty great programmer who underperforms.
If your company can offer the combination of difficult tasks, aggressive schedule and high benefits, go for it.
Otherwise you don't need the IT recons, deltas and rangers. They would die of boredom in your place. Let them go to cool startups, googles and facebooks, for the challenge they're worth.
You see, between all the proprietary crap messing with peoples heads, and all the technology egos, and those daytime prisons they call public schools, it becomes really hard to find a good programmer.
IMHO, the best way to find a great programmer is to find some high-school kid who hasn't been corrupted by the public school system, who can think analytically, and who has a good attitude, and train him.
Programmers and related IT folk are the absolute bottom of the corporate barrel - below custodians, below security guards, below the cafeteria staff. Only programmers / analysts / sysadmins / etc. are expected to take 6 month "contract-to-hire" positions. Only IT professionals can work in a job hierarchy with very few, if any, opportunities to advance to senior management. Mainly only IT professionals are told to take salary cuts, work extra hours, and train their successors due to outsourcing.
And now you want a contest to decide who to hire? Do accountants, operations staff, finance staff, and marketing have contests to see who will be hired? Even in sales you're hired for a position - you need to meet your quota, but there's none of this patently demeaning treatment of IT professionals as mere expendable cogs in the machine.
So what if you win the contest? Are you expected to perform at that amped up level every day of your work career? Are you supposed to quit when some new young buck / buckette does better in the contest next year? Is your education, prior experience, ability to work with others totally irrelevant? And damnit, do you have any sense of dignity in your job?
I've worked in IT for 15 years. During that time I've seen friends from undergraduate days and graduate school days move steadily up the ladder while nearly every person I've worked with in programming are stuck in the same ruck - everyone's a "Senior Engineer" or "Architect." And now we can look forward to job duels? Coding against each other endlessly in a competition to stay gainfully employed.
Don't accept this garbage. Being a productive employee is far more than just the ability to spew some excellent code in a contest. We have to make our field a profession, not a joke.
/* Dang, I can't type that well. */
Pwn20wn is probably the high bar for programming contests. Charlie Miller can walk in there and drop a 0-day for Chrome on the judges and walk out with a check for $100k. Is this not the market at work? Is browser sandbox security now such an integral part of the internet security landscape that it warrants the need to pay, and pay fruitfully for the knowledge of how these exploits work? A young guy, maybe 18, nobody heard of, with no real internet 'cred' from Eastern Europe, who had no real degree or CS education, walked in with an IE9 exploit that defeated DEP once it broke out. It also worked around address randomization.
Unless that guy shows up and earns his rep that way, what's he going to do? Just start a blog and put the exploit code on it?
I would argue that these programming contests are pure market forces, about as pure an application of the free market as one will see on the world stage.
Yes it is important that the candidate can program and can problem solve, but its not often that individuals analyze, design, code, and test an application, you work as part of a big team.
Further, a team of egotistical coding superstars is never going to be an effective team. Dull plodders who have an attention to detail are as important as the superstar programmer. You have to have a mix.
So yes, there may be a place for coding challenges, but a good coder is not necessarily a good analyst, a good tester, or a good integration guy.
And given the above, the only tests I'd want to see are those conducted at the company where you can ask "Why that way? Why not this way? What if I needed these changes now? How would you scale that idea? How could you best document that for the testers? How could you make that easier to integrate with this?". It would be difficult to get any of that out of a coding challenge.
...their:
Experience
Work ethic
Ability to work with others
Software ENGINEERING capabilities (not to be confused with 'programming')
Et cetera...
Loading...
I almost agree with you. I'll take the programmer who would have lost because they took the time to do a robust job, but didn't because he didn't waste his time and focused on something productive instead.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
In Bioshock Infinite, Fink allowed potential to employees to bid on jobs. "Who can do it in 30 minutes?" I have 30 minutes. How about 15, who'll bid 15."
Utopia.
Although some might think it's insulting to a professional to have to bid for a job and pay for the bid with free work, I feel that this innovative thinking tells you what a wonderful place Facebook would be to work.
"MIT betrayed all of its basic principles."
Decent programming skills are a requirement for a software developer, but only one of many skills required. Given how many people lie about their experience and fail *VERY* simple interview programming tests, having a programming test screening procedure wouldn't be a bad thing, but only to drop the worst, not to automatically hire the best.
All code submissions become the property of the company sponsoring the contest.
Brilliance in your programming does not equate to a job offer. After all, we already got the work we needed done out of you.
The effort on the side of the programmer is far to big. This means you will only get desperate people and people that cannot manage their time effectively. Not good. If you want programing samples, pay them.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Indeed. And there have already been contests used to get some algorithm implemented on the cheap, with nobody getting the job afterwards.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Apparently many of the posts on this thread come from people who didn't look at the contest.
This is not a programming contest in the sense that they are asking people to create a program. It's a data mining contest where they are asking people to solve a data mining problem.
Yelp is a business directory that allows people to post reviews. The programming challenge is to create a program/algorithm/method to determine how many "this was useful useful" votes a review will receive. Presumably they want this information to inform their reviewers on how best to write a review - what to avoid, how to phrase, &c.
Winning the contest isn't writing the program per-se, it's making a better prediction algorithm than anyone else.
Also it's a long-term contest (8 weeks) instead of the "overnight hackathon challenge" you might be thinking of. You can make several submissions and get feedback for how your algorithm is doing, and how it stacks up to other teams.
Also also it's a team effort. You can enter solo or as a team.
The job offering is for a data scientist at Yelp, so it makes sense that they are looking for people who can manipulate data.
The job listing doesn't have the typical "must have 4 years experience in XXX" listings that everyone hates. It states:
Everyone complains about the HR "minefield" that sorts candidates by requiring useless or immaterial experience instead of raw coding ability. This is a new type of job search that doesn't have these problems.
This doesn't appear to be what everyone thinks it is.
top prize being a trip to the sponsoring company's headquarters to interview for a job
Last time I checked (a bit over a year ago), the normal cost of that "prize" is to spend a few hours on making an updated resume. Granted, I may be on the lucky side of having the experience needed to open doors, but I suspect that as a rule skilled people find more convenient ways to get doors open.
That stated, if the potential job had a particularly impressive salary, I might change my mind.
You clearly don't have any experience with such competitions. You usually get zero points in these competitions if your program does anything wrong and you get a penalty every time you submit a wrong program - if you are even allowed to resubmit after that. In most cases, the more significant challenge is not typing stuff out on the keyboard quickly. The challenge is in figuring out a solution that is simultaneously efficient enough that it can be accepted and simple enough that you can implement it correctly within the deadline.
The kinds of people who win these competitions are highly intelligent, they are world-class experts on algorithm design and data structures, they are able to write code quickly and with few bugs, they are good at thinking of corner cases and coming up with tests for those and they are able to identify bugs very quickly. It's no wonder companies want to hire these people.
You are right that programming competitions like these do not measure the ability to write code that is readable to other people. There is no guarantee that these people can get along with other people or communicate effectively. The competitions do not involve finding a good design for a large system. These skills are not measured and therefore neither are they penalized - there is no reason to think that competition winners would be worse at these things than average. If you are smart enough to win such competitions, you are likely a quick study who can learn new skills rapidly.
Sorry, but such contests are ridiculous at best.
Who will enter such a contest? Probably young programmers without much experience who have the time to actually do such ridiculous "tests", who view the whole thing as a big game show. I can't use game show candidates, I need people I can TRUST. Not people who'll play the price is right today and jeopardy tomorrow. I need people who also have the ability to stand up in a meeting and declare that they will NOT bend over and take it from marketing or accounting, and I highly doubt someone who submits himself to such humiliating ordeals has the guts and spine to face them.
If that's what you're looking for, a spineless gambler who views your company as some sort of game or toy, go ahead and use such methods to find your candidate. Please tell me if you do so I can short your stock.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
Are contest the best way to find programmers? It depends. If you are looking for programmers who can write use once, throw away code that doesn't have to be maintained, then the answer may be "Yes." On the other hand, if you are wanting programmers who can write code that can be maintained by somebody else five years from now and easily modified, then probably not.
Will your contest winner be able to readily adapt to your entity's coding style/standards? Will your contest winner be able to adapt to your client's needs (and if it is for internal coding, then your internal customer's needs). And finally, will your contest winner, be there for the long haul or will he/she get bored and look for the next puzzle/contest, leaving you high and dry?
A contest isn't enough to hire somebody. If you want the best, you need somebody who can do more than just code. People skills are what set apart wizards from the efficient.
For those who seek perfection there can be no rest on this side of the grave.
[fuckthatshit.jpg]
Seriously if I win your contest and I don't even get a job offer, it's probably not a company worth working for in the first place. Jesus quit being dumbasses and just require a coding screen in the interview process.
Some programmers can work in contests, Some can work alone in a cubical and some can work in a team or fit into all three groups. Holding a contest is a great way to spot the programmers who can think quickly on there feet and jam it into high gear but can lead to missing the other types of programmers. Just because I can't program in a contest doesn't mean I can't write great programs. If you put Linus, creator of Linux, into a contest and he doesn't win does that mean he's not a great programmer?
The best way to find good programmers who can work with your codebase effectively is to hire them from the mailing list of your Free (libre) and Open Source Software.
We interviewed the top contestants for a software developer position supporting internal systems such as accounting, claims processing and inventory.
- one had severe Aspergers.
- one never tested, on principle.
- none of them ever documented anything.
- one refused to code in anything but Haskell.
- no-one had ever heard of accounting, claims processing or inventory.
- one did not speak any English.
There was one old guy who came last in the contest. He was slow but he wrote good code, tested thoroughly, documented, had mastered several languages, had extensive experience of common corporate information systems and seemed quite personable. In the end we went with the Aspergers guy. He accepted minimum wage so big win for the company. Right?
Don't you think a programmer who can churn out working code in a short span can do "properly engineered code" when the need arises? In coding contests, speed is important, it is the "goal" hence there is no need to "engineer" it properly. It just needs to work. A guy who actually "engineers" it for sake of robustness, readability, etc in a coding contest has lost track of the goals of the contest and would be more of an idealist than a practical man. The sort of people who rank high in coding contests are generally very good programmers, very strong in algorithms, pragmatic and fast. I would take them any day.
Yeah you just exactly demonstrated the point I was badly trying to make, that not enough people emphasise or even realise how productive doing a thorough job is, rather than doing a superfcial one.
>> Don't you think a programmer who can churn out working code in a short span can do "properly engineered code" when the need arises?
Possibly but not usually, as it tends to be a polar opposite mindset.
>> If you are smart enough to win such competitions, you are likely a quick study who can learn new skills rapidly.
But I would also guess, way more likely to be an arrogant dick who doesn't play well with others. Most business do not want 'stars', they just want compliant code monkeys that simply follow orders.
FYI, IAAP. I here this all of the time, but where is your basis that it saves money? I've been on many teams where code was shoved out the door and as fast as possible hacks and all. Yes, it required costly re-work down the road, but non-existent features don't make any money.
Quick bad math example: I work for an eCommerce company. Feature A is expected to generate $12 million over the course of a year. We can do a bunch of hacks and get it out the door 1 month earlier, but it will require 3 months of rework by a 10 person team. You're still banking quite a lot of money.
To get people to want to work this hard just to land an interview there needs to be something special about your company. Most companies talk about how special they are, but most are generic. I have found that most companies who use the line 'we want people who want more than money', don't pay well and have insane hours. They end up with people who just want a job and won't quit. There are some people who will work insane hours for below market pay (or even right at market) who are not H1Bs and just do what they are told. Most won't.
Facebook has the 'cool' think going for them. I also think they tend to pay very well. If you don't have anything special you won't get people to do this. Also, this is more for entry/junior level technical staff. People with 10 years experience will just blow you off. We don't want to hear your BS. We have done alot of interviews and know what to ask to evaluate people. It doesn't take all this. I have found that really long interviews tend to just be rounds of people come in 1-2 at a time and all asking basically the same questions over and over again.
If you want to recruit good people
-- have a good place to work. Don't BS about it being good.
-- have quality benefits including good Health Insurance (everyone says they have good health insurance,but you don' get the details until you start such as $1500 deductibles and so on)
-- work/life balance. If I am good, what do I get out of living at the office for you just to work for salary? I can go down the street and work less hours. I have a life.
-- don't let me google you and find that you are shipping jobs to India and laying off non-H1Bs. Why would I waste my time?
-- if I show up and its all 25 year olds, that tells me you are loaded with inexperienced people and you don't want to pay anything. Young people are good. However, if that is all you have, then you are doing it to keep salaries down. It tells me that in 5 years when those 25 year olds turn 30 they are quitting since you won't pay them.
-- don't give me BS about 'wanting people who want more than money'. We all know that is code word for we don't pay and we want you to live here.
-- if I ask for a salary range up front, tell me. Every company has an HR department with a range. Any that won't discuss it up front, don't pay well and are just doing a cattle call.
-- don't expect me to show up for a cattle call where you give expect me to blow a day with 100 other people. These other people are desperate for a job. I am not.
-- Ask me competent and quality technical questions. I can tell when they are stupid.
-- Technical staff need to be present in interviews. If its only managers, you are an idiot and think you know more than you do.
-- Don't exaggerate in the interview about how you are the greatest thing ever. You remind me of a used car sales men. Be honest and up front. Be positive, but don't be a clown salesmen. If I ask you for weaknesses in your organization, don't get insulted and like the question. It means I am interviewing you also.
-- do a phone screen first. I won't take a face to face interview without a phone screen. When I was younger I had to drive for a number of interviews that I clearly wasn't qualified for or didn't want. I had to wait 45 minutes because the manager was 'busy'. Just to be asked 'do you know cobol'? We couldn't do this over the phone or better yet by email? good people have options and we don't have to show up without a phone screen. Only the desperate do.
-- Quoting the Science Fiction writer John Scalzi: Fuck you. Pay me.
As a bonus they will also drive away any competent programmers you might have.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
I find "do you have 10+ years of experience in HTML5?" to be the most useful.
Giving stupid answers to stupid questions.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
It don't know if it's the best, but it definitely tells more about coding abilities than questions like 'what is your biggest weakness?'.
Kryptonite.
Socialism: a lie told by totalitarians and believed by fools.
Its actually much cheaper in real (but unfortunately largely hidden) costs to take the time to get it right before you deliver to the customer. I'll take the programmer who loses these competitions because they took the time to do a robust job thanks.
Absolutely correct - And largely irrelevant to the modern business environment.
In almost twenty years of working as a programmer, I've had the luxury of doing it "right" exactly four times. Outside that, the speed with which I can hack something together has mattered far, far more in my day to day job performance. Yeah, I can build you the Bugatti Veyron of code, given $X budget and Y months; but everyone just wants a slightly used moped, preferably by 2pm yesterday.
Yes, you have it correct, some people can't handle large projects but might do okay on a coding contest - Certainly a problem. But some people don't know when not to turn a request for a moped into a Veyron. That hypothetical god-like software engineer who fails your contest? He failed not in doing "too good" of a job, but in ignoring the actual project requirements, simple as that: "Make it work, ASAP, ACAP".
I've given out a homework problem to candidates for several years now and I really like the insight it gives me. the problem is based on this: http://steve-yegge.blogspot.com/2008/10/universal-design-pattern.html - just design a class that implements the prototype/property pattern. Amazing the stuff people come up with
In a position I held a while ago I had a pair of programmers about as unlike as you can get.
One was fast, quick to offer a solution, never wanted to write docs and always made a hash of them when he did. He got code out fast, but it was often flawed in every way you can imagine and frequently didn't pass anything but the most obvious use case.
The other was quite slow. She asked me a lot of questions and we spent time going over the best strategies for solving a problem before she wrote her code. Her code was always excellent, well thought out, to standards, and documented. It ran well with only the occasional need for me to provide a re-work to ensure it scaled to meet the demands of a 24/7 high volume web site.
After numerous years of working together (both programmers) we slowly would down that account as we lost it to a joint venture from Microsoft, Accenture and Dell. When asked who we should downsize it didn't even take a moment to decide, the guy who churned out the code at lightning speed got the axe.
The reasons?
His code hardly ever worked beyond a the most casual testing.
He had a low aptitude for learning.
I re-wrote a lot of his work, which was expensive for our company as I was client facing and senior and he was mid-level.
He was a bit of a twat.
In the long run the slow, methodical, programmer was far better at her job than Mr 24 hours, earned the company more many and caused us far less grief.
Code comps are a terrible way to decide who to hire.
All those moments will be lost in time, like tears in rain.
If contests are no good and interviews are no good, perhaps we need to come up with something better.
But how can we find it? Hey, how about a contest!
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Candidate: I can quickly write a program that solves the problem. ...
You: Perhaps you can, but it would be probably buggy. We don't want you.
Candidate: Actually I'm pretty good and the code is bug-free and robust.
You: Perhaps you can, but you are probably a dick. We don't want you.
Candidate:
Is that how it works?
Case and point, look at the releases by John Carmack and the ID team vs some of the other game companies (EA) that rush to get their products out to the market by a specific date. Yeah, the ID games have bugs too, but nothing in comparison to some of the other games I've played... And man does their code look polished!
I think you mean "Case in point", and I think you have cause and effect backwards.
Its not that they are fast programmers who happen to somehow also write good code when given a limited time; rather Its exactly because they take the time up front to write robust code that they can get games out fast with minimal bugs, due to the subsequent lack of hidden extra time costs that most managers dont understand so never account for, such as subsequent rework needed.
These contests, as focused as they are, are indeed vulnerable to attack when that focus is recast as tunnel vision. While it is true that most of the effort in a a software project is figuring out the right question to ask, the performance on a well-structured problem _does_ have value. It seems disingenuous for a misdirection towards less measurable skills, it allows 'I could do that just as well' to be plausible, just not to me. As someone who has proven worthy in these contests, and having been faced with naysayers, I have found that when I try to figure out what other aspects are important, I find that others have no real clue what they want, yet they blame me for not delivering it.