Slashdot Mirror
Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process (theoutline.com)
A number of programmers have taken it to Twitter to bring it to everyone's, but particularly recruiter's, attention about the grueling interview process in their field that relies heavily on technical questions. David Heinemeier Hansson, a well-known programmer and the creator of the popular Ruby on Rails coding framework, started it when he tweeted, "Hello, my name is David. I would fail to write bubble sort on a whiteboard. I look code up on the internet all the time. I don't do riddles." Another coder added, "Hello, my name is Tim. I'm a lead at Google with over 30 years coding experience and I need to look up how to get length of a python string." Another coder chimed in, "Hello my name is Mike, I'm a GDE and lead at NY Times, I don't know what np complete means. Should I?" A feature story on The Outline adds: This interview style, widely used by major tech companies including Google and Amazon, typically pits candidates against a whiteboard without access to reference material -- a scenario working programmers say is demoralizing and an unrealistic test of actual ability. People spend weeks preparing for this process, afraid that the interviewer will quiz them on the one obscure algorithm they haven't studied. "A cottage industry has emerged that reminds us uncomfortably of SAT prep," Karla Monterroso, VP of programs for Code2040, an organization for black and Latino techies, wrote in a critique of the whiteboard interview. [...] This means companies tend to favor recent computer science grads from top-tier schools who have had time to cram; in other words, it doesn't help diversify the field with women, older people, and people of color.
Would be a trial (as in free trial). Throw a fairly standard problem at them, but not one with a simple, common place implementation. Drop them at a computer with internet access, give them a couple hours, and see what they have at the end of it. It's not perfect, but it's probably a better way to evaluate skills.
I have interviewed many people and I don't believe in trick questions. Programmers are not supposed to memorize every algorithm. They should understand how to attack and solve a problem. I was always more interested in their thought process rather than if they get the right answer. How they look at the problem is more important
in other words, it doesn't help diversify the field with women, older people, and people of color.
While there are some good reasons to dislike "code on a whiteboard" interviews, this is not one of them.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
Hi, my name is Vince. I interviewed for Amazon, specifically for their PHP API for AWS development team. Despite an entire background of 10+ years of developing front-facing PHP APIs for other businesses, plus having a major part of my code available for public review on GitHub, I failed their interview process because they wanted me to write a specific type of searching and sorting algorithm, by hand, on white-board. This type of code would never have been used on the job, ever. Yet this is what they interview on. The job was to build a PHP API so PHP developers can call basic PHP functions, and the library would translate them over to HHTPS calls to AWS. All of the complex computing/searching/sorting is handled by the existing AWS services.
"Hello my name is Mike, I'm a GDE and lead at NY Times, I don't know what np complete means. Should I?"
Hey Mike, I once worked for the NY Times Shared Services Center. And, generally, no.
It must have been something you assimilated. . . .
Any "confession" along the lines that "hey, when I need to get the syntax of some specific function or language feature that I'm supposed to be proficient in, I have to google for it, look up the Open Group POSIX help page, check StackOverflow etc" is not really a confession. That's the way most of us work these days.
There's always that one guy on the interview team that would rather stroke his own ego by asking a "trick question". 100% of the time it will be either on an obscure function or feature of a language that may be used once in a career, or it will be so poorly worded as to be unrecognizable. I really don't believe that any other profession runs into this problem. With 40 years experience, an extensive resume, 100's of successful projects, I'm still treated like I graduated yesterday and am "tested" on what I know. It's insulting and companies need to stop. I'm at the point in my career that when the "trick question" person hits the white board, I ignore them and redirect the conversation back to the people asking real questions. If you are an interviewer you'll do yourself and the potential hire a much greater service by either presenting them with a challenge you've recently overcome or asking them to narrate one they've recently overcome.
IOW - trick question guy, knock the shit off, you're annoying the rest of us. It is much more important to determine the prospects problem solving methods and skills than their fluency in the programming flavor of the month.
"Show me an example of a program that you wrote and are proud of"
(and then go over the program with them to make sure they understand how it works and why they wrote it the way they did)
The proof is in the pudding.
I don't care if it's 90,000 hectares. That lake was not my doing.
I did some interviews for a company I worked for a few years back, and my goal for those things wasn't so much to see if they could complete the problem successfully. My goal was to see if I thought the guy I was interviewing would work well with the team. I kept lowering the bar on the programming problem until it was a string reverse, which is just ridiculously simple. Even more so, I allowed the person to do it in the language they felt strongest in. For a couple of the OO scripting languages, that could be as easy as string.reverse(), and I would allowed it if anyone had ever said that. Even so, I was deliberately ambiguous about some things -- did I want the string reversed in place? Were there any error conditions I wanted returned, and should those be exceptions or return values? I had answers prepared, if any of them had ever asked me. I also had a very nice whiteboard, which they would almost to a person go up to and start crapping code onto, immediately. If they'd interacted with me and the whiteboard a bit before doing that, I would have actually stopped them before they'd written a single line of code.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
I'm an employer. I've interviewed nearly everybody we employ at my company. And treating a hiring interview like a rote memory exam is a terrible way to qualify a potential developer hire!
What do programmers actually do? Try testing that!
We do "whiteboard style" for part of our interviews, but only to cover basic comprehension of algorithms. More than anything, we look for basic familiarity with logic structure, and the demonstrated ability to solve problems. Our coding section of our interview process is in the subject's language of choice, including pseudo code, and is "open book" - we want to see what happens when the dev runs into a problem they don't already know! (Critical test: can they come up with a working, supportable algorithm for a problem they don't yet already know an answer for?)
After 20 years of programming experience, I STILL routinely look up the order of arguments for function calls via Google. Who cares to remember when Google has the answer in 0.10 seconds?
Test what the devs will actually DO in an anticipated normal work day and make your decisions based on that.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Sins, not fetishes.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
Quick cheat sheets are important. Not everyone can memorize every single library function in every single language they use on a daily basis, even simple functions.
Is it:
strlen(string)
len(string)
length(string)
count(string)
string.len
string.len()
string.length
string.length()
or any number of other variations.
As a developer, I'm sure most everyone knows the task they're trying to do (get the length of a string), but there are so many variations of the same function between libraries and languages, that it is often quicker and easier just to search which one is appropriate for the given language, than to simply type each one out and test the code to see which one doesn't bail on execution.
... I was interviewing a lady for a clerk job at Mobil Oil, where she'd be doing data entry.
I was looking for:
1.) Dedication to accuracy and detail
2.) Willingness to work overtime
3.) Ability to get along with others
She was a single mom who was hungry to work; liked people; her children were almost grown, so she had the time.
SHE FLUNKED THE GODDAM TYPEWRITER TEST!
Typewriter? I told HR I didn't have a goddam typewriter -- test her keyboard skills.
Nope.
That was in the mid-Dilbert years at Corporation.
It little behooves the best of us to comment on the rest of us.
and I don't know what "truth" means. I look up things like "democracy" and "separation of powers". I don't do riddles.
Escher was the first MC and Giger invented the HR department.
These technical interview approaches aren't very good, in my opinion, because they basically assume that the beginning and end of all software development training happened in a second year algorithms class. Algorithms are very cool, I understand why people want to talk about them, but they represent a minority programming challenge in today's world.
Speaking only for myself, in a given month of coding I may only have to consider which algorithms I should use once or twice. The rest of my time is spent on GUI design, communicating with coworkers, working on documentation, and switching between projects. Putting aside the value of algorithms in an interview, how can the interviewer ascertain all of my other software development skills if we spend 2 hours mapping trees on a white board? I would argue that they can't, and by asking technical questions about algorithms or brainteasers, they really aren't properly evaluating the skills of a professional software developer.
This reminds me of one the anecdotes I picked up from one of my college engineering professors in regards to his philosophies on tests:
"Would you fly in an airplane forcibly designed in one hour with no notes?"
And before someone starts busting my balls, yes students should learn the underlying fundamental mechanics of the subject matter. It's more of a protest of the unnecessary aspect of memorizing the details that have no bearing on someone's understanding.
As a CS professor, I find the use of these kind of review processes kind of contradictory.
In education, we often based pass/fail on two hours long exams where the student has no access to outside information. Then companies say that's not how the real world works. CS evaluation sucks!
We change to continuous evaluation or project based learning, where student has longer time (well, certainly not 2 hours in a closed room) and can access external resources. You know, like the real world. And now it's companies the ones evaluating candidates the old fashioned way!
Let's go back further in time and go for corporal punishments...
If you can't write bubble sort or can't figure out how to get a length of a string in Python, you shouldn't be hired.
Tell us what company you work for so we'll all know where not to apply.
How does a programming interview discriminate against "people of color"?
Red-black trees I suppose.
. But if you can't code a sorting algorithm without a reference, ... then you don't have the skills I need.
Except for a very small subset of the industry, the only skill you need to code a sorting algorithm is knowing how to type in the search box on google (or stackoverflow). Or know how to call whatever the sort() function is in whatever language you're using.
It's simply not knowledge that's relevant anymore.
Programming is not about rote procedure. It is about finding a way to accomplish a goal.
Clean and efficient code is a bonus, but not mandatory. Complying with any else's definition of Good Practices may be a consideration, but only to the ones making the definition. Working well with a carefully assembled maximally-diverse team may be helpful, or may be something to overcome.
In any event, rule # 1 is paramount. Get the job Done. Everything else is value-add, at best.
*Statistics independently verified by Slashdot
What I've always found funny about this interview process is that the assumption is always that the interviewer knows the correct answer(s) to the question. It's painful when they don't.
Years ago I interviewed at Google and was asked a question about bit counting (some variation on "given a bit vector, wat's the fastest way to count the number of 1s?"). I quickly answered, "well, if your processor has a population count instruction, stream your vector through that and accumulate the result in a register". Having just evaluated bit counting methods as part of my Ph.D. dissertation, I knew this was the fastest way to do it, assuming the instruction was available (it's not on x86, but is on Power/VMX and most DSPs support it as well).
After I got a blank stare back from the interviewee, I said, "Oh, you were looking for the lookup table answer". We could have left it at that, but he went on to explain using some very convoluted logic how the lookup table would actually be faster than using a dedicated instruction and that my answer was clearly wrong. I mentioned a little bit about the number of cycles required for each approach, but he had none of it. In his mind, I had the wrong answer, even though my second answer was what he was looking for.
It was at that moment that I realized Google was not going to be a good place to work.
-Chris
I don't get the impression most interviewers know what they are looking for. Just like with software, testing in general is hard. You have to understand what you're really testing for and whether your tests actually tests for those qualities and rules out anti-qualities.
I find it hard to take anyone who thinks these coding tests really test for anything.
Those who do not learn from commit history are doomed to regress it.
Technical expertise is only one part of the whole picture. But it may be the part people thrust into the interviewer chair are most familiar with. Sometimes the interviews feel like a final exam and I wonder if they interviewer had final exams pretty recently.
I have been known to point out these annoying little things to my colleagues when we are hunting to fill positions:
Do the candidates' personalities mesh well with yours? Do you think you can stand being around them and working with them day after day?
Will they be reliable?
Do they seem easy to train? They will need to learn how this group does business and works together after all.
Do they express curiosity when they don't know the real answer?
Do they make things up to fake an answer? Or do they say "I don't know the answer, but based on my experience I would guess this..."?
Do they communicate well? Do they listen well?
From the IT side of the house, I can say I've been there. In the development world, it's the whiteboard test, but in the IT world it's a tool-matching exercise and trivia contest. I freely admit that I have a very bad memory and constantly look up information unless it's actively used in my daily work. I feel the field of IT is too broad for one person to remember even the basics of _everything_. I've failed interviews because I couldn't recall some trivia questions, and I've done well on others when I was given the chance to demonstrate skills I'm comfortable with. Worse yet, I've gotten shut out of interviews because I haven't used the exact brand and version of some tool they have in the toolbox, regardless of how easy it is to pick up on the job.
In an IT context, matching a laundry list of tools, platforms and software isn't going to get you the right candidate. I hate to self-promote, but I've been told by many employers that the reason they like me is my willingness to dig in and learn new stuff, then document what I know and teach people before moving on to something else. One example I like to cite is that this is essentially the 4th time I'm relearning Citrix XenApp at a level deep enough to do a new deployment -- when this is done I'll get assigned some other project working with a completely different tool set. Some people in our field are experts in Foo but not Bar, or worse yet, specific Foo 3.6.1 experts. There are products that I work with daily (Citrix XenApp, SCCM, etc.) that are easily full time positions, and are so complex that people get to be tunnel-vision geniuses on them. Same goes for platforms -- there are Cisco configuration and management experts who go so far down in the weeds that they can't see things like SDN and hybrid network gear slowly taking over. They'll continue to get paid a lot for quite a while, but eventually the contracts and FTE positions will dry up as the product is phased out. Look at how different a typical SAN engineer's job is these days compared to the times when you had to be an EMC or NetApp savant to get things functioning.
If you're looking for someone fresh out of school, the whiteboard interview is only going to get you a sense of whether or not the candidate absorbed basic concepts from their computer science degree. In IT, most of us don't come from CS and end up here because we're crazy people who like complexity and troubleshooting/firefighting. Smart employers recognize this, but often you get programmer-style interviews when you are trying out for a spot at a software or IT services company.
Experts know that it is almost always better to use the sort from the language's library, and if they can't then they would look up the code for Quick sort and use that.
When I was interviewing for my current job I got asked one of these types of questions. I was honest with the interviewer, I told them I had a vague understanding of the concept, but I'd have to look up or ask someone how to actually implement it since it's not something that I'd ever done before. I figured I had nothing to lose by being honest since I wasn't going to be able to BS my way to a solution anyway. The interviewer appreciated my honesty and said that collaboration and being able to find solutions to problems you don't have the immediate knowledge to do were important and I eventually ended up getting the job. I'm not sure if that's what they were actually looking for or if they liked my other skills so much that they were willing to overlook it. To this day I've never come close to having to program what they were asking me to do in that interview.
It's more like asking an expert pianist to tune a piano. Yes, some can, and that's great. But others can't, and it's usually not relevant--because you usually hire a piano tuner to tune the piano. Just like how you really should use an existing function for sorting.
No FOSS projects? Ask them to re-apply later.
Not everyone is going to spend months doing work for free just so they can apply to your company. Some of them have better things to do, like actual paid work.
By using a "name-brand algorithm" which I learned in 2nd year or whatever, umpteen years ago, you are testing for cramming ability, recency of test-cramming at school, and mental copy-paste of whatever was crammed.
You are definitely not testing for problem comprehension and creativity of solution or methodical approach to solution. All of those skills will be severely impaired by time-pressure panic, if you have a good programmer in front of you who has not crammed that particular cookie-cutter name-brand algorithm lately.
It would be one step better if you just said: write an algorithm to sort this array/list of values. But you still chose a problem for which comprehension, creativity, and methodical solution skills are not needed, if the person happens to pattern match your posed problem with the bubble-sort they just memorized for interview-readiness purposes.
So if anything, choose a simple-ish problem that is NOT one that has readily available mental copy-paste cheats available.
Where are we going and why are we in a handbasket?
Hello, my name is Anonymous Coward. I am a software developer.
I was talking to you about a database kernel position which is a field I have spent much of my career in and in one case was a key member of the design/coding team developing the initial (and several subsequent) releases of a revolutionary enterprise class scalable DBMS which is still being sold over thirty years later and in production at many of the biggest companies. You, of course, must have known that because you contacted me and, unusually, I decided to stop by and chat with you because I have a soft spot for startup companies, database, and engineering managers who look around seeking talent themselves rather than relying solely on recruiters. You asserted that your startup was developing an enterprise class scalable DBMS system so it seemed like it might be interesting.
You then asked me to write, on the whiteboard, a 'debit/credit' function that took three arguments and returned, IIRC, the amount transferred (admittedly that was a little odd). The first two arguments were account objects (passed by reference if I recall) representing the source and destination accounts respectively and the third argument was an amount (presumably in the same currency as both accounts were maintained in) to transfer (a float if I recall -- that, of course, is likely stupid in itself when dealing with currency).
I, understandably, was perplexed -- you must want something more than this simple function (I'm not fresh out of high school looking for a summer job) and I tried to get more requirements out of you and got none except when I asked if this function was to provide its own concurrency control or if a higher level would have insured both accounts were sufficiently locked you said something like "Okay, you need to provide the locking" and agreed that I could assume existence of "Acquire/ReleaseWriteLockAccount(AccountObject)" functions.
Okay -- so I'm thinking this is a concurrency problem, no problem (still lame, but okay...). Obviously I need to deal with deadlock potential so I ask if I there is some unique attribute of each account (you didn't give me a description of the account object's public members) that had a well defined order that I (and other developers) agreed to use to order locks when locking multiple accounts to eliminate or at least reduce deadlocks. You weren't prepared for the question so I asked if I could assume (I think) that the account number was a public field, unique among all accounts, and supported the LessThan operator with traditional semantics. You said, sure.
I wrote the code making sure to lock the accounts in order of account number, transfer the funds, unlock correctly and return the required value.
You were not happy -- what you really wanted me to check, it turns out, was that the source account would not end up with a negative balance and fail to do the transfer and return 0 as the result.
What you didn't seem to realize is that in the real world cash balances on individual accounts go negative sometimes. In fact, at that instant, I had one (of multiple) accounts at one brokerage firm which had had a small negative cash balance due to an inter-account transfer out for over six months and no one cared because I had multiple accounts at that firm with a net value of well over a million dollars and cash balances (well, "cash equivalents"), overall, of a few hundred thousand dollars.
Both of us made a decision in that discussion. You decided you didn't want to hire me. I decided I would never work for you or your company as I didn't want to risk working on/at a team/company which was so poor at evaluating people and would pass on good developers who know how the real world works and would likely hire unqualified people who answered your kindergarten questions the particular way you wanted.
Of course, I was glad that we both made those decisions because your company seems to have never got past a very modest A round and seems to have disappeared off the face of the Earth quietly.
Why is there an "insightful" mod and why isn't it "-1"? If I wanted insight, I wouldn't be reading
The point is, the FOSS project(s) are your portfolio, similar to how a graphic artist, photographer, architect demonstrates their ability with a portfolio.
The difference is, the artist's portfolio is built up with their paid work. Every project they work on gets added to it. A programmer can't simply take company code with them when they go interviewing, so the project has to be a hobby project.
Can't be bothered to do at least one hobby/self-training/benevolent programming project? You don't love programming enough to be that good at it
That logic doesn't follow. Not everyone who's great at programming works on hobby projects, and not every such project will be published on Github. I work with 2 people who don't program after work, and they are just as productive as everyone else. As for myself, I also don't publish my non-work projects, because I can't be bothered to write documentation for them and I've coded them to serve my use case specifically.
Confession: I run technical tests when recruiting IT persons. But it is on a computer, with Internet access, and looking up documentation or forums is fine.
In fact, using Internet or documentation to find something is a very good indicator that the candidate has skills and experience.
Yours [Walt Sellers'] was one of the few insightful-rated comments that really seemed to merit the mod. If the mods were logarithmic then high (e^5) mods would really mean something.
Or maybe you just read Work Rules! from the google guy? Most of your comments follow along the lines of the google interview process as reported in that book. Mostly admirable ideas, but I still think the old "Don't be evil" motto has been replaced by "All your attention are belong to us." (I've been reading a lot of google-related books lately, but nothing else in the various parts of the LARGE discussion that I read seemed to ring any other bells.)
My suggested improvement to the google's interview process would involve making videos of the interviews, either just the interviewer's face or both sides. Then the interviewer could focus on the interview itself rather than making the notes at the same time. If the interviewee agreed to both sides, then the two-sided interview video would actually allow other people to get a much better feel for the interview without taking any of the candidate's time... (Also much better for evaluating the skills of the interviewers.)
However the insight I was looking for and failed to find in the comments was a bit upstream of the actual interviews. It was how the job-search websites earn their income from the corporations by helping them drive salaries down. It's kind of like lotteries focusing their advertising on the winners while ignoring all the losers who made the lottery profitable. Of course they boast about the few winning jobs (as with the google) that attract lots of candidates, but where their money is really coming from is all the lesser candidates who compete for the lousy jobs by seeing who is most willing to accept the position.
I wish there were a job-search website that was driven by the candidates. An economic model where the website actually profited by helping each person get the best job. Perhaps the economic model aligned from the employees' side would produce rather different results?
Talk about a pipe dream. Whatever I've been smoking, I better cut back.
Freedom = (Meaningful - Coerced) Choice != (Speech | Beer^2), and sad sock puppets' bad mods avail them naught.
Way back in the 1980s, one of my physics professors told the following joke:
The trustees of the university want to find out if the professors really knew their stuff so they came up with a question: What's 2+2?
They go to the math department and the professors there said, "Oh, that's easy. It's 4."
Then the go to the physics department and the professors there said, "Oh, it's 4.000000000 with an uncertainty of another decimal place."
Then the go to the engineering department and the professors there said, "Just a minute while I get my handbook."
Finally, they go to the accounting department and the professors there look around to see if anyone can hear them and then the whisper, "What do you want to be?"