Slashdot Mirror

adamba writes: "Why are manhole covers round?" "How many gas stations are there in the United States?" "How would you design a remote control for venetian blinds?" "What company is famous for interview questions like those?" You might not know the answer to the first three questions, but you probably know the last one. The notion of asking "Microsoft interview questions," quick logic puzzles and brainteasers, has become accepted wisdom for many technology companies. In comparison, the questions asked during traditional interviews, such as "Describe your typical day" and "What is your greatest weakness?" seem too simplistic, too easy to handle with a prepared answer, too prone to allowing weak candidates to slip through: they simply don't reveal enough about the person. While the Microsoft questions appear to be a better way to evaluate people, the issue has never really been seriously examined. Microsoft's success would seem to make the argument pointless: Can $250 billion in market capitalization be wrong?" Read on for an interesting look at the details and justifications for this kind of interview. How Would You Move Mount Fuji? Microsoft's Cult of the Puzzle author William Poundstone pages 288 publisher Little Brown & Company rating 9 reviewer Adam Barr ISBN 0316919160 summary The scoop on Microsoft interviews--with answers!

Now comes a new book, How Would You Move Mount Fuji? Microsoft's Cult of the Puzzle - How the World's Smartest Company Selects the Most Creative Thinkers by science writer William Poundstone. Poundstone talked to various people who have been involved in Microsoft hiring, including those who were interviewed, and those who gave interviews (full disclosure: I worked at Microsoft for ten years and was one of the people he talked to). He includes a lengthy list of questions, and most interestingly for many people, he also includes answers.

In the book, Poundstone traces the origins of this type of question, providing some fascinating information on the history of intelligence testing. He then chronicles how a certain type of puzzle interview caught on in the high-tech industry. Microsoft was not the first company to ask such questions, but it certainly popularized it.

Poundstone explains that responding to a problem you can't solve could be thought of as the fundamental problem in Artificial Intelligence (AI), and then continues,

"The problems used in AI research have often been puzzles or games. These are simpler and more clearly defined than the complex problems of the real world. They too involve the elements of logic, insight, and intuition that pertain to real problems. Many of the people at Microsoft follow AI work closely, of course, and this may help to explain what must strike some readers as peculiar--their supreme confidence that silly little puzzles have a bearing on the real world."

It could be--or maybe Microsoft employees assume that since they were hired that way, it's a great way to hire (and complaints from those who were not hired are just sour grapes). Most developers I knew thought of AI as a pretty academic discipline, and were more concerned with putting a dialog box up at the right location on the screen than trying to pass the Turing Test.

Nevertheless, as companies seek to emulate Microsoft, the questions have caught on elsewhere. And as Poundstone put it, such questions have now "metastasized" to other industries, such as finance.

This makes the effectiveness of these questions an important issue. Poundstone first presents evidence that "Where do you see yourself in five years" and "What are you most proud of" are fairly pointless questions. In one experiment he describes, two trained interviewers conducted interviews with a group of volunteers. Their evaluations were compared to those of another group who saw a fifteen second video of the interview: the candidate entering the room, shaking hands, and sitting down. The opinions correlated strongly; in other words, when you are sitting in an interview telling the interviewer what you do on your day off and what the last book you read was, the interviewer has already made up his or her mind, based on who knows what subjective criteria. As Poundstone laments, "This would be funny if it weren't tragic."

Puzzle interviews could hardly be worse than that, but it turns out the evidence that they are better is doubtful. Poundstone shows how intelligence tests are on very dubious scientific standing, and points out that Microsoft's interviews are a form of IQ test, even though Microsoft does not admit that publicly. In his 1972 book of puzzles Games for the Superintelligent, Mensa member James Fixx wrote, "If you don't particularly enjoy the kinds of puzzles and problems we're talking about here, that fact alone says nothing about your intelligence in general". Yet virtually every Microsoft employee accepts the "obvious" rationale, that only people who do well in logic puzzles will do well at Microsoft.

There is another important point about puzzle-based interviews: although you would think that they were naturally more objective than traditional interviews--more black or white, right or wrong, and therefore less subject to interpretation by the interviewer--in fact, interviewers' evaluation of answers can be extremely subjective. Once you have formed your impression of a candidate from the enter/handshake/sit-down routine at the start of the interview, it is easy to rationalize a candidate's performance in an interview, either positively or negatively. They needed a bunch of hints to get the answer? Sure, but they were just small hints and it's a tough problem. They got the correct answer right away? No fair, they must have seen it before.

Given the ease with which the answers to logic puzzles can be spun, it is highly probable that Microsoft interviewers are also making fifteen-second judgements of candidates, without even realizing it.

Three years ago Malcolm Gladwell wrote a New Yorker article about job interviews called The New-Boy Network. Gladwell quotes much of the same research as Poundstone, and relates the story of Nolan Myers, a Harvard senior who is being recruited by Tellme and Microsoft. He has done a one-hour interview with Hadi Partovi of Tellme, and spoken to Gladwell, the author, in a coffee shop for about ninety minutes. His initial interaction with Microsoft was much briefer: he asked Steve Ballmer a question during an on-campus event, which led to an exchange of emails.

As Gladwell writes, "What convinced Ballmer he wanted Myers? A glimpse! He caught a little slice of Nolan Myers in action and--just like that--the C.E.O. of a four-hundred-billion-dollar company was calling a college senior in his dorm room. Ballmer somehow knew he liked Myers, the same way Hadi Partovi knew, and the same way I knew after our little chat at Au Bon Pain."

So Steve Ballmer, who obviously does not feel that he is choosing people based on traditional interviewing techniques, and in fact was one of the originators of the "Microsoft questions," is more prone to making fifteen-second judgements than he would probably admit.

The flaw, if any, may simply be in ascribing too much value to the puzzles themselves. The actual questions may be secondary: the company might do as well asking geek-centric trivia questions, like "What was the name of Lord Byron's niece?" That does not mean Microsoft is hiring the same people that an investment bank is going to hire. The cues they look for may be different: instead of a firm handshake and the right tie, they may be looking for intelligent eyes and fast speech, or whatever non-verbal cues ubergeeks throw off.

A Microsoft interview candidate will typically talk to four or five employees, and in general must get a "hire" recommendation from all of them. Even if the employees are actually basing their recommendations not on puzzle-solving ability but on a subconscious evaluation, it is unlikely that all of them will be subconsciously using the same criteria. Emitting the proper signals to satisfy four different Microsoft employees may be as good a judge of a candidate as any, and Microsoft may be good at interviewing simply because it tends to hire people that are similar in some unknown way to the current group of employees. If another company adopts puzzle interviews, they may discover that they are not hiring the smartest people, just the people most like themselves.

In the end, the best thing that can be said about puzzle interviews is that as a screening technique, they are no worse than traditional interviews. And there are some side effects: some candidates may be more prone to accept a job with Microsoft because of the interview style, and imparted wisdom about the technique may function as a useful pre-screening of prospective applicants. And of course, employees may get a kick out of showing a candidate how smart they are, although this can have a downside: How Would You Move Mount Fuji? has several examples of interviewers who seemed more concerned with proving their intelligence than in gauging that of the candidate. One former Microsoftie admits they asked candidates a question they did not know the answer to, just to see what they would do.

Two chapters of the book, entitled "Embracing Cluelessness" and "How to Outsmart the Puzzle Interview," attempt to help interview candidates who are confronted with such puzzle questions. The official advice is scarce: Microsoft's Interview Tips page advises candidates "Be prepared to think," which isn't much help, since presumably nobody is advising the opposite. Some of the recruiters who go to college campuses have their own little tips; for example, one recruiter named Colleen offers a quote from Yoda: "Do or do not, there is no try." Other recruiter tips include "Stay awake" and "Always leave room for dessert." Luckily, Poundstone gives advice that is a bit more concrete than that.

Microsoft puzzles can be divided into two types: those where the methodology is more important than the answer, and those where only the answer matters.

The "methodology" puzzles break into two classes, "design" puzzles ("How would you design a particular product or service?") and "estimation" puzzles ("How much of a certain object occupies a certain space?"--for example, "How much does the ice in a hockey rink weigh?")

Design questions exist because at Microsoft, responsibility for product development is split between two groups, the developers and the program managers. Developers write code: program managers design the user interface, trying to balance the needs of users with the technical constraints from developers. As Poundstone points out, while estimation questions and general logic puzzles are universal, the design questions are reserved for program managers.

The reason is that program management does not require the specific skills of development. Designing software is something any reasonably intelligent person can attempt, so the design questions are aimed at finding people who are really good at design. In fact one program manager I worked with told me that the best way to distinguish a potential program manager from a potential developer was to ask them to design a house: a developer would jump right in, while a program manager would step back and ask questions about the constraints on the house.

(Developers, meanwhile, are usually asked to write code on the whiteboard, an experience that program management candidates are spared. Books exist that discuss coding problems in more detail, such as Programming Interviews Exposed: Secrets to Landing Your Next Job by John Mongan and Noah Suojanen, which covers many standard programming questions and even includes answers to a few of the logic puzzles that Poundstone addresses).

Poundstone does include some of these design questions and provides sample answers. But the "answer" to these questions is really the process involved: ask questions, state assumptions, propose design. That's all you need to know about them. If you are wondering why Microsoft did not use this logical procedure when confronted with the question "Design a response to the open source movement," but instead seems to have spouted off the first five things that popped into its collective head--that's just more proof that performance in interviews is not necessarily a great indicator of future job performance.

Another recruiter, Stacey, gives the following interview tip: "The best interview tips I can give you are to relax and think for yourself. For a Microsoft interview, be prepared to answer both technical and problem solving questions. Ask clarifying questions and remember to think out loud. We are more interested in the way your are thinking through a problem then we are in your final answer!"

That approach works for the "methodology" questions: design and estimation. What about the other kinds--the more traditional brainteasers? For those questions, forget your methodology. What Microsoft interviewers want is the right answer.

James Fixx, writing three years before Microsoft was founded, offers some advice that may hearten potential Microsoft recruits: "One way to improve one's ability to use one's mind is simply to see how very bright people use theirs." With that in mind, we can follow along with Poundstone as he explains the solutions to the puzzles that the very bright people at Microsoft ask during interviews. He certainly delivers the goods: 100 pages of answers. Unfortunately, it's not clear whether seeing those answers help you tune up your brain to answer problems that do not appear in the book.

In his book, Fixx spends some time trying to explain what, as he so delicately puts it, "the superintelligent do that's different from what ordinary people do." For example, trying to describe how a superintelligent person figures out the next letter in the sequence "O T T F F S S", he advises people to think hard: "Persistence alone will now bring its reward, and eventually a thought occurs to him." Talking about how to arrange four pennies so there are two straight lines with three pennies in each line, he writes "The true puzzler...gropes for some loophole, and, with luck, quickly finds it in the third dimension." Further hints abound: "The intelligent person tries... not to impose unnecessary restrictions on his mind. The bright person has succeeded because he does not assume the problem cannot be solved simply because it cannot be solved in one way or even two ways he has tried." This advice sounds great in theory, but how do you apply it in practice? How do you make your mind think that way? As Poundstone quotes Louis Armstrong, "Man, if you have to ask 'What is it?' you ain't never goin' to know."

Poundstone recognizes that the flashes of insight that Fixx describes, and that Microsoft interviewers expect, are more of a hit-or-miss thing than the inevitable result of hard thinking by an intelligent person: "What is particularly troubling is how little 'logic' seems to be involved in some phases of problem solving. Difficult problems are often solved via a sudden, intuitive insight. One moment you're stuck; the next moment this insight has popped into your head, though not by any step-by-step logic that can be recounted."

During interview training I participated in when I worked there, Microsoft would emphasize four attributes that it was looking for when hiring: intelligence, hard work, ability to get things done, and vision. Intelligence was always #1, yet despite this, Poundstone says that the official Microsoft people he talked to would shy away from the word "intelligence", preferring to use terms like "bandwidth" and "inventiveness". Indeed Microsoft's Interview Tips web page says "We look for original, creative thinkers, and our interview process is designed to find those people." No mention of the word intelligence or any notion that interviews are some sort of intelligence test.

In fact, although I think that most Microsoft people would consider the puzzle tests to be mainly a test of intelligence, they may do better at testing some of the other desired attributes. Psychologist and personnel researcher Harry Hepner once said, "Creative thinkers make many false starts, and continually waver between unmanageable fantasies and systematic attack." Poundstone explains that you have to figure out when your fantasies have become too unmanageable: "To deal effectively with puzzles (and with the bigger problems for which they may be a model), you must operate on two or more levels simultaneously. One thread of consciousness tackles the problem while another, higher-level thread monitors the progress. You need to keep asking yourself 'Is this approach working? How much time have I spent on this approach, and how likely is it to produce an answer soon? Is there something else I should be trying?'"

This is great advice, not just for a puzzle, but for a job, and life in general. So watching someone think through a puzzle might be a great way to see how they would tackle a tough problem at work--the "hard work" and "get things done" abilities that Microsoft is also looking for. As James Fixx writes in the sequel More Games for the Superintelligent, "While the less intelligent person, unsure of ever being able to solve a problem at all, is easily discouraged, the intelligent person is fairly sure of succeeding and therefore presses on, discouragements be damned."

Unfortunately, the typical Microsoft interviewer is not looking at the approach to puzzle questions as a test of perseverence. Someone who tries five different attempts might demonstrate more resourcefulness than someone who just "gets it"--but they would get turned down. Interviewers who ask puzzle questions are probing the "intelligence" category, and they want the right answer.

The last chapter of the book is titled "How Innovative Companies Ought to Interview" and deals with a soon-to-be-problem: How will the industry be affected by the publication of this book? Will interviews still work if everyone knows the secrets?

Knowledge of Microsoft-style questions is already out there on the Internet. Since the candidates who participate in the interviews do not sign a Non-Disclosure Agreement, they are free to tell others the questions they were asked, and from these reports databases of questions have been built up. Poundstone includes the URLs of several sites, including Kiran Bondalapati's "Interview Question Bank", Michael Pryor's "Techinterview", Chris Sells' "Interviewing at Microsoft", and William Wu's "Riddles". These sites generally don't include answers, but certainly knowing the types of questions to expect can be an advantage.

Microsoft employees are aware of such sites. Once, when I sent email describing the questions I had asked a Microsoft candidate, I got a nasty reply from someone else at the company: Didn't I know that the question I had asked was posted on a website of known Microsoft interview questions? On the other hand, with no official internal Microsoft list of questions, some employees are undoubtedly using these sites to come up with material. Even within Microsoft there is debate about which questions are reasonable. In an unscientific survey I took of former Microsoft program managers, opinion was divided on the validity of some of the questions. A question described by one person as a good test of a candidate's ability was dismissed by another as foolish.

Poundstone does point out that some questions are silly and should not be asked ("Define the color green"), but he gives serious answers to others which I don't think are worthwhile either, including "If you could remove any of the fifty U.S. states, which would it be?" and "How do they make M&Ms?" Furthermore, I would argue that if an entire class of questions can be "tainted" by How Would You Move Mount Fuji?, they don't deserve to be asked in the first place. Estimation questions might be invalidated by the revelation that the way to solve them was to multiply together a bunch of wild guesses. The strategy of using a design question to to differentiate program management candidates from developer candidates might also go the way of the dodo. Is that necessarily a bad thing?

How Would You Move Mount Fuji? is worth reading even if you don't plan on interviewing at Microsoft. It has some interesting history, a few good Microsoft tidbits, and puzzles that are entertaining on their own. For those considering a job at Microsoft, the book may ratchet up the "arms race" of questions. Microsoft employees may assume that people interviewing have read the book--so if you are going to interview there, or anywhere else that imitates their style, you should probably read it too.

You can purchase How Would You Move Mount Fuji? Microsoft's Cult of the Puzzle from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.