Early in his career Kahneman served in the Israeli military, and he was given the responsibility of improving the interview process by which recruits were assessed for combat-readiness and assigned to various units. The current process consisted largely of unstructured interviews, Kahneman recalled in Thinking, Fast and Slow, and "was almost useless for predicting the future success of recruits."
Kahneman drew on the work of psychologist Paul Meehl, who pioneered research into the advantages of statistical algorithms, to apply the following guidelines in overhauling the interview process: 1) Create a structured set of questions to prevent the interviewer from simply pursuing the questions that they find most interesting, 2) Use the limited time available to obtain as much information as possible about the interviewee's life in their normal environment, 3) Do not make decisions solely on the basis of interviewers' global evaluations, and 4) Rely upon statistical summaries of separately evaluated attributes.
Kahneman identified six attributes that he deemed relevant to combat performance and then composed a series of factual questions about the recruit's life related to each attribute. The interviewers were instructed to complete the set of questions for a given attribute and score the recruit on a five-point scale before moving on to the next attribute, as a precaution against the halo effect, in which first impressions influence subsequent judgments.
But the interviewers rebelled against Kahneman's new process, which asked them merely to record recruits' responses to standardized questions and made no allowance for their intuitive judgment. Kahneman relented: "'Carry out the interview exactly as instructed,' I told them, 'and when you are done, have your wish: close your eyes, try to imagine the recruit as a soldier, and assign him a score of 1 to 5.'"
The new process proved successful: "The sum of our six attribute ratings predicted soldiers' performance much more accurately than the global evaluations of the previous interviewing method." But it also contained a surprise:
The intuitive judgment that the interviewers summoned up in the 'close your eyes' exercise also did very well, indeed just as well as the sum of the six specific ratings. I learned from this finding a lesson that I have never forgotten: intuition adds value even in the justly derided selection interview, but only after a disciplined collection of objective information and disciplined scoring of separate traits. I set a formula that gave the "close your eyes" evaluation the same weight as the sum of the six trait ratings. A more general lesson that I learned from this episode was do not simply trust intuitive judgment--your own or that of others--but do not dismiss it either.
Kahneman concludes with some specific guidance for an individual leader seeking to apply these principles when hiring:
First, select a few traits that are prerequisites for success in this position... Don't overdo it- - six dimensions is a good number. The traits you choose should be as independent as possible from each other, and you should feel that you can assess them reliably by asking a few factual questions. Next, make a list of those questions for each trait, and think about how you will score it, say on a 1-5 scale. You should have an idea of what you will call "very weak" or "very strong."
To avoid halo effects, you must collect the information on one trait at a time, scoring each before you move on to the next one. Do not skip around. To evaluate each candidate, add up the six scores. Because you are in charge of the final decision, you should not do a "close your eyes." Firmly resolve that you will hire the candidate whose final score is the highest, even if there is another one whom you like better.. A vast amount of research offers a promise: you are much more likely to find the best candidate if you use this procedure than if you do what people normally do in such situations, which is to go into the interview unprepared and to make choices by an overall intuitive judgment.
It's worth noting Kahneman's approach is consistent with Google's experience regarding the benefits of structured interviews:
Structured interviews are better at indicating who will do well on the job: Results show that structured interviews are more predictive of job performance than unstructured interviews when comparing interview scores to the performance scores of those hires across functions and levels.
Interviewers are happier and saving time: Using pre-made, high-quality questions, guides, and rubrics saves on average 40 minutes per interview. Googlers conducting structured interviews reported that they felt more prepared when interviewing the candidate.
Structured interviews make candidates happier: The team has seen an uptick in candidate satisfaction in feedback scores for structured interview candidates. Interestingly, scores indicated an especially big difference in candidate satisfaction rates when comparing rejected candidates. Rejected candidates who had a structured interview were 35% happier than those who did not have a structured interview.
Despite the benefits of structured interviews, there are several difficulties that must be overcome. It can be extremely challenging to identify the "traits that are prerequisites for success," craft a set of questions that reliably elicit evidence of those traits, and then calibrate scores across responses. And yet even undertaking the effort can help an organization better understand their approach to interviewing, uncover sources of bias, and devise a more objective process.
Further, in many circumstances--and particularly when hiring for more senior roles--interviewers are seeking not only to select the best candidate, but also to sell that candidate on the opportunity to join the organization. At times it's possible to distinguish between selection conversations and selling conversations, but often the two must be woven together, and here it can be useful simply for the interviewer to be more deliberate in adopting the appropriate mindset in a given stage of the conversation, rather than defaulting to what feels more comfortable.
Interviews aren't going away--and they don't have to. When organizations correct for the likelihood of cognitive biases, they can take steps to avoid assembling a team of harmonious, homogeneous under-performers. By substituting a more structured process, they increase the odds that interviews