Better by the Bunch: Evaluating Job Candidates in Groups

Re-posted from Working Knowledge, HBS
Published: June 18, 2012
Author: Maggie Starvish

New research suggests that organizations wishing to avoid gender stereotyping in the hiring or promotion process-and employ the most productive person instead—should evaluate job candidates as a group, rather than one at a time.

A recent collaboration between Harvard Business School and the Harvard Kennedy School (HKS) reveals that you're much less likely to stereotype by gender if you apply an "evaluation nudge"—an intervention aimed at overcoming biased assessments, in which people are evaluated jointly rather than separately regarding their future performance.

These findings are presented in the working paper When Performance Trumps Gender Bias: Joint versus Separate Evaluation, coauthored by HKS Professor and academic dean Iris Bohnet, HKS doctoral student Alexandra van Geen, and HBS Professor Max H. Bazerman.

"This project integrates my prior work on using joint decision-making to create more ethical decisions with Iris's long-standing expertise in developing strategies to promote gender equality," says Bazerman. "Together, the three of us have produced one of the most exciting papers that I have worked on."

The idea for the paper grew from a conundrum that Bohnet faced when she became faculty director of the Kennedy School's Women and Public Policy Program (WAPPP). Although she'd built a career on using behavioral economics to help others improve their decision-making, she couldn't figure out how to apply those insights to gender issues.

"It took me a few months," says Bohnet,"and then all of a sudden it became very obvious to me."

Employing the behavioral economics toolbox

What became obvious, Bohnet says, was that she needed to look at gender stereotyping as just another heuristic that people employ during the decision-making process. Such heuristics are often subconscious, and though they help people make decisions faster, they don't necessarily help them make better ones.

"A lot of discrimination happens because of implicit biases where people like you and me —who don't want to do the wrong thing, who don't want to discriminate, or don't want to treat people differently based on their demographic characteristics—fall into these stereotype traps because stereotypes have something to do with how the world is," says Bohnet. "As long as we don't see women in leadership positions, for example, we don't associate leadership with women. I thought if that is true, then I can treat that mistake the way I treat other mistakes people make."

This meant applying insights from the behavioral economics toolbox. The research team was interested in how organizations can change to accommodate what are often natural and subconscious individual behaviors, making it easier for a person to make the right choice. The 2008 book Nudge, written by Richard H. Thaler and Cass R. Sunstein, piqued their interest. (The book also inspired the term "evaluation nudge.")

"The contribution of the book was to show that we can start changing environments to make it easier for our minds, which aren't perfectly rational and don't absorb information perfectly, to succeed in life," Bohnet explains.

While the book's authors applied their theory to policy changes—such as the Pension Protection Act of 2006, which mandated in part that employees be automatically enrolled in a pension plan upon hiring and must choose to opt out rather than opt in—Bohnet, Bazerman, and van Geen decided to apply the theory to evaluation decisions at organizations.

"Max had thought of the particular question that we look at—joint versus separate evaluation—a long time ago," Bohnet says, adding that the idea for the paper came from "a combination of Max's earlier work thinking about joint versus separate evaluation in the domain of product evaluation, Alexandra's expertise in experimental economics and her passion for gender, and my interest in working on closing gender gaps."

The experiment

The trio's research included a two-stage experiment conducted at the Harvard Decision Science Laboratory involving some 654 male and female participants. Of those, 554 played the role of an employer selecting an employee for a job assignment, while 100 played the role of employee.
In stage one, the "employees" completed two rounds of a math or verbal task for which they were paid based on performance; they also completed a brief demographic questionnaire that included, most importantly, their gender. The nature of the tasks were deliberate: "Most studies that measure explicit gender attitudes find that females are believed to be worse at math tasks and better at verbal tasks than males," the paper explains.

In stage two, each "employer" was asked to choose an employee among the pool of participants, with the knowledge that employers would be paid according to that employee's round-two performance. The employers were informed of the employees' gender and past performance. (For this stage of the experiment, the researchers deliberately narrowed the employee pool down to participants whose stage-one task performances were similarly mediocre, such that there was no obvious star performer to choose.)

In some cases the employer was performing a joint evaluation, in which he or she was presented with the round-one results of both a male and a female employee. In others, the employer was presented with the round-one results of only one male or female employee—a separate evaluation.

The results were striking. In cases of separate evaluation, the employers were more likely to choose male employees for the math tasks and females for the verbal tasks. But in cases of joint evaluation, stereotypes did not seem to matter at all. The researchers attribute their findings to frame of reference.

"Our hunch is that the mechanism works something along the following lines: if you look at one pair of shoes, it's hard to evaluate the quality of those shoes," Bohnet explains. "You will be much more likely to go with stereotypes or heuristics or rules of thumb about shoes. But if you have several pairs of shoes available, you're much more likely to be able to compare different attributes of the shoes."

Next steps

The researchers are currently testing their theory at a large company. Instead of submitting promotion recommendations to upper management individually, supervisors from several departments are evaluating their promotion recommendations together, as well as submitting the recommendations jointly.

Bohnet has also presented the working paper at the World Economic Forum, on Capitol Hill, and at the Harvard Faculty Club.

"It's a paper that's very close to my heart," she says. "This paper has potential in academics and for the world of practice because it has insights that could be implemented by organizations tomorrow if they so choose."

The researchers encourage businesses to consider the benefits of joint evaluation not only to close the gender gap, but also because of the potential positive effects on a company's bottom line. "By basing decisions on performance rather than uninformative stereotypes," Bohnet says, "instead of hiring who you think is the most productive person, you'll hire who actually is the most productive person."

For Bazerman, the paper is a great illustration of the collaborative process among the various schools at Harvard. "Working with Iris Bohnet and Alexandra van Geen is a good example of this," he says. "Iris is a well-recognized scholar on the role of gender in decision-making and public policy, and Alexandra is a spectacular doctoral student. Like HBS, HKS and WAPPP care deeply about how to improve decision-making and find solutions to our bounded ethicality."