HUBweek What Works: Designing Inclusive Organizations

This week marked the second annual HUBweek, a celebration of art, science, and technology in the Boston area. As part of a series on “Ideas to Impact,” Professor Iris Bohnet presented insights from her new book What Works: Gender Equality by Design in conversation with Meghna Chakrabarti, host of Radio Boston on WBUR.

Professor Bohnet began by describing Heidi Roizen, a successful Silicon Valley venture capitalist and the protagonist of a case study used by business schools around the country. Researchers at Columbia University provided half of a group of students with the case with Heidi’s name attached; the other half received the same case, but with Heidi’s name replaced with “Howard.” Both male and female students rated Howard and Heidi similarly in terms of competence, but they didn’t like Heidi and didn’t want to work with her. Heidi defies our norms in two ways – she’s not what we expect a venture capitalist to look like, and she defies our conception of what a “good woman” is.

Why are our brains doing this? We are all accustomed to certain patterns in this world, and we try to fit our circumstances to these existing patterns. Look at the figure below: are boxes A and B different colors? Initially, it appears that A is much darker. However, when we eliminate the extraneous context, we can see the squares for what they really are – the same color. This is precisely what Professor Bohnet says she is trying to achieve with behavioral design.

What can behavioral design do for us that other approaches can’t? Previous studies have suggested little evidence that interventions like diversity training are working. Our minds appear to be pretty stubborn when it comes to implicit bias. Instead of trying to change mindsets, we can more effectively change environments to promote gender equality. 

Professor Bohnet gave examples of using behavioral design to promote gender equality in the workplace, including holding structured interviews for job candidates, including work-sample tests as part of the evaluation, and not allowing managers to use employee self-evaluations in their own assessments of employee performance. However, we can begin employing behavioral design much earlier. Standardized tests like the SAT that incorporate a penalty for guessing may disadvantage women, who are generally less willing to take risks. Though the test isn’t designed to encourage or reward risk-taking, there can be an appreciable score difference between students who guess and those who just skip questions, regardless of academic ability. As part of a larger revision of the SAT, College Board has now removed the penalty for guessing and, in doing so, mitigated a major source of gender bias. (For more findings like this, be sure to check out our Gender Action Portal!) 

In her first question, Ms. Chakrabarti discussed her reaction to hosting Professor Bohnet on her radio show in April. The potential for behavioral design both inspired her and caused a little sadness, as it confronts us with the idea that implicit biases are so difficult to overcome that it’s more effective and easier to make environmental or structural changes. Professor Bohnet replied that there’s good news and bad news associated with the prevalence of implicit bias – the good news is that it’s about all of us (not some “good” unbiased individuals versus “bad” biased ones) – but the bad news is that we’re all biased! Seeing is believing, she says. Changing the environment means that we start to see different role models, counter-stereotypical images, and our brains begin to create new patterns that mitigate implicit bias.

In thinking about the A and B squares image, Ms. Chakrabarti asked whether there’s such a thing as removing too much context, such that you no longer see the whole person. Perhaps in implementing interventions like using only structured interviews for hiring, we’re missing out on other diverse perspectives that applicants bring to the table. In response, Professor Bohnet said that we have to continue to measure. We need to understand what factors are predictive of future performance and which are not. Structured interviews can still measure “soft” skills that we think are important. Employers need to be intentional about what values they want to test – be it emotional intelligence or social competence – and incorporate those into questions in their structured interviews. 

Ms. Chakrabarti asked whether there is data utopianism in this approach, a sense that measurement will solve all of our problems. Professor Bohnet responded that generally, organizations just don’t measure enough to know what works and what doesn’t. Even if it’s not a panacea, we still have a long way to go in terms of collecting data so that we’re not just relying on internal stereotypes. Where we have been able to measure, the data show that these structural interventions are effective in a short amount of time. In the UK and Australia, civil service employees are now evaluated blindly. This intervention started with a pilot test, and since its implementation has doubled the number of women in senior leadership positions. The starting point is to recognize that everything is designed and become more intelligent about how we structure our world.  

The question and answer session raised a variety of issues, from the potential for gender bias in the presidential debate format, whether it might be possible to design in positive hiring bias toward diverse perspectives, and how to include transgender and gender non-conforming individuals in using behavioral design to promote gender equality. These conversations are an important starting point in generating new research, new innovations, and new behavioral insights.

Achieving Meritocracy in the Workplace

Knowing what we do about the gender wage gap and implicit bias in the workplace, what sort of measures could we take to improve pay equity? Could a meritocratic system, with pay based strictly on performance, be the answer?

This week’s WAPPP seminar featured Emilio Castilla, NTU Professor of Management at the MIT Sloan School of Management (during his spring break, no less!). Professor Castilla’s work focuses on the social aspects of work and how social and organizational processes influence employment outcomes – with important managerial implications.

Performance-reward practices are now standard within most industries – 69% of organizations offer variable bonuses based on performance, and some type of merit pay exists in almost all companies. If these pay practices are truly meritocratic, demographics shouldn’t matter. However, the gender and racial implications of merit-based employer efforts are not well understood – do these company practices improve workplace equity and diversity?

In theory, if merit pay was based solely on performance, we would see similar levels of pay for men and women at each performance rank. However, one of Professor Castilla’s studies showed that the benefits of a higher performance rank were much greater for men than for women. A man’s performance bonus was significantly larger than a woman’s at the same performance level, and the gender gap increased the higher the performance quintile.

Skeptics may argue that women may have less experience than men, may be less productive than men, or may not ask for bonuses. To test these assertions, Professor Castilla conducted a company field study of performance rewards over time. There was no evidence in the company that women or minorities received lower starting salaries than white men within a given job and work unit. However, there was evidence of performance-reward bias. Even with the same performance scores, women and minorities with the same job, work unit, supervisor, and human capital received lower salary increases than white men. This study provides some evidence that in trying to close the gender wage gap, we should really be looking at the processes and outcomes surrounding bonus pay.

The key question in thinking about organizational processes is whether the gender gap exists because of or despite employer meritocratic efforts. What is the alternative to performance-based pay? If we eliminated merit pay, would we see no bias? Or are these results better than what we had under traditional systems?

To answer these questions, Professor Castilla took to the lab. Participants were given a limited budget of $1,000 to distribute among employees in bonuses. In the meritocratic condition, participants were given performance evaluations of each employee and were told that this was the only criterion for distributing bonuses. In the non-meritocratic condition, participants still had access to performance evaluations, but were told that it was up to the manager’s discretion to distribute bonuses. In addition, researchers varied the gender of the employees to be evaluated, who each had equal performance ratings. In the non-meritocratic condition, participants awarded male and female employees about the same level of bonus, with no statistically significant difference (on average, women received about $399 and men received $401). Shockingly, in the meritocratic condition, there was an enormous gender gap in favor of men. Men received about $420 in bonus pay in the meritocratic condition, compared to $374 for women. Professor Castilla calls this the paradox of meritocracy—in an organization that emphasizes meritocracy in its culture, without specific structures or processes in place, bias is even greater.

Demographic inequality persists despite employer merit-based efforts. This finding is consistent with “moral credentialing” – in contexts where people are led to feel unbiased, fair, or objective, they are actually more likely to then behave in biased ways. By making employees feel like they’re working in a meritocracy that strictly evaluates performance, they are more likely to unleash bias. This is true for both male and female study participants distributing bonuses!

Organizational practices and processes matter, and there is unlikely to be only one solution to bias in the workplace. Still, it is critical to investigate which employer practices help to reduce workplace inequality and increase diversity. Professor Castilla undertook one large firm-level study designed to empirically test whether introducing organizational accountability and transparency policies reduced any pay gap based on employee demographics. Performance management at this company was a three-step process: supervisors met with employees once a year to discuss their performance and provide feedback, but were not responsible for setting compensation. This step is characterized by high accountability and transparency: the supervisor is responsible for providing feedback, and the employee has to acknowledge receiving the feedback and register their reaction to it. However, steps two and three are less visible. In step two, compensation is decided at the work unit level, and in step three HR rubber stamps the work unit’s decision. The key intervention here would be to increase accountability and transparency in steps two and three.

The company created a Performance-Reward Committee that Professor Castilla emphasized was a task force in the truest sense of the term: they had a defined task and the force to intervene if they found any evidence of pay discrepancy and bias. Once compensation-setters had to be accountable and visible in terms of their process and outcome, demographic variables played no significant role in bonus pay. Instead, performance ratings became even more statistically significant.

Professor Castilla’s work illustrates some of the critical challenges faced by employers. Meritocracy is harder than it looks! Merit-based efforts can activate implicit bias above and beyond traditional processes. However, Professor Castilla emphasizes, the key lesson here is not “we shouldn’t adopt practices to increase fairness and equity.” Instead, we should pay close attention to implementation, accountability, and transparency.

Read the paper: 

The Paradox of Meritocracy in Organizations 
Authors: Emilio J. Castilla Stephan Benard 
Organizations that emphasize merit-based cultures, while intending to increase opportunities, fairness, and equity, may inadvertently be disadvantaging women.