Showing posts with label experimental design. Show all posts
Showing posts with label experimental design. Show all posts

Saturday, December 8, 2012

Under Pressure!


Papers, finals, job search, the holidays – the pressure of it all! I was hoping to learn in this seminar that being a woman would somehow improve my performance. What I did learn was not quite so straightforward, but still fascinating. Some experimental studies in recent years had found that in a mixed-sex environment, the pressure of competition improved men’s performance, but not women’s, and that women often opted out of competitive environments. Olga Shruchkov, Assistant Professor of Economics at Wellesley College, noticed something about those experiments. The “tasks” which subjects performed under competitive pressure were invariably math-oriented, like addition problems and mazes, and there was no range of performance quality, only right or wrong answers.

Shruchkov designed an experiment that would fill these gaps – a Boggle-like verbal task, a comparable math task and a difference between competitive pressure and time pressure. This experimental design was not only creative, but also rigorous, with relevant controls, task-consistency mechanisms and a whole range of variables to measure. The experiments replicated previous findings that under time pressure on math tasks men attained higher scores and had a higher propensity to select into tournaments.

The innovative experimental design also yielded important new insights. On verbal tasks under time pressure, women performed as well as men and had a similar propensity to enter competitions. In a verbal competition under time pressure, women did slightly better than men. Things really took off once the time pressure was lowered – women outperformed men on the verbal tasks under competition. On the math task with low time pressure, women and men performed similarly both with and without competition. Interestingly, giving more time for the math task doubled the number of women who elected to compete. So it’s not the pressure of the competition or inherent math abilities that were hurting the performance of women in earlier experiments – it was time!

Why did time help women so much more than the men? Making the competition about total points, not speed, allowed for a measurement of the quality dimension. A look at the kinds of words women were finding when given the time reveals an emphasis on quality – they made longer words and made fewer mistakes. The men, on the other hand, went for quantity of words and in the process lost points for misspellings and typos. Perhaps more women participated in the math tournament when they had more time, because it allowed them to check their calculations more thoroughly.

Professor Shruchkov was careful with drawing policy implications out of experimental results, but she suggested that take-home exams in math and science might help more girls excel in those subjects early on, which could get more of them interested in pursuing those careers. Personally, I am tempted to use the results to ask for a final paper extension right now! I wonder what Shruchkov tells her students when they try to do that. 

Anya Malkov is an MPP candidate at the Harvard Kennedy School, a WAPPP Cultural Bridge Fellow, and an alumna of From Harvard Square to the Oval Office.

Friday, October 19, 2012

Professors Are Biased Too


An ordinary e-mail once caught the eye of Modupe Akinola,Assistant Professor of Management at Columbia Business School. It was from a prospective Ph.D. student, an African-American woman, asking for a meeting.

Dr. Akinola wondered: “Would I have reacted differently if this was from a man? Would it make a difference if the request was for today and not next week?” She joined with colleagues Katherine Milkman and DollyChugh to turn these questions into research questions in a field experiment measuring discrimination in academia.

The creative and elaborate experiment started with putting together a representative sample of over 6,500 faculty members from 260 universities. The team used publicly available demographic information on professors to ensure a good sample. The researchers then created a few prospective Ph.D. students. The names of these imaginary students were tested and proved to be easily recognizable as either male or female and as Caucasian, Black, Hispanic, Indian or Chinese.

At 8:00 am on a Monday, e-mails from Brad Andersons, Latoya Browns and 18 others went out to unsuspecting subjects, asking to meet “today” in half the cases and “next Monday” in the other half. The research team was impressed by how quickly most faculty wrote back and how often they agreed to meet, but when and with whom they agreed to meet was more interesting.

One would think academia, with its affirmative action and enlightened world views, might be a post-racial, post-gender sphere. As it turns out – not so much, especially not when there is time to think about it.

In the now condition, Akinola, Milkman and Chugh found no significant difference between number of responses to white males and to others, but when the request was for later, white men received both more responses and more meeting acceptances than any other prospective students. The white men fared better than all other racial categories across almost all disciplines. The effect remained even when the race or gender of faculty and student matched.

According to construal level theory (CLT) immediate events demand concrete reasoning focused on feasibility, whereas decisions about distant events trigger abstract thinking and a focus on desirability of the event. Thus, future events create more room for unconscious stereotypes and biases to enter the decision-making process. In this study, for instance, the responses received to the “now” requests systematically displayed a focus on “how,” whereas the “later” e-mails were more likely to be answered with a request for credentials or more information – a “why” response.

The point of this experiment and other studies that spotlight the influence of stereotypes and biases on consequential decisions is not to call people racist or sexist. The point is to come up with ways to address the biases and to improve minority outcomes. For instance, referring all prospective students to the Ph.D. program coordinator would take the burden off individual professors and ensure that all students get their questions answered. This is just one option that an experiment like this might bring up for discussion.

Unfortunately, some faculty did not take kindly to the experiment. When debriefed and told that they had been used in a study of bias, many expressed outrage and demanded to speak with the Institutional Review Board.

To me, their reaction suggests the thorny subjects of race and gender are not studied enough in academia. Perhaps academia, more so than other spheres, makes it taboo to have biases, but shaming does not make biases go away! Nobody wants to be called a racist or sexist, not even a little bit, but simply feeling guilty is not productive. We can structure decision-making in ways that level the playing field, and we need creative researchers like Akinola, Milkman and Chugh to make that happen.

Anya Malkov is an MPP2 at the Harvard Kennedy School, a WAPPP Cultural Bridge Fellow, and an alumna of From Harvard Square to the Oval Office.