Can Symbolic Awards Really Motivate? Evidence from a Field Experiment at Wikipedia

ICan symbolic awards motivate individuals to contribute their ideas and knowledge to a common project? This is the question that Jana Gallus, Postdoctoral Fellow with the Behavioral Insights Group at the Harvard Kennedy School, asked in her research, presented at the latest WAPPP Seminar. She found the answer by going where many professors tell students never to go for reliable information: Wikipedia.

Dr. Gallus talking about Wikipedia's editor retention problem

Wikipedia can be an enigma to economists. It is a place where motivated souls go to work for no pay. It has been enormously successful at compiling massive amounts of human knowledge through the sheer dedication of "Wikipedians", who put in time and effort to produce online articles on a range of topics. Unfortunately, initial excitement over the site peaked and the number of editors has been on a downward trend ever since. The situation is a significant problem for Wikipedia. Fortunately, they ran into Gallus, an economist who is contributing to the emerging literature on awards, who had some ideas about how to resolve this problem. 

Awards come in many shapes and sizes. There's great variety of fields, awarding parties, prize components, and periodicities. Dr. Gallus argues that there are important differences between ex ante awards that are previously announced and select winners through clear criteria, such as contests or performance-based awards, and what she calls ex post awards, that are presented for an achievement after the fact, and where the receiving party may not even be aware that they are eligible for an award. Examples of the former type could be an innovation contest where contestants enter to win, or a company bonus for the best sales performance, whereas examples of the latter can include something like a Pulitzer Prize, an Academy Award, or a Nobel Prize. Gallus ran a field experiment at Wikipedia to identify whether ex post awards could be used as motivational tools for newcomers to a field or whether they were only effective as metaphorical crowns for individuals who are already high-achieving.

Wikipedia and Dr. Gallus created a digital badge, the Edelweiss Flower, that would be randomly awarded to new editors on the website (she made sure to implement a rules-based preselection to filter out, for instance, vandals and corporate accounts, so they wouldn't receive anything). Out of the 4,007 editors that first joined Wikipedia in that role between September 2012 and July 2013, 1,617 were included in an award cohort and 2,390 were in the control group, receiving nothing and unaware that the experiment was even happening. The "honored" editors would receive a message thanking them for their contribution and informing them that they had been awarded the Edelweiss. The message included the image of the "medal", which users quickly made into a badge that could be displayed on their profile.

Did users who receive the award stay on as editors more than those who did not? Yes! 43.5% of the editors who did not receive the award did not come back, but only 34.8% of those honored left. A very noticeable difference! What is more is that this effect can be causally attributed to the granting of the award, because the treatment and the control group are virtually no different except for this factor. Gallus commented that theory would explain this observed effect by positing that 1) the badge gives people status, "it gives them reputational capital that they can later use", in her words; 2) they also derive social identity through this kind of categorization, and finally, 3) they value recognition from others. She also found statistically significant effects extending up to six months after the new editors received the Edelweiss.

Though impressive, her work with Wikipedia is far from over. The online encyclopedia identified a very wide gender gap through a survey of editors, and they are looking for ways to close it. Currently, more than 80% of Wikipedia editors are men. Gallus hypothesizes that this may be due to lack of free time, a "harsh" online forum culture, or even "self-stereotyping". Ways to tweak the awards so they may be enticing for women could be to vary the degree of publicity, consider group awards, use bottom-up nomination mechanisms, de-bias juries, and of course, favor ex post awards. As more and more knowledge is produced and stored in places like Wikipedia, her important work will contribute to increase the amount of women sharing ideas and furthering human knowledge.

What Works: Gender Equality by Design

Imagine the following situation: You are a young professional musician. And you are really good. Music is what makes your blood flow. Ever since you were a little girl, piano and violin lessons excited you rather than bored you. Today is a very important day for you: You're auditioning for a position in the National Symphony Orchestra. How would you feel if you knew that the minute you walked on stage, before even playing your instrument, your chances of being hired would decrease significantly?

Before research showed that having musicians audition behind a curtain, so the jury would not be able to tell their gender, increased the chance that a woman would be hired or promoted, and that these "blind" auditions alone could account for a third of the increase in the proportion of women musicians hired into top-tier American symphonies, female musicians would face just that scenario. The implicit biases of possibly well-meaning members of the jury would too often reduce women's chances to succeed in the audition.

Although we would all like to think we do not suffer from the same biases as the members of those juries, the opposite is likely true. During the first HKS Women and Public Policy Program seminar of the academic year, Professor Iris Bohnet explained that we are all biased in one way or another, "because seeing is believing". We observe patterns in the world, such as most kindergarten teachers being female, or most software engineers being male, so we come to expect people to fill those roles. Don't believe it? Take the test yourself.

Professor Bohnet is the Director of the HKS Women and Public Policy Program (WAPPP) and Co-chair of the Behavioral Insights Group at the Center for Public Leadership at HKS. During the seminar, she presented a preview of her forthcoming book “What Works: Gender Equality by Design”, in which she argues that we can use insights we learn from Behavioral Economics to close gender gaps caused by implicit biases.

Professor Iris Bohnet, Director of the HKS Women and Public Policy Program

These insights allow us to create “nudges", which are small actions designed to obtain the most desirable reactions from people, building on knowledge of how the -often irrational- human mind actually works. In the book, she talks about "nudges we can use to make the world a better place", because they can reframe the environments in which we work. Best of all, they are mostly cheap and can be introduced quickly.

Professor Bohnet described previous approaches to increasing diversity in the workforce, such as diversity, leadership, and negotiation training, and underscored that there is not enough evidence to prove that these interventions work. On the other hand, interventions like long-term capacity-building or mentoring have been found to be very promising. In a study that followed the career trajectories of women economics professors who were randomly assigned into a long term mentorship program, the professors in the program fared better than those in the control group.

She mentioned many other nudges to redesign the work environment, like putting up more images of female leaders -"what you see matters in what you think is possible for yourself"-, avoiding panel interviews, assessing job candidates on a pre-determined set of questions immediately after the interview, highlighting the increased presence of gender mixed corporate boards rather than their low proportion, and many more. Professor Bohnet is handing in the manuscript for the book next week, so look forward to reading more when it comes out!

When to Compete: Do Women Know Better?

Women are more attuned to competitive environments and change their behavior accordingly, argued Harvard Business School doctoral candidate Pinar Fletcher, at this week’s WAPPP seminar. Titled “Competing at All Costs: Gender and Dysfunctional Competition,” the presentation focused on destructive competitive behavior and how it varies by gender and environment.

Past research featured in a 2013 WAPPP seminar, indicated that there are significant gender differences when it comes to competition. A randomized control trial revealed that 73% of men chose to compete in winner-takes-all competitions for potentially higher pay over working independently for lower pay under piece-rate pay schemes, while just 35% of women chose the winner-takes-all option. Researchers have been trying to explain this gender gap for years now, and Fletcher hopes to use the field of behavioral economics to better explain the issue.

Some arguments frame this gender difference as a problem, arguing that women’s propensity to not compete is a weakness, but Fletcher pointed out that men often lose money by choosing the competitive scheme. Since men are not always benefitting from their higher propensity to compete, it could be useful to examine the competitive context.

Fletcher’s research, which she conducted with coauthor and HBS Professor Kathleen McGinn, looks explicitly at destructive competitive behavior, i.e. competitive behavior that aims to hurt competitor(s) but has the potential to give net losses or net gains to the instigator. In an organizational context, the factors that determine the costs and benefits of destructive competitive behavior are (1) the incentive systems, (2) the performance feedback systems, meaning what your performance is compared to, and (3) behavioral norms of the work environment.

There are two organizational contexts in which this kind of competition can occur: high intensity and low intensity. Low intensity environments are still competitive, but individuals feel that their performance matters regardless of how they compare with others, usually because they receive feedback in relation to how they did over time or based on objective standards. Here, the expected payoff of destructive competitive behavior is uncertain because the costs of competing in that manner outweigh the benefits. High intensity environments are just the opposite, stressing that winning over someone else is key. In these environments, there is more certainty that destructive competition will bring more benefits than costs.

Fletcher’s hypothesis was that the propensity for destructive competitive behavior would be greater in high intensity environments than in low intensity ones but predicted that an interaction between gender and competition intensity would mitigate these differences. The authors predicted that in high intensity situations, the certainty of the expected payoff of competitive behavior would wipe out any gender differences, while in low intensity situations, the ambiguity would allow gender differences to creep back in.

Fletcher and McGinn conducted three randomized control trials involving partnered tasks with varying payment schemes, all of which confirmed these hypotheses. The first and second studies focused on incentive systems and social comparison/performance feedback, while the third study focused on behavioral norms.

The first study, in which everyone was paired with a relatively well-performing competitor, participants were assigned to either a high intensity or low intensity environment and made decisions in “strong” – or certain – contexts as well as “weak” – or ambiguous – contexts. In strong situations with high intensity competitions, men and women competed just as much as men.
In weak situations with low intensity competitions, men performed better than women. However, as Fletcher noted, women’s actual net payoff was greater on average than men’s.

The second study manipulated the incentive scheme and performance feedback type separately to parse apart how men and women reacted differently to incentives. This study replicated the findings of the first study, showing that women change their competitive behavior in response to the environment, while men exhibit the same competitive behavior across conditions. Additionally, men factor in incentive scheme to some extent, but not as well as women do.

The third study revealed that the overwhelming majority (~90%) of destructive competitive behavior was driven by a utilitarian or self-serving justification but that it varied by the competitive behavioral norms of the environment.

In short, men are more likely to engage in reflexive competitive behavior, while women are more context-dependent and attuned to the “rules of the game,” but gender differences in destructive competitive behavior only reveal themselves in low intensity competition settings. In addition, women in high intensity settings perceive their competition to be with female colleagues, while the perception in “low intensity” settings is competition with their male colleagues. Women reap more benefits from destructive competition in high intensity situations, however, so future research looking into women’s advantages in these settings may help organizations perform better.