The post is dedicated to Judge Nancy Gertner of Harvard Law School (former member of the Presidential Commission on the US Supreme Court) for her pioneering work in the area of reproductive health.
In the early years of the 1990s, Lani Guinier and her co-authors in “On Becoming Gentlemen: Women’s Experiences at One Ivy League Law School” chronicle a law school experience stratified by gender. Based on survey and focus group data, the authors argue that women at our law school, the University of Pennsylvania Law School, 30 years ago were significantly more likely to experience both discomforts with their class performance and alienation from the learning environment. Two of the hypotheses put forward to examine the causal links between academic performance and classroom experience and overall law school performance and mentorship.
Thirty years later, in the fall of 2020, the class on Women, Law and leadership Class became an incubator to explore these hypotheses through a set of interviews and collection of qualitative data. We hypothesized that women students' experience in our class would be different from their predecessors studied in “Becoming Gentlemen” by Guinier and her co-authors. Based on the over 100 interviews of male allies at Penn Law, we claim that this change is mainly due to the transformation in the attitudes of the male peers in the classroom and the conduct of male leaders in the workplace.
The 100 plus male students who were interviewed supported their female peers and the values of gender equality in general. The changes in male attitudes were key to altering the learning and working environments. While Guinier and co-authors showcased how women were becoming “bi-cultural” and adopting male tendencies to succeed, we noted that men rather than the women were becoming “bi-cultural.” Men were now more likely to embrace gender-sensitive attitudes and more systemic and structural change on caregiving and workplace organizational behavior. Most of our respondents found it important to amplify women’s voices, not only because it was the right thing to do but because these diverse voices enriched their own insights on law and life.
The male ally interviews were combined with mini-surveys on how women in the class experience bias. These mini-surveys were two-pronged: The first survey included fifty women: women in the class on Women, Law, and Leadership, and the women students in the Policy Lab on Sexual Harassment. The second survey included Black women in the class and their peers from outside the class.
The initial impulse for these surveys grew out of our in-class study of Deborah Rhode’s extensive work on the experiences of women in the legal profession, David Wilkins’s corpus of scholarship on diversity in the legal profession, Kenneth Mack’s work on Sadie T.M. Alexander and the history of Black women in the legal profession, Martha Minow’s scholarship on inclusion, and Vicki Schultz on her examination of implicit bias and women’s experiences in the workplace. We also immersed ourselves in the intersectionality work of Kimberly Crenshaw. We also read text on stereotype threat, such as Claude Steele’s “Whistling Vivaldi and Other Clues to How Stereotypes Affect Us” and discussed modern-day variations of bias, including Isabel Wilkerson’s “Caste: The Origins of Our Discontent.”
Growing out of class discussions of the “Thousand Papercuts” in Vicki Schultz’s retelling of the biases that women in Silicon Valley face, students suggested mini-surveys of what would constitute modern-day papercuts—those daily indignities and exclusions that women face that in isolation may not be problematic, but in the aggregate could cause damage.
Although our qualitative data shows that attitudes among male students at Penn Law have changed dramatically, how women internalize stereotypes and the threat that these unexamined assumptions pose remain real and have changed little over the thirty years.
It is clear that women then as now internalize stereotypes to their detriment. In “Becoming Gentlemen,” a female student stated 30 years ago: “After I discovered I was being called a feminized dyke, I never spoke in class again.” In 2020, the vernacular may have changed, but harmful stereotypes still lurk in the shadows, and women tend to self-censure based on the fear of those tropes of the Janus- faced “aggressive” and “meek” female. However, what differentiates the current women and men of Penn Law from their predecessors 30 years ago is that they are no longer silent about gender and intersectional stratification issues.
Our data show that subtle biases and stereotypes remain pervasive and might be masked by social protocols that normalize such behavioral attitudes. However, men and women are aware of these invisible barriers to success and are no longer passive bystanders to a parade of caste protocols. In the final analysis, there is a marked shift from Penn Law women “becoming gentlemen” to both Penn Law women and men “becoming norm entrepreneurs” who are interested in changing social norms.
Survey of Gender Bias in Elite Law Firms in China
Through a survey of close to 450 emerging women leaders in the law, interviews with male allies and female partners at major law firms in China, four Chinese lawyers who are LLM students at the University of Pennsylvania Law School identified some key barriers to women's leadership and the role of male allyship. In this groundbreaking study, the students used a mixed-methods approach to gather qualitative and quantitative data about the gendered factors affecting women lawyers in selected elite law firms in China.
One part of this study surveyed nearly 450 young lawyers aged 25- 30 in elite law firms on their experience of bias and attitudes. What was significant was their attitude toward pregnancy. Given the age group, only 7.92 percent of the women had children, but 100 percent of this cohort agreed that having a child had a negative impact on their career. Both the quantitative and qualitative data showed a fear of the “motherhood burden” and young women lawyers' fears of the negative impact of motherhood on a career in elite law firms.
"Once female lawyers take parental leave, their clients would be grabbed by other lawyers. To endure fairness, I think the only solution is to have mandatory parental leave on both female and male lawyers . . . I am still single, but I am so worried."
Another lawyer stated: "I feel sad that there exists no discussion about gender bias in the legal industry in China." This is an important finding. As Joan Williams says in the ABA Commission on Women in the Profession report on "Interrupting Racial and Gender Bias in the Legal Profession, "You cannot change what you cannot see." Seeing then is the first step. As our researchers point out, “It is important that we be trained to "see" gender bias lest it becomes invisible and ignored.”
Black Women Future Leaders
The Report on “Black Women Future Leaders” analyzes the results of a survey of BLSA students and finds that the threat of stereotypes acts as a silencing tool. Even when students may not have had a personal experience of being labeled, they adjust their behavior and speech patterns to avoid those labels.
As Isabel Wikerson writes in “Caste: The Origins of Our Discontent:”
“Caste is more than rank; it is a state of mind that holds everyone captive, the dominant imprisoned in an illusion of their own entitlement, the subordinate trapped in the purgatory of someone else's definition of who they are and who they should be."
The reports that grew out of our class discussion examine the insidious and often invisible undercurrents of bias that confine women, especially minority women, in a way that deprives all of us of the use of a basic human trait, the power of our imagination to see outside of a narrowly imagined sense of the world.
On that note of the power of imagination, the speakers in our class shared their art as a powerful tool to open up difficult conversations and share stories. David Hornik shared with us his extensive art collection by Black and Asian artists. He also shared with us Edward McClunny’s print of Thurgood Marshall. We include it in our report.
New York Public Library General Counsel Michele Mayes, an avid art collector, showed the class a mixed-media piece by Charly Palmer depiction of Martin Luther King Jr. standing in line to vote with one of his daughters.
Lawyer and art entrepreneur Shalini Ganendra discussed curating practices in light of racial injustice. As a Fellow at Oxford, she discussed her work on the influence of colonization on art and art critique.
The art depicted in the covers of our reports and the reports themselves are an invitation to engage in conversation about these modern-day biases that are hard to address. As Wilkerson says: “Modern-day caste protocols are less often about overt attacks or conscious hostility… They are like the wind, powerful enough to knock you down but invisible as they go about their work.”
I urge governments to put women and girls at the centre of their efforts to recover from COVID-19. That starts with women as leaders, with equal representation and decision-making power. Nearly 60 percent of women around the world work in the informal economy, earning less, saving less, and at greater risk of falling into poverty. As markets fall and businesses close, millions of women’s jobs have disappeared. At the same time as they are losing paid employment, women’s unpaid care work has increased exponentially as a result of school closures and the increased needs of older people. These currents are combining as never before to defeat women’s rights and deny women’s opportunities.
Our friend, Okonjo Iweala, is also known as “Wahala,” a popular Pidgin English word in Nigerian meaning trouble. She told us, “I loved this nickname …. To me, it was a badge of honor.” As a world-renowned development economist, author, and advocate, Okonjo Iweala has been a force for gender-equal economic development, sustainable financing, and anti-corruption. Okonjo Iweala has held some of the most distinguished positions in the government of Nigeria, the World Bank, and in global multilateral institutions. Okonjo Iweala’s impressive ability to drive change makes her one of the most influential figures on the world stage. In a time of global volatility, it is a time to take stock of how she will govern at the WTO.
In February 2019, we conducted a two-week-long interview on redefining leadership with Okonjo Iweala for our study on “Redefining Leadership in the Age of SDGs.”
"If you find problems, you must find solutions,” she often says. When we asked her about her favorite leadership philosophy, she told us: “Investing in women is smart economics, and investing in girls, catching them upstream, is even smarter economics.” She often turns to Nelson Mandela’s leadership for guidance. Her favorite quote is from that historic day on the evening of May 2, 1994, when Mandela claimed victory in the first democratic elections in South Africa: “I am your servant, I don’t come to you as leader… Leaders come and go, but the organization and the collective leadership that has looked after the fortunes and reverses of this organization will always be there.” Okonjo Iweala feels the same about being a servant leader, a servant leader who is also not afraid to cause trouble. She is also inspired by Desmond Tutu’s definition of the Ubuntu principles. It is a difficult concept to translate into English: “A person is a person through other people.” In Xhosa ubuntu ungamntu ngabanye abantu, and in Zulu umuntu ngumuntu ngabanye means “I am human because I belong, I participate, and I share.” Recently, she sent us her favorite Igbo quote on leadership: “Aka nni kwo aka ekpe, aka ekpe akwo aka nni nwancha adi ocha”. Translated into English: "When the right hand washes the left hand, and the left hand washes the right hand, both are clean.” It speaks to helping each other, partnering, and sharing responsibility together.
A globalist and an African to the core, she sees new opportunities where others see challenges. She sees the problem of a “single story” about any region, especially Africa. The telecom revolution has created a mini-revolution in the area. Africa is ahead with mobile money-pay for solar with cards. Another innovation is the mitigating effects of climate change on 32 countries in Africa through the African Risk Capacity—the weather-based insurance initiative that Okonjo Iweala is heading. The idea she explained to us was for Africans to look for solutions in their own region.
Trained at Harvard and MIT as an economist, Okonjo Iweala served two terms as Finance Minister of Nigeria from 2003-2006 and 2011-2015, and as Nigeria’s Foreign Minister in 2006. She was the first woman to hold both positions. She has spent more than two decades at the World Bank as a development economist, rising to the number-two position of managing director, which she served from 2007-2011. While at the World Bank, she was responsible for an $81 billion operational portfolio, including Europe and Central Asia, South Asia, and Africa. In 2012, she and Colombia’s Jose Antonio Ocampo squared off against American physician Jim Yong Kim in The World Bank’s first-ever contested presidential election. Although she was unable to break the traditional gentleman's agreement on the World Bank leadership, Okonjo Iweala helped to challenge business as usual with her candidacy and has laid the foundation for future challenges from non-Americans, especially from developing countries.
Okonjo Iweala was the head of the board of the Global Alliance for Vaccines and Immunization (GAVI) Board. In this role, along with Larry Summers and the Ministers of Health of developing countries, she spearheaded crucial immunizations and health services to children, focusing on girls in developing countries. A pre-pandemic study in Health Affairs covering 73 GAVI-supported countries over the 2011-2020 period shows that for every US$1 spent on immunization, US$16 are saved in healthcare costs, lost wages, and lost productivity due to illness, and return on investment increases to US$ 44 when taking into account the broader benefits of people living longer and healthier lives. Also, Okonjo Iweala has used her role as chair for the Board of GAVI to introduce a new era of public-private partnerships between multilateral organizations, the private sector, civil society, developed and developing country governments.
As a development economist with a feminist perspective, Okonjo-Iweala implemented a budgetary incentive program that would motivate ministries to implement initiatives to empower girls and women in their sector. Simultaneously, the Ministry of Agriculture had developed a new e-wallet system, which transferred subsidies directly to farmers through the financial technology, removed the government from the supply changes, and allowed farmers to directly purchase the fertilizer and pesticides they needed. Seizing the moment, Okonjo-Iweala offered the ministry a budget increase as a reward for bringing this new, innovative technology to more women. Okonjo Iweala’s leadership efforts enabled 3 million women to participate in the e-wallet program in 2014. Furthermore, as Minister of Finance, Okonjo-Iweala was able to leverage the resources at her disposal to work with other ministers such as the Minister of Agriculture to bring technological and financial resources to women in rural areas. Additional achievements with budget-incentives include the Ministry of Water Resources developing a new system for women to manage their communities’ water and sanitation centers and the Ministry of Public Works developing a new training regimen to propel women into subcontractors' positions in procurement. Within Nigeria, she helped support entrepreneurial citizens through the You WIN Program, the GWiN program (Growing Girls and Women in Nigeria), and the Development Bank of Nigeria.
The greatest war she has fought has been against corruption. A 2002 African Union study estimated that corruption costs the continent roughly $150 billion a year. To compare, developed countries gave $22 billion in aid to sub-Saharan Africa in 2008, according to the Organization for Economic Cooperation and Development. It has been estimated that Nigeria has lost more than 600 billion to corruption since independence. This crusade has threatened her personal security. In 2012, Okonjo Iweala’s 83-year-old mother was kidnapped in retaliation for Okonjo Iweala’s leadership in anti-corruption policies and Nigerian government reform. The kidnappers demanded that Okonjo Iweala publicly resign from office. She did not, and her mother escaped. Okonjo Iweala discusses the danger of confronting deep-seated corruption in her most recent book, Fighting Corruption Is Dangerous (2018). Okonjo Iweala’s vision for macroeconomic reform in Africa considers and combines African culture and history. In Reforming the Unreformable: Lessons from Nigeria (2012), Okonjo Iweala presents a framework used by her and her team that stabilized the macroeconomy, increased economic growth and fiscal transparency, reduced the debt burden, strengthened the integrity of public and civil service, and redirected resources being siphoned to private interest back to the people and the poor. She also details the challenges and confronts head-on the daunting complexity of pushing for macroeconomic and development economic reform, as well as pushing back against corrupt trade, tariffs, and customs practices.
In her most recent co-authored book with Hon. Julia Gillard, Australia's first woman head of state, Okonjo Iweala analyzes leadership lessons of women leaders from Hillary Clinton to Jacinda Arden. At a time when women are helping to steer their countries out of the pandemic, as the first woman and first African to head the WTO, Okonjo Iweala is poised to bring these critical inclusive perspectives to the complex task of a global economic recovery.
NgoziOkonjo-Iweala is the fifth Angelopoulos Global Public Leaders Fellow at the Harvard Kennedy School and delivered the Robert S. McNamara Lecture on War and Peace at Harvard in 2019.
As our public reckoning on systemic racism and structural bias reaches its climax following weeks of Black Lives Matter protests around the world, we have been asking ourselves and our academic colleagues what role AI plays in all this – and what role AI ought to play in the future. In the same way that the MeToo Movement challenged our collective conscience about sexism in the workplace, we believe that Black Lives Matter and other movements that aim to address systemic discrimination in society have the potential to reshape our thinking on bias in AI.
Writing recently in the Atlantic, Secretary Hillary Clinton called for gender blind reviews of resumes in line with the success of gender-blind orchestra auditions. All of this calls for a fresh interrogation of the role of AI in addressing implicit bias in the workplace.
Many before us have detailed the causes of bias in AI and in doing so have contributed hugely to the positive reforms that have followed. Yet, few have gone further than identifying and addressing the biases that exist on the surface, and fewer still have made the case for AI’s potential to provide the greatest step forward yet towards substantive equality.
Substantive equality, as opposed to formal equality, is a fundamental concept in human rights law which requires proactive and positive measures to be taken to ensure that persons who have faced historic discrimination have a genuinely equal chance of satisfying the criteria for access to a particular social good, such as employment or education.2 Whereas formal equality models disavow policies that aim to redress imbalances on a systemic level, a substantive sense of equality envisages an inclusive and intersectional approach that takes into account discriminatory barriers in all its forms, not just those that are obvious or intended.3 In practice, this involves analyzing the particular experiences that a person has lived through, discerning the extent of any disadvantage they have faced, and equalizing the playing field to that same extent, as and when that person seeks to access a social good.
As support for this substantive conception of equality grows, we argue that the time is ripe to shift our focus from reducing surface level bias in AI to find ways to utilize AI as a means of dismantling institutionalized discrimination. Using the employment context to support our position, we submit that this means ending our reliance on formal equality metrics to measure the success of AI systems, to make way for one based on the principle of substantive equality. While we recognize that there are risks to manipulating AI to achieve a particular outcome or result, particularly in countries with a poor track record of public governance and adhering to the rule of law, we argue that by requiring diverse interpretations of the norms subject to manipulation, and by ensuring that the underlying process is fully transparent, such risks can be effectively mitigated.
The causes of bias in AI
The causes of bias in AI are well-documented but it is important to reiterate them here, for an understanding of such causes is crucial for recognizing why formal equality models fail.
In general, bias is said to creep into AI systems in three ways:
1. PROGRAMMER BIAS: In its most obvious form, bias can get into AI systems through the conscious and unconscious biases of their human programmers. If, for example, a company wanted to use AI to screen resumes and identify leaders within an applicant pool, and that company either conscious or unconsciously believes masculine qualities to be demonstrative of leadership, then the AI system may become discriminatory as a result of the company’s discriminatory interpretation of what constitutes leadership.
2. DATA BIAS: Bias can also find its way into AI programs through source data. Gender, racial and other prejudices can creep into data sets because data sets are often reflective of the deep-seated prejudices in society. When an AI system uses a data set that contains these prejudices, it will reproduce them in its algorithmic outcomes.
3. LEARNING ALGORITHM BIAS: Finally, AI systems that use machine-learning tools can also be biased. Machine-learning algorithms produce outcomes that are in part based on training data and in part based on their own ‘learning’ – it is this second component that can give rise to problems, for the AI system may be capable of drawing its own bias conclusions, and these conclusions will be difficult to identify.
Addressing bias using the formal equality model
After identifying the causes of bias outlined above, most of our colleagues understandably go on to discuss the various ways that we can reduce such bias. For instance, Kimberley Houser, in relation to gender-based bias in AI, argues that we need only follow ‘responsible’ practices when developing and deploying AI to mitigate the risk of bias – she explains that ‘responsible’ practices include cleaning source data before use and employing diverse programmers.4 Her view is reflective of the general academic stance on tackling bias in AI.
The problem with this approach is two-fold. First, these practices have severe limitations in and of themselves. While it is true that data sets can be balanced and more diverse slates of programmers can be employed, this, on its own, will not be enough to tackle bias to the extent that Houser suggests. Bias and discrimination are complicated phenomena that have proven to take many forms. Even if we are able to modify data sets so that they are reflective of, say, gender, this will not address the full and intersectional spectrum of bias and discrimination that women experience. For instance, Houser discusses balancing data sets by replicating the profiles of women within the data set – but how will this solve the problem if the profiles of women being replicated are of white women who have never had childcare responsibilities or never been a victim of violence; Houser’s solutions ignore the particular disadvantages attached to the circumstances a person finds themselves in. It is this complexity that suggests more is needed than formal equality.5
The second issue, and the one that is of primary concern in this article, is that the approach does not go far enough; it does not redress the institutionalized bias that may exist even when the algorithmic outcome is ‘accurate’. To give an example, consider again an AI system that screens resumes and identifies leaders. Prima facie, the Al system may be doing exactly as the designer intends – producing consistently accurate outcomes that are reached regardless of the candidate’s gender, race, sexuality, or other protected characteristic. Indeed, at this point, many would argue that we have achieved “equality”. However, removing these barriers does not mean that minorities and women who have faced a history of discrimination will in fact be equal. As Fredman explains, “those who lack the requisite qualifications as a result of past discrimination will still be unable to meet job-related criteria.”6 The formal equality model assumes that once we have equal opportunities, nothing more needs to be done. But equality of opportunity is compatible with unequal results.7
In fact, traditional AI approaches completely miss the point that diversity is a far broader issue than mere checkboxes on a few external demographic factors. To understand what diversity truly means in the human race, we must understand the underlying notion of group collective intelligence. Collective intelligence is a term used to describe a group’s collective capacity and capability to solve diverse problems.8 Human collective intelligence is created by differences in people’s perspective, heuristics, interpretations, and predictive models, and those, in turn, are shaped by not just a narrow definition of demographic but a far more inclusive view that includes all identities, experiences, and training. The human race thrives because humans as a species have understood and mastered the utilization of collective intelligence; we have acted on the insight of superior decision capability that diverse groups of average problem solvers consistently outperform homogenous groups of excellent problem solvers.9 Thus, bringing diversity into human organizations is even more about creating more effective human organizations than social justice alone. Therefore, it is critical to adjust AI algorithms to remove bias, resulting in maximal collective intelligence, a key facet of human excellence that comes from greater diversity.
Addressing AI through a substantive equality lens
In view of the above, and in support of BLM and other movements to end institutionalized discrimination and victimization, we believe that AI must adhere to the substantive equality model. Targeting disadvantage rather than aiming at neutrality allows us not only to redress the historic and deep-seated legacy of bias and discrimination by leveling the playing field but emphasizes a representation-reinforcing theory of participation.10 Indeed, given that past discrimination and other social mechanisms have perpetually blocked the avenues for political participation by particular minorities, representation-reinforcing equality laws are needed in AI to compensate for the muffling of political voice and to open the channels for greater participation in the future.
But we ought to offer one word of caution. We recognize that there are risks to manipulating AI to ensure a particular outcome or result; we acknowledge that our position may open the pandora’s box for abusive governments to manipulate AI to suit their agendas. However, we argue that these risks can be mitigated if two safeguards are put in place:
1. DIVERSITY OF INTERPRETATIONS: We submit that manipulation must exclusively be focused on addressing the underlying conditions causing bias. This means ensuring respect for diversity of interpretations, rather than retroactively trying to manipulate the end outcome itself. For instance, if AI is to identify leadership qualities in a resume, the algorithm should be trained to identify leadership within a context of pluralism. Leadership, in this case, should not be interpreted through the lenses of a narrow conception of masculinity, but instead look at plural definitions of leadership, including feminist and intersectional views. In other words, AI must adjust for diverse interpretations of a concept or equitable outcome.
2. TRANSPARENCY: We further submit that any manipulation should be fully transparent as well as subject to public governance. We posit that the checks and balances in a functional democracy mitigate most of the risks of foul play. This is analogous to the societal approach to ensuring the voting process in a democracy works; at no stage do we enable any manipulation of say, a government’s voting outcome or outcome based on corporate self-interest; rather, we ensure that the underlying process is fully transparent, free from bias and under public governance.
Dear Justice Ginsburg:I am attaching a very personal letter to the formal invitation. I know that my friend Andreas will make sure you get both.When my father was visiting at Columbia law School in 1977, he asked his much-admired friend Oscar Schachter for the honor of meeting you. When Prof. Schachter asked him why, he said simply, "I have a little daughter." My father brought back to Sri Lanka for me the photo with the three of you. Later in the 1980's he bought Tribe's Constitutional Law treatise, and underlined Frontiero v. Richardson for me, reading excerpts aloud to me on warm nights. When I was a law student in Sri Lanka, I insisted that my Dean Sharya De Soysa include the case along with Marbury v. Madison in our comparative constitutional law lecture. After all, my dean was a Harvard woman too.In very auspicious ways, that winter of 1994 in Joan Williams' class on Feminist Jurisprudence, the first case we read was Harris v. Forklift -- your first case on the Supreme Court and fittingly on sexual harassment in the workplace. I still remember the excitement in our seminar room as we discussed how you would make a difference on the Supreme Court and in the world.I went on to highlight VMI's heightened review standard as well as your early intermediate scrutiny test in Craig v. Boren on sex-based classification in my courses on Women and Comparative Law, which I taught with Judge Nancy Gertner in China. Your work continues to be a leitmotif throughout my life.Twenty-five years after coming to this country and twenty-five years after Harris v. Forklift, I am now at Penn Law. I know that having you with us will make the same difference to our brilliant students that you made in my life.Thank you,RangitaRangita de Silva de AlwisUniversity of Pennsylvania Carey Law SchoolNonresident Leader in Practice at Harvard Kennedy School's Women and Public Policy Program (2019-2021)