IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25759.html
   My bibliography  Save this paper

Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation

Author

Listed:
  • Julian Kolev
  • Yuly Fuentes-Medel
  • Fiona Murray

Abstract

For organizations focused on scientific research and innovation, workforce diversity is a key driver of success. Blinded review is an increasingly popular approach to reducing bias and increasing diversity in the selection of people and projects, yet its effectiveness is not fully understood. We explore the impact of blinded review on gender inclusion in a unique setting: innovative research grant proposals submitted to the Gates Foundation from 2008-2017. Despite blinded review, female applicants receive significantly lower scores, which cannot be explained by reviewer characteristics, proposal topics, or ex-ante measures of applicant quality. By contrast, the gender score gap is no longer significant after controlling for text-based measures of proposals’ titles and descriptions. Specifically, we find strong gender differences in the usage of broad and narrow words, suggesting that differing communication styles are a key driver of the gender score gap. Importantly, the text-based measures that predict higher reviewer scores do not also predict higher ex-post innovative performance. Instead, female applicants exhibit a greater response in follow-on scientific output after an accepted proposal, relative to male applicants. Our results reveal that gender differences in writing and communication are a significant contributor to gender disparities in the evaluation of science and innovation.

Suggested Citation

  • Julian Kolev & Yuly Fuentes-Medel & Fiona Murray, 2019. "Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation," NBER Working Papers 25759, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:25759
    Note: PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25759.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Roberto M. Fernandez & Santiago Campero, 2017. "Gender Sorting and the Glass Ceiling in High-Tech Firms," ILR Review, Cornell University, ILR School, vol. 70(1), pages 73-104, January.
    2. Fiona Murray & Leigh Graham, 2007. "Buying science and selling science: gender differences in the market for commercial science," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 16(4), pages 657-689, August.
    3. Hengel, E., 2017. "Publishing while Female. Are women held to higher standards? Evidence from peer review," Cambridge Working Papers in Economics 1753, Faculty of Economics, University of Cambridge.
    4. Astrid Kunze & Amalia R. Miller, 2017. "Women Helping Women? Evidence from Private Sector Data on Workplace Hierarchies," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 769-775, December.
    5. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    6. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    7. Ray Reagans & Ezra W. Zuckerman, 2001. "Networks, Diversity, and Productivity: The Social Capital of Corporate R&D Teams," Organization Science, INFORMS, vol. 12(4), pages 502-517, August.
    8. Lisa D. Cook & Chaleampong Kongcharoen, 2010. "The Idea Gap in Pink and Black," NBER Working Papers 16331, National Bureau of Economic Research, Inc.
    9. Emilio J. Castilla, 2015. "Accounting for the Gap: A Firm Study Manipulating Organizational Accountability and Transparency in Pay Decisions," Organization Science, INFORMS, vol. 26(2), pages 311-333, April.
    10. Goldin, Claudia D. & Rouse, Cecilia, 2000. "Orchestrating Impartiality: The Impact of “Blind†Auditions on Female Musicians," Scholarly Articles 30703974, Harvard University Department of Economics.
    11. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    12. Gerald Marschke & Allison Nunez & Bruce A. Weinberg & Huifeng Yu, 2018. "Last Place? The Intersection of Ethnicity, Gender, and Race in Biomedical Authorship," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 222-227, May.
    13. Cody Cook & Rebecca Diamond & Jonathan V Hall & John A List & Paul Oyer, 2021. "The Gender Earnings Gap in the Gig Economy: Evidence from over a Million Rideshare Drivers [Measuring the Gig Economy: Current Knowledge and Open Issues]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 88(5), pages 2210-2238.
    14. Lerchenmueller, Marc J. & Sorenson, Olav, 2018. "The gender gap in early career transitions in the life sciences," Research Policy, Elsevier, vol. 47(6), pages 1007-1017.
    15. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lawson, Cornelia & Salter, Ammon, 2023. "Exploring the effect of overlapping institutional applications on panel decision-making," Research Policy, Elsevier, vol. 52(9).
    2. Peveri, Julieta & Sangnier, Marc, 2023. "Gender differences in re-contesting decisions: New evidence from French municipal elections," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 574-594.
    3. Dominique Meurs & Patrick A. Puhani, 2021. "Culture as a Hiring Criterion: Systemic Discrimination in a Procedurally Fair Hiring Process," RF Berlin - CReAM Discussion Paper Series 2106, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    4. Ohto Kanninen & Sanni Kiviholma & Tuomo Virkola, "undated". "Anatomy of An Anonymous Hiring Pilot," Working Papers 338, Työn ja talouden tutkimus LABORE, The Labour Institute for Economic Research LABORE.
    5. Magdalena Formanowicz & Marta Witkowska & Weronika Hryniszak & Zuzanna Jakubik & Aleksandra Cisłak, 2023. "Gender bias in special issues: evidence from a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2283-2299, April.
    6. Sabrina T. Howell & Ramana Nanda, 2019. "Networking Frictions in Venture Capital, and the Gender Gap in Entrepreneurship," Harvard Business School Working Papers 19-105, Harvard Business School, revised Oct 2019.
    7. Horbach, Serge P.J.M. & Schneider, Jesper W. & Sainte-Marie, Maxime, 2022. "Ungendered writing: Writing styles are unlikely to account for gender differences in funding rates in the natural and technical sciences," Journal of Informetrics, Elsevier, vol. 16(4).
    8. Mancuso, Raffaele & Rossi-Lamastra, Cristina & Franzoni, Chiara, 2023. "Topic choice, gendered language, and the under-funding of female scholars in mission-oriented research," Research Policy, Elsevier, vol. 52(6).
    9. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    10. Yongchao Ma & Ying Teng & Zhongzhun Deng & Li Liu & Yi Zhang, 2023. "Does writing style affect gender differences in the research performance of articles?: An empirical study of BERT-based textual sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2105-2143, April.
    11. Christine L Exley & Judd B Kessler, 2022. "The Gender Gap in Self-Promotion," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(3), pages 1345-1381.
    12. Danielle Li & Lindsey R. Raymond & Peter Bergman, 2020. "Hiring as Exploration," NBER Working Papers 27736, National Bureau of Economic Research, Inc.
    13. Farré, Lídia & Ortega, Francesc, 2019. "Selecting Talent: Gender Differences in Participation and Success in Competitive Selection Processes," IZA Discussion Papers 12530, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Desai, Pranav, 2021. "Essays in corporate finance and innovation," Other publications TiSEM 1ef5fdc6-9c52-43df-be1a-d, Tilburg University, School of Economics and Management.
    2. Lorenzo Ductor & Sanjeev Goyal & Anja Prummer, 2018. "Gender & Collaboration," Working Papers 856, Queen Mary University of London, School of Economics and Finance.
    3. Committee, Nobel Prize, 2023. "Scientific Background to the Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2023," Nobel Prize in Economics documents 2023-2, Nobel Prize Committee.
    4. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    5. Barron, Kai & Ditlmann, Ruth & Gehrig, Stefan & Schweighofer-Kodritsch, Sebastian, 2020. "Explicit and implicit belief-based gender discrimination: A hiring experiment," Discussion Papers, Research Unit: Economics of Change SP II 2020-306, WZB Berlin Social Science Center.
    6. Chen Liang & Yili Hong & Bin Gu & Jing Peng, 2018. "Gender Wage Gap in Online Gig Economy and Gender Differences in Job Preferences," Working Papers 18-03, NET Institute.
    7. Arceo-Gomez, Eva O. & Campos-Vazquez, Raymundo M., 2019. "Gender stereotypes: The case of MisProfesores.com in Mexico," Economics of Education Review, Elsevier, vol. 72(C), pages 55-65.
    8. Yana Gallen, 2018. "Motherhood and the Gender Productivity Gap," Working Papers 2018-091, Human Capital and Economic Opportunity Working Group.
    9. Pushkar Maitra & Ananta Neelim, 2024. "Discrimination in Developing Countries," Monash Economics Working Papers 2024-03, Monash University, Department of Economics.
    10. Karol Jan Borowiecki & Martin Hørlyk Kristensen & Marc T. Law, 2024. "Where are the Female Composers? Evidence on the Extent and Causes of Gender Inequality in Music History," Working Papers 0252, European Historical Economics Society (EHES).
    11. Laura Hospido & Carlos Sanz, 2021. "Gender Gaps in the Evaluation of Research: Evidence from Submissions to Economics Conferences," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(3), pages 590-618, June.
    12. Merouani, Youssouf & Perrin, Faustine, 2024. "Women Inventors: On the Origins of the Gender Patenting Gap," Lund Papers in Economic History 255, Lund University, Department of Economic History.
    13. Abigail LeBlanc & Stephen Sheppard, 2022. "Women artists: gender, ethnicity, origin and contemporary prices," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 46(3), pages 439-481, September.
    14. Sabrina T. Howell & Ramana Nanda, 2019. "Networking Frictions in Venture Capital, and the Gender Gap in Entrepreneurship," NBER Working Papers 26449, National Bureau of Economic Research, Inc.
    15. Rossello, Giulia & Cowan, Robin & Mairesse, Jacques, 2020. "Ph.D. research output in STEM: the role of gender and race in supervision," MERIT Working Papers 2020-021, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    16. Lerchenmueller, Marc J. & Sorenson, Olav, 2018. "The gender gap in early career transitions in the life sciences," Research Policy, Elsevier, vol. 47(6), pages 1007-1017.
    17. Laura Hospido & Carlos Sanz, 2019. "Gender gaps in the evaluation of research: evidence from submissions to economics conferences (Updated March 2020)," Working Papers 1918, Banco de España, revised Mar 2020.
    18. Dominique Meurs & Patrick A. Puhani, 2021. "Culture as a Hiring Criterion: Systemic Discrimination in a Procedurally Fair Hiring Process," RF Berlin - CReAM Discussion Paper Series 2106, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    19. Hunt, Jennifer & Garant, Jean-Philippe & Herman, Hannah & Munroe, David J., 2012. "Why Don't Women Patent?," IZA Discussion Papers 6886, Institute of Labor Economics (IZA).
    20. Sansone, Dario, 2019. "Pink work: Same-sex marriage, employment and discrimination," Journal of Public Economics, Elsevier, vol. 180(C).

    More about this item

    JEL classification:

    • D70 - Microeconomics - - Analysis of Collective Decision-Making - - - General
    • J16 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Gender; Non-labor Discrimination
    • M14 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - Corporate Culture; Diversity; Social Responsibility
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25759. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.