IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25759.html
   My bibliography  Save this paper

Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation

Author

Listed:
  • Julian Kolev
  • Yuly Fuentes-Medel
  • Fiona Murray

Abstract

For organizations focused on scientific research and innovation, workforce diversity is a key driver of success. Blinded review is an increasingly popular approach to reducing bias and increasing diversity in the selection of people and projects, yet its effectiveness is not fully understood. We explore the impact of blinded review on gender inclusion in a unique setting: innovative research grant proposals submitted to the Gates Foundation from 2008-2017. Despite blinded review, female applicants receive significantly lower scores, which cannot be explained by reviewer characteristics, proposal topics, or ex-ante measures of applicant quality. By contrast, the gender score gap is no longer significant after controlling for text-based measures of proposals’ titles and descriptions. Specifically, we find strong gender differences in the usage of broad and narrow words, suggesting that differing communication styles are a key driver of the gender score gap. Importantly, the text-based measures that predict higher reviewer scores do not also predict higher ex-post innovative performance. Instead, female applicants exhibit a greater response in follow-on scientific output after an accepted proposal, relative to male applicants. Our results reveal that gender differences in writing and communication are a significant contributor to gender disparities in the evaluation of science and innovation.

Suggested Citation

  • Julian Kolev & Yuly Fuentes-Medel & Fiona Murray, 2019. "Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation," NBER Working Papers 25759, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:25759
    Note: PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25759.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:hrv:faseco:30703974 is not listed on IDEAS
    2. Roberto M. Fernandez & Santiago Campero, 2017. "Gender Sorting and the Glass Ceiling in High-Tech Firms," ILR Review, Cornell University, ILR School, vol. 70(1), pages 73-104, January.
    3. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    4. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, Oxford University Press, vol. 134(2), pages 647-713.
    5. Fiona Murray & Leigh Graham, 2007. "Buying science and selling science: gender differences in the market for commercial science," Industrial and Corporate Change, Oxford University Press, vol. 16(4), pages 657-689, August.
    6. Hengel, E., 2017. "Publishing while Female. Are women held to higher standards? Evidence from peer review," Cambridge Working Papers in Economics 1753, Faculty of Economics, University of Cambridge.
    7. Astrid Kunze & Amalia R. Miller, 2017. "Women Helping Women? Evidence from Private Sector Data on Workplace Hierarchies," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 769-775, December.
    8. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    9. Lerchenmueller, Marc J. & Sorenson, Olav, 2018. "The gender gap in early career transitions in the life sciences," Research Policy, Elsevier, vol. 47(6), pages 1007-1017.
    10. Kevin J. Boudreau & Eva C. Guinan & Karim R. Lakhani & Christoph Riedl, 2016. "Looking Across and Looking Beyond the Knowledge Frontier: Intellectual Distance, Novelty, and Resource Allocation in Science," Management Science, INFORMS, vol. 62(10), pages 2765-2783, October.
    11. Ray Reagans & Ezra W. Zuckerman, 2001. "Networks, Diversity, and Productivity: The Social Capital of Corporate R&D Teams," Organization Science, INFORMS, vol. 12(4), pages 502-517, August.
    12. Lisa D. Cook & Chaleampong Kongcharoen, 2010. "The Idea Gap in Pink and Black," NBER Working Papers 16331, National Bureau of Economic Research, Inc.
    13. Emilio J. Castilla, 2015. "Accounting for the Gap: A Firm Study Manipulating Organizational Accountability and Transparency in Pay Decisions," Organization Science, INFORMS, vol. 26(2), pages 311-333, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dominique Meurs & Patrick A. Puhani, 2021. "Culture as a Hiring Criterion: Systemic Discrimination in a Procedurally Fair Hiring Process," CReAM Discussion Paper Series 2106, Centre for Research and Analysis of Migration (CReAM), Department of Economics, University College London.
    2. Sabrina T. Howell & Ramana Nanda, 2019. "Networking Frictions in Venture Capital, and the Gender Gap in Entrepreneurship," Harvard Business School Working Papers 19-105, Harvard Business School, revised Oct 2019.
    3. Danielle Li & Lindsey R. Raymond & Peter Bergman, 2020. "Hiring as Exploration," NBER Working Papers 27736, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, Oxford University Press, vol. 134(2), pages 647-713.
    2. Desai, Pranav, 2021. "Essays in corporate finance and innovation," Other publications TiSEM 1ef5fdc6-9c52-43df-be1a-d, Tilburg University, School of Economics and Management.
    3. Rossello, Giulia & Cowan, Robin & Mairesse, Jacques, 2020. "Ph.D. research output in STEM: the role of gender and race in supervision," MERIT Working Papers 2020-021, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    4. Lorenzo Ductor & Sanjeev Goyal & Anja Prummer, 2018. "Gender & Collaboration," Working Papers 856, Queen Mary University of London, School of Economics and Finance.
    5. Hunt, Jennifer & Garant, Jean-Philippe & Herman, Hannah & Munroe, David J., 2012. "Why Don't Women Patent?," IZA Discussion Papers 6886, Institute of Labor Economics (IZA).
    6. Banal-Estañol, Albert & Macho-Stadler, Inés & Pérez-Castrillo, David, 2019. "Evaluation in research funding agencies: Are structurally diverse teams biased against?," Research Policy, Elsevier, vol. 48(7), pages 1823-1840.
    7. Pierre Azoulay & Danielle Li, 2020. "Scientific Grant Funding," NBER Working Papers 26889, National Bureau of Economic Research, Inc.
    8. Edoardo Ferrucci & Francesco Lissoni & Ernest Miguelez, 2020. "Coming from afar and picking a man’s job:Women immigrant inventors in the United States," Working Papers hal-03098102, HAL.
    9. Ke, Qing, 2020. "Technological impact of biomedical research: The role of basicness and novelty," Research Policy, Elsevier, vol. 49(7).
    10. Goodall, Amanda H. & Osterloh, Margit, 2015. "Women Have to Enter the Leadership Race to Win: Using Random Selection to Increase the Supply of Women into Senior Positions," IZA Discussion Papers 9331, Institute of Labor Economics (IZA).
    11. Albert Banal-Estañol & Ines Macho-Stadler & David Pérez-Castrillo, 2016. "Key Success Drivers in Public Research Grants: Funding the Seeds of Radical Innovation in Academia?," CESifo Working Paper Series 5852, CESifo.
    12. Albert Bravo-Biosca, 2020. "Experimental Innovation Policy," Innovation Policy and the Economy, University of Chicago Press, vol. 20(1), pages 191-232.
    13. Barron, Kai & Ditlmann, Ruth & Gehrig, Stefan & Schweighofer-Kodritsch, Sebastian, 2020. "Explicit and implicit belief-based gender discrimination: A hiring experiment," Discussion Papers, Research Unit: Economics of Change SP II 2020-306, WZB Berlin Social Science Center.
    14. Nicolas Carayol, 2016. "The Right Job and the Job Right: Novelty, Impact and Journal Stratification in Science," Post-Print hal-02274661, HAL.
    15. Arceo-Gomez, Eva O. & Campos-Vazquez, Raymundo M., 2019. "Gender stereotypes: The case of MisProfesores.com in Mexico," Economics of Education Review, Elsevier, vol. 72(C), pages 55-65.
    16. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    17. John Van Reenen, 2020. "Innovation and Human Capital Policy," NBER Chapters, in: Innovation and Public Policy, National Bureau of Economic Research, Inc.
    18. Jay Bhattacharya & Mikko Packalen, 2020. "Stagnation and Scientific Incentives," NBER Working Papers 26752, National Bureau of Economic Research, Inc.
    19. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castillo, 2019. "Funding academic research: grant application, partnership, award, and output," Economics Working Papers 1658, Department of Economics and Business, Universitat Pompeu Fabra.
    20. Hunt, Jennifer & Garant, Jean-Philippe & Herman, Hannah & Munroe, David J., 2013. "Why are women underrepresented amongst patentees?," Research Policy, Elsevier, vol. 42(4), pages 831-843.

    More about this item

    JEL classification:

    • D70 - Microeconomics - - Analysis of Collective Decision-Making - - - General
    • J16 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Gender; Non-labor Discrimination
    • M14 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - Corporate Culture; Diversity; Social Responsibility
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25759. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.