IDEAS home Printed from https://ideas.repec.org/p/feb/artefa/00728.html

What Do Blind Evaluations Reveal? How Discrimination Shapes Representation and Quality

Author

Listed:
  • Haruka Uchida

Abstract

Concealing candidate identities during evaluations ("blinding") is often proposed to combat discrimination, yet its effects on the composition and quality of selected candidates, as well as its underlying mechanisms, remain unclear. I conduct a field experiment at an international academic conference, randomly assigning all 657 submitted papers to two blind and two non-blind reviewers (245 total) and collecting paper quality measures---citations and publication statuses five years later. I find that blinding significantly shrinks gaps in reviewer scores and acceptances by student status and institution rank, with no significant effects by gender. These increases in representation are not at the expense of quality: papers selected under blind review are of comparable quality to those selected non-blind. To understand mechanisms, I run a second field experiment that again implements blind and non-blind review, and elicits reviewer predictions of future submission outcomes. I combine my experiments to estimate a model of reviewer scores that uses blind scores to decompose non-blind disparities into distinct forms of discrimination. I find that the nature of discrimination differs by trait: student score gaps are explained by inaccurate beliefs about paper quality (inaccurate statistical discrimination) and alternative objectives (such as favoring authors whose acceptance benefits others), while institution gaps are attributable to residual drivers of discrimination such as animus.

Suggested Citation

  • Haruka Uchida, 2023. "What Do Blind Evaluations Reveal? How Discrimination Shapes Representation and Quality," Artefactual Field Experiments 00728, The Field Experiments Website.
  • Handle: RePEc:feb:artefa:00728
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00728.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Goldin, Claudia D. & Rouse, Cecilia, 2000. "Orchestrating Impartiality: The Impact of “Blind†Auditions on Female Musicians," Scholarly Articles 30703974, Harvard University Department of Economics.
    2. David Card & Stefano DellaVigna & Patricia Funk & Nagore Iriberri, 2020. "Are Referees and Editors in Economics Gender Neutral?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 135(1), pages 269-327.
    3. Annabelle Krause & Ulf Rinne & Klaus Zimmermann, 2012. "Anonymous job applications in Europe," IZA Journal of European Labor Studies, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 1(1), pages 1-20, December.
    4. John A. List, 2020. "Non est Disputandum de Generalizability? A Glimpse into The External Validity Trial," NBER Working Papers 27535, National Bureau of Economic Research, Inc.
    5. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    6. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    7. Steven D. Levitt & John A. List, 2007. "What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?," Journal of Economic Perspectives, American Economic Association, vol. 21(2), pages 153-174, Spring.
    8. Olof Åslund & Oskar Nordströum Skans, 2012. "Do Anonymous Job Application Procedures Level the Playing Field?," ILR Review, Cornell University, ILR School, vol. 65(1), pages 82-107, January.
    9. Blank, Rebecca M, 1991. "The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review," American Economic Review, American Economic Association, vol. 81(5), pages 1041-1067, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zanoni, Wladimir & Duryea, Suzanne & Paredes, Jorge, 2024. "Exploring Gender Discrimination: A Multi-Trial Field Experiment in Urban Ecuador," IDB Publications (Working Papers) 13705, Inter-American Development Bank.
    2. Laura Hospido & Carlos Sanz, 2021. "Gender Gaps in the Evaluation of Research: Evidence from Submissions to Economics Conferences," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(3), pages 590-618, June.
    3. Ohto Kanninen & Sanni Kiviholma & Tuomo Virkola, 2023. "Anatomy of An Anonymous Hiring Pilot," Working Papers 338, Työn ja talouden tutkimus LABORE, The Labour Institute for Economic Research LABORE.
    4. Filip Matejka, 2013. "Attention Discrimination: Theory and Field Experiments," 2013 Meeting Papers 798, Society for Economic Dynamics.
    5. Hinnerich, Björn Tyrefors & Höglin, Erik & Johannesson, Magnus, 2011. "Ethnic Discrimination in High School Grading: Evidence from a Field Experiment," SSE/EFI Working Paper Series in Economics and Finance 733, Stockholm School of Economics, revised 27 Jun 2011.
    6. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    7. Andreas Leibbrandt & John A. List, 2018. "Do Equal Employment Opportunity Statements Backfire? Evidence From A Natural Field Experiment On Job-Entry Decisions," NBER Working Papers 25035, National Bureau of Economic Research, Inc.
    8. Marianne Bertrand & Esther Duflo, 2016. "Field Experiments on Discrimination," NBER Working Papers 22014, National Bureau of Economic Research, Inc.
    9. List John A., 2007. "Field Experiments: A Bridge between Lab and Naturally Occurring Data," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 6(2), pages 1-47, April.
    10. Lavy, Victor & Sand, Edith, 2018. "On the origins of gender gaps in human capital: Short- and long-term consequences of teachers' biases," Journal of Public Economics, Elsevier, vol. 167(C), pages 263-279.
    11. Fredrik Carlsson & Åsa Löfgren & Thomas Sterner, 2012. "Discrimination in Scientific Review: A Natural Field Experiment on Blind versus Non‐Blind Reviews," Scandinavian Journal of Economics, Wiley Blackwell, vol. 114(2), pages 500-519, June.
    12. Nikoloz Kudashvili & Philipp Lergetporer, 2019. "Do Minorities Misrepresent Their Ethnicity to Avoid Discrimination?," CESifo Working Paper Series 7861, CESifo.
    13. Kai Barron & Ruth Ditlmann & Stefan Gehrig & Sebastian Schweighofer-Kodritsch, 2025. "Explicit and Implicit Belief-Based Gender Discrimination: A Hiring Experiment," Management Science, INFORMS, vol. 71(2), pages 1600-1622, February.
    14. Sara Ayllón & Lars J. Lefgren & Richard W. Patterson & Olga B. Stoddard & Nicolás Urdaneta Andrade, 2025. "‘Sorting’ Out Gender Discrimination and Disadvantage: Evidence from Student Evaluations of Teaching," NBER Working Papers 33911, National Bureau of Economic Research, Inc.
    15. Victor Lavy & Rigissa Megalokonomou, 2019. "Persistency in Teachers’ Grading Bias and Effects on Longer-Term Outcomes: University Admissions Exams and Choice of Field of Study," NBER Working Papers 26021, National Bureau of Economic Research, Inc.
    16. Marciano Siniscalchi & Pietro Veronesi, 2020. "Self-image Bias and Lost Talent," NBER Working Papers 28308, National Bureau of Economic Research, Inc.
    17. Victor Lavy, 2013. "Gender Differences in Market Competitiveness in a Real Workplace: Evidence from Performance‐based Pay Tournaments among Teachers," Economic Journal, Royal Economic Society, vol. 123(569), pages 540-573, June.
    18. Bøg, Martin & Kranendonk, Erik, 2011. "Labor market discrimination of minorities? yes, but not in job offers," MPRA Paper 33332, University Library of Munich, Germany.
    19. Hamish Low & Luigi Pistaferri, 2019. "Disability Insurance: Error Rates and Gender Differences," Economics Series Working Papers 889, University of Oxford, Department of Economics.
    20. Menzel, Andreas & Woodruff, Christopher, 2021. "Gender wage gaps and worker mobility: Evidence from the garment sector in Bangladesh," Labour Economics, Elsevier, vol. 71(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:artefa:00728. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Franco Daniel Albino (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.