IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/nd2y6_v1.html
   My bibliography  Save this paper

Missing Results of Discrimination: A Systematically Comparative Meta-Analysis and Re-Examinations of Racial, Gender, and Intersectional Discrimination using Correspondence Audits

Author

Listed:
  • Gaddis, S. Michael

    (NWEA)

  • Quadlin, Natasha
  • Larsen, Edvard Nergård
  • Crabtree, Charles

    (Dartmouth College)

  • Holbein, John B.

Abstract

Intersectionality theory argues that race and gender cannot be fully understood in isolation or through an additive approach. However, field experiments frequently examine aggregate racial or gender discrimination without accounting for within-category differences (e.g., gendered variations in racial discrimination) or intersectionality. Building on theoretical perspectives of intersectionality, we introduce a systematically comparative analytic framework designed to (a) identify missing results of correspondence audits and (b) provide recommendations to help scholars conduct more holistic analyses. We conduct a meta-analysis of 52 correspondence audits, encompassing nearly 330,000 tests for discrimination, and a re-examination of each study using seven discrimination ratios. The meta-analysis reveals a previously overlooked pattern in rental housing discrimination: compared to White men, Black men experience discrimination, Black women's outcomes are statistically similar, and White women receive preferential treatment. Additionally, our re-examination uncovers ten ways scholars may unintentionally miss key findings when failing to adopt a systematically comparative intersectional approach. We conclude with best-practice recommendations to guide scholars in designing, analyzing, and citing correspondence audits, helping them avoid these problematic scenarios. Importantly, our framework extends beyond field experiments focused on race and gender and can be broadly applied to research on disparities, enabling more comprehensive analyses across numerous dimensions.

Suggested Citation

  • Gaddis, S. Michael & Quadlin, Natasha & Larsen, Edvard Nergård & Crabtree, Charles & Holbein, John B., 2025. "Missing Results of Discrimination: A Systematically Comparative Meta-Analysis and Re-Examinations of Racial, Gender, and Intersectional Discrimination using Correspondence Audits," SocArXiv nd2y6_v1, Center for Open Science.
  • Handle: RePEc:osf:socarx:nd2y6_v1
    DOI: 10.31219/osf.io/nd2y6_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6807d1e565ea5ac98d568ba9/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/nd2y6_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Devah Pager, 2003. "The mark of a criminal record," Natural Field Experiments 00319, The Field Experiments Website.
    2. Phillips, David C., 2017. "Landlords avoid tenants who pay with vouchers," Economics Letters, Elsevier, vol. 151(C), pages 48-52.
    3. Peter Christensen & Christopher Timmins, 2023. "The Damages and Distortions from Discrimination in the Rental Housing Market," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(4), pages 2505-2557.
    4. Conor Lennon, 2021. "How Do Online Degrees Affect Labor Market Prospects? Evidence from a Correspondence Audit Study," ILR Review, Cornell University, ILR School, vol. 74(4), pages 920-947, August.
    5. Jacquemet, Nicolas & Yannelis, Constantine, 2012. "Indiscriminate discrimination: A correspondence test for ethnic homophily in the Chicago labor market," Labour Economics, Elsevier, vol. 19(6), pages 824-832.
    6. Gaddis, S. Michael, 2015. "Discrimination in the Credential Society: An Audit Study of Race and College Selectivity in the Labor Market," SocArXiv 6qjue, Center for Open Science.
    7. Christopher T. Bennett, 2023. "Labor Market Returns to MBAs From Less‐Selective Universities: Evidence From a Field Experiment During COVID‐19," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 525-551, March.
    8. Scheim, Ayden I. & Bauer, Greta R., 2019. "The Intersectional Discrimination Index: Development and validation of measures of self-reported enacted and anticipated discrimination for intercategorical analysis," Social Science & Medicine, Elsevier, vol. 226(C), pages 225-235.
    9. Bauer, Greta R., 2014. "Incorporating intersectionality theory into population health research methodology: Challenges and the potential to advance health equity," Social Science & Medicine, Elsevier, vol. 110(C), pages 10-17.
    10. Michael Ewens & Bryan Tomlin & Liang Choon Wang, 2014. "Statistical Discrimination or Prejudice? A Large Sample Field Experiment," The Review of Economics and Statistics, MIT Press, vol. 96(1), pages 119-134, March.
    11. Devah Pager & Rebecca Goldstein & Helen Ho & Bruce Western, 2022. "Criminalizing Poverty: The Consequences of Court Fees in a Randomized Experiment," American Sociological Review, , vol. 87(3), pages 529-553, June.
    12. Catherine E. Harnois & João L. Bastos & Salma Shariff-Marco, 2022. "Intersectionality, Contextual Specificity, and Everyday Discrimination: Assessing the Difficulty Associated With Identifying a Main Reason for Discrimination Among Racial/Ethnic Minority Respondents," Sociological Methods & Research, , vol. 51(3), pages 983-1013, August.
    13. Hanson, Andrew, 2017. "Do college admissions counselors discriminate? Evidence from a correspondence-based field experiment," Economics of Education Review, Elsevier, vol. 60(C), pages 86-96.
    14. James D. Paul & Albert A. Cheng & Jay P. Greene & Josh B. McGee, 2023. "The Value of College Athletics in the Labor Market: Results from a Resume Audit Field Experiment," Journal of Sports Economics, , vol. 24(3), pages 329-351, April.
    15. repec:osf:socarx:6qjue_v1 is not listed on IDEAS
    16. Sharma, Rajiv & Mitra, Arnab & Stano, Miron, 2015. "Insurance, race/ethnicity, and sex in the search for a new physician," Economics Letters, Elsevier, vol. 137(C), pages 150-153.
    17. Riach Peter A & Rich Judith, 2006. "An Experimental Investigation of Sexual Discrimination in Hiring in the English Labor Market," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 6(2), pages 1-22, January.
    18. Erika L. Kirgios & Aneesh Rai & Edward H. Chang & Katherine L. Milkman, 2022. "When seeking help, women and racial/ethnic minorities benefit from explicitly stating their identity," Nature Human Behaviour, Nature, vol. 6(3), pages 383-391, March.
    19. David C. Phillips, 2020. "Do Low-Wage Employers Discriminate against Applicants with Long Commutes?: Evidence from a Correspondence Experiment," Journal of Human Resources, University of Wisconsin Press, vol. 55(3), pages 864-901.
    20. Peter Christensen & Ignacio Sarmiento-Barbieri & Christopher Timmins, 2022. "Housing Discrimination and the Toxics Exposure Gap in the United States: Evidence from the Rental Market," The Review of Economics and Statistics, MIT Press, vol. 104(4), pages 807-818, October.
    21. Benjamin Edelman & Michael Luca & Dan Svirsky, 2017. "Racial Discrimination in the Sharing Economy: Evidence from a Field Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 1-22, April.
    22. Costa, Mia, 2017. "How Responsive are Political Elites? A Meta-Analysis of Experiments on Public Officials," Journal of Experimental Political Science, Cambridge University Press, vol. 4(3), pages 241-254, December.
    23. Daniel M. Butler & David E. Broockman, 2011. "Do Politicians Racially Discriminate Against Constituents? A Field Experiment on State Legislators," American Journal of Political Science, John Wiley & Sons, vol. 55(3), pages 463-477, July.
    24. Alexandre Flage, 2018. "Ethnic and Gender Discrimination in Rental Housing Market, Evidence from Meta-Analysis of Correspondence Tests, 2006-2017," Working Papers 2018-04, CRESE.
    25. Nunley John M. & Pugh Adam & Romero Nicholas & Seals R. Alan, 2015. "Racial Discrimination in the Labor Market for Recent College Graduates: Evidence from a Field Experiment," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(3), pages 1093-1125, July.
    26. Patrick Kline & Evan K Rose & Christopher R Walters, 2022. "Systemic Discrimination Among Large U.S. Employers [“Teachers and Student Achievement in the Chicago Public High Schools,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(4), pages 1963-2036.
    27. repec:hal:pseose:hal-00745109 is not listed on IDEAS
    28. Birkelund, Gunn Elisabeth & Lancee, Bram & Larsen, Edvard Nergård & Polavieja, Javier G. & Radl, Jonas & Yemane, Ruta, 2022. "Gender Discrimination in Hiring: Evidence from a Cross-National Harmonized Field Experiment," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 38(3), pages 337-354.
    29. Flage, Alexandre, 2018. "Ethnic and gender discrimination in the rental housing market: Evidence from a meta-analysis of correspondence tests, 2006–2017," Journal of Housing Economics, Elsevier, vol. 41(C), pages 251-273.
    30. Katherine Levine Einstein & David M. Glick, 2017. "Does Race Affect Access to Government Services? An Experiment Exploring Street‐Level Bureaucrats and Access to Public Housing," American Journal of Political Science, John Wiley & Sons, vol. 61(1), pages 100-116, January.
    31. Amanda Agan & Sonja Starr, 2018. "Ban the Box, Criminal Records, and Racial Discrimination: A Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 191-235.
    32. Jens Frankenreiter & Michael A. Livermore, 2023. "Are Lawyers’ Case Selection Decisions Biased? A Field Experiment on Access to Justice," The Journal of Legal Studies, University of Chicago Press, vol. 52(2), pages 273-304.
    33. Baker, Rachel & Dee, Thomas & Evans, Brent & John, June, 2022. "Bias in online classes: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 88(C).
    34. Fang, Albert H. & Guess, Andrew M. & Humphreys, Macartan, 2019. "Can the Government Deter Discrimination? Evidence from a Randomized Intervention in New York City," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 81(1), pages 127-141.
    35. Bowleg, L., 2012. "The problem with the phrase women and minorities: Intersectionality-an important theoretical framework for public health," American Journal of Public Health, American Public Health Association, vol. 102(7), pages 1267-1273.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gaddis, S. Michael, 2018. "An Introduction to Audit Studies in the Social Sciences," SocArXiv e5hfc, Center for Open Science.
    2. Gaddis, S. Michael & DiRago, Nicholas V., 2021. "Audit Studies of Housing in the United States: Established, Emerging, and Future Research," SocArXiv fn4ta, Center for Open Science.
    3. repec:osf:socarx:fn4ta_v1 is not listed on IDEAS
    4. repec:osf:socarx:e5hfc_v1 is not listed on IDEAS
    5. Melo, Vitor & Rocha, Hugo Vaca Pereira & Sigaud, Liam & Warren, Patrick L. & Gaddis, S. Michael, 2024. "Understanding Discrimination in College Admissions: A Field Experiment," SocArXiv 5ctms, Center for Open Science.
    6. Button, Patrick & Walker, Brigham, 2020. "Employment discrimination against Indigenous Peoples in the United States: Evidence from a field experiment," Labour Economics, Elsevier, vol. 65(C).
    7. repec:osf:socarx:5ctms_v1 is not listed on IDEAS
    8. Schaerer, Michael & du Plessis, Christilene & Nguyen, My Hoang Bao & van Aert, Robbie C.M. & Tiokhin, Leo & Lakens, Daniël & Giulia Clemente, Elena & Pfeiffer, Thomas & Dreber, Anna & Johannesson, Mag, 2023. "On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 179(C).
    9. Rich, Judy, 2014. "What Do Field Experiments of Discrimination in Markets Tell Us? A Meta Analysis of Studies Conducted since 2000," IZA Discussion Papers 8584, Institute of Labor Economics (IZA).
    10. repec:osf:socarx:9khds_v1 is not listed on IDEAS
    11. Judd B. Kessler & Corinne Low & Colin D. Sullivan, 2019. "Incentivized Resume Rating: Eliciting Employer Preferences without Deception," American Economic Review, American Economic Association, vol. 109(11), pages 3713-3744, November.
    12. Pfaff, Steven & Crabtree, Charles & Kern, Holger L. & Holbein, John B., 2018. "Does religious bias shape access to public services? A large-scale audit experiment among street-level bureaucrats," SocArXiv 9khds, Center for Open Science.
    13. Raffaele Grotti & Helen Russell & Bertrand Maître & Davide Gritti, 2024. "The Experience of Housing Discrimination and Housing Deprivation Across Social Groups in Ireland," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 175(1), pages 195-215, October.
    14. Chen, Ruoyu & Jiang, Hanchen & Quintero, Luis E., 2023. "Measuring the value of rent stabilization and understanding its implications for racial inequality: Evidence from New York City," Regional Science and Urban Economics, Elsevier, vol. 103(C).
    15. Nicolás Ajzenman & Bruno Ferman & Sant’Anna Pedro C., 2023. "Discrimination in the Formation of Academic Networks: A Field Experiment on #EconTwitter," Working Papers 235, Red Nacional de Investigadores en Economía (RedNIE).
    16. Ghekiere, Abel & Verhaeghe, Pieter-Paul, 2022. "How does ethnic discrimination on the housing market differ across neighborhoods and real estate agencies?," Journal of Housing Economics, Elsevier, vol. 55(C).
    17. Marianne Bertrand & Esther Duflo, 2016. "Field Experiments on Discrimination," NBER Working Papers 22014, National Bureau of Economic Research, Inc.
    18. Lepinteur, Anthony & Menta, Giorgia & Waltl, Sofie R., 2025. "Equal price for equal place? Demand-driven racial discrimination in the housing market," Regional Science and Urban Economics, Elsevier, vol. 111(C).
    19. Halla, Martin & Kah, Christopher & Sausgruber, Rupert, 2021. "Testing for Ethnic Discrimination in Outpatient Health Care: Evidence from a Field Experiment in Germany," Department of Economics Working Paper Series 319, WU Vienna University of Economics and Business.
    20. Auspurg, Katrin & Schneck, Andreas & Thiel, Fabian, 2020. "Different samples, different results? How sampling techniques affect the results of field experiments on ethnic discrimination," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 0.
    21. Andrew Hanson & Zackary Hawley, 2023. "Restricted access: Real estate agent response to client race, ethnicity, gender, and side of market," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 51(4), pages 855-890, July.
    22. Bunel, Mathieu & Tovar, Elisabeth, 2024. "Fairness of the first-come, first-served rule on the rental housing market: Insights from a hypothetical survey experiment," Journal of Housing Economics, Elsevier, vol. 65(C).
    23. Yulia Evsyukova & Felix Rusche & Wladislaw Mill, 2025. "LinkedOut? A Field Experiment on Discrimination in Job Network Formation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 140(1), pages 283-334.
    24. Gouveia, Filipe & Nilsson, Therese & Berggren, Niclas, 2020. "Religiosity and discrimination against same-sex couples: The case of Portugal's rental market," Journal of Housing Economics, Elsevier, vol. 50(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:nd2y6_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://arabixiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.