IDEAS home Printed from https://ideas.repec.org/p/cdl/indrel/qt3z72m9kn.html
   My bibliography  Save this paper

Audits as Evidence: Experiments, Ensembles, and Enforcement

Author

Listed:
  • Kline, Patrick
  • Walters, Christopher

Abstract

We develop tools for utilizing correspondence experiments to detect illegal discrimination by individual employers. Employers violate US employment law if their propensity to contact applicants depends on protected characteristics such as race or sex. We establish identification of higher moments of the causal effects of protected characteristics on callback rates as a function of the number of fictitious applications sent to each job ad. These moments are used to bound the fraction of jobs that illegally discriminate. Applying our results to three experimental datasets, we find evidence of significant employer heterogeneity in discriminatory behavior, with the standard deviation of gaps in job-specific callback probabilities across protected groups averaging roughly twice the mean gap. In a recent experiment manipulating racially distinctive names, we estimate that at least 85% of jobs that contact both of two white applications and neither of two black applications are engaged in illegal discrimination. To assess the tradeoff between type I and II errors presented by these patterns, we consider the performance of a series of decision rules for investigating suspicious callback behavior under a simple two-type model that rationalizes the experimental data. Though, in our preferred specification, only 17% of employers are estimated to discriminate on the basis of race, we find that an experiment sending 10 applications to each job would enable accurate detection of 7-10% of discriminators while falsely accusing fewer than 0.2% of non-discriminators. A minimax decision rule acknowledging partial identification of the joint distribution of callback rates yields higher error rates but more investigations than our baseline two-type model. Our results suggest illegal labor market discrimination can be reliably monitored with relatively small modifications to existing audit designs.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Kline, Patrick & Walters, Christopher, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Institute for Research on Labor and Employment, Working Paper Series qt3z72m9kn, Institute of Industrial Relations, UC Berkeley.
  • Handle: RePEc:cdl:indrel:qt3z72m9kn
    as

    Download full text from publisher

    File URL: https://www.escholarship.org/uc/item/3z72m9kn.pdf;origin=repeccitec
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Angrist, Joshua D., 2014. "The perils of peer effects," Labour Economics, Elsevier, vol. 30(C), pages 98-108.
    2. Roland G. Fryer & Steven D. Levitt, 2004. "The Causes and Consequences of Distinctively Black Names," The Quarterly Journal of Economics, Oxford University Press, vol. 119(3), pages 767-805.
    3. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    4. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Leveraging Lotteries for School Value-Added: Testing and Estimation," The Quarterly Journal of Economics, Oxford University Press, vol. 132(2), pages 871-919.
    5. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, Oxford University Press, vol. 133(3), pages 1163-1228.
    6. Eva O. Arceo-Gomez & Raymundo M. Campos-Vazquez, 2014. "Race and Marriage in the Labor Market: A Discrimination Correspondence Study in a Developing Country," American Economic Review, American Economic Association, vol. 104(5), pages 376-380, May.
    7. Joshua D. Angrist & Peter D. Hull & Parag A. Pathak & Christopher R. Walters, 2017. "Erratum to “Leveraging Lotteries for School Value-Added: Testing and Estimation”," The Quarterly Journal of Economics, Oxford University Press, vol. 132(4), pages 2061-2062.
    8. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    9. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    10. Thomas J. Kane & Douglas O. Staiger, 2008. "Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation," NBER Working Papers 14607, National Bureau of Economic Research, Inc.
    11. Timothy B. Armstrong, 2014. "Adaptive Testing on a Regression Function at a Point," Cowles Foundation Discussion Papers 1957R, Cowles Foundation for Research in Economics, Yale University, revised Feb 2015.
    12. Jonathan Guryan & Kerwin Kofi Charles, 2013. "Taste‐based or Statistical Discrimination: The Economics of Discrimination Returns to its Roots," Economic Journal, Royal Economic Society, vol. 123(11), pages 417-432, November.
    13. Yusuke Narita, 2019. "Experiment-as-Market: Incorporating Welfare into Randomized Controlled Trials," Working Papers 2019-025, Human Capital and Economic Opportunity Working Group.
    14. Victor Chernozhukov & Whitney K. Newey & Andres Santos, 2015. "Constrained conditional moment restriction models," CeMMAP working papers CWP59/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    15. Raj Chetty & John N. Friedman & Nathaniel Hendren & Maggie R. Jones & Sonya R. Porter, 2018. "The Opportunity Atlas: Mapping the Childhood Roots of Social Mobility," NBER Working Papers 25147, National Bureau of Economic Research, Inc.
    16. Nunley John M. & Pugh Adam & Romero Nicholas & Seals R. Alan, 2015. "Racial Discrimination in the Labor Market for Recent College Graduates: Evidence from a Field Experiment," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 15(3), pages 1093-1125, July.
    17. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    18. Müller, Ulrich K. & Wang, Yulong, 2019. "Nearly weighted risk minimal unbiased estimation," Journal of Econometrics, Elsevier, vol. 209(1), pages 18-34.
    19. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, January.
    20. Henry S. Farber & Dan Silverman & Till von Wachter, 2016. "Determinants of Callbacks to Job Applications: An Audit Study," American Economic Review, American Economic Association, vol. 106(5), pages 314-318, May.
    21. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Cass R. Sunstein, 2019. "Discrimination In The Age Of Algorithms," NBER Working Papers 25548, National Bureau of Economic Research, Inc.
    22. Kerwin Kofi Charles & Jonathan Guryan, 2008. "Prejudice and Wages: An Empirical Assessment of Becker's The Economics of Discrimination," Journal of Political Economy, University of Chicago Press, vol. 116(5), pages 773-809, October.
    23. repec:hrv:faseco:30749073 is not listed on IDEAS
    24. Pietro Tebaldi & Alexander Torgovitsky & Hanbin Yang, 2019. "Nonparametric Estimates of Demand in the California Health Insurance Exchange," NBER Working Papers 25827, National Bureau of Economic Research, Inc.
    25. Manski, Charles F., 2000. "Identification problems and decisions under ambiguity: Empirical analysis of treatment response and normative analysis of treatment choice," Journal of Econometrics, Elsevier, vol. 95(2), pages 415-442, April.
    26. Yusuke Narita & Shota Yasui & Kohei Yata, 2018. "Efficient Counterfactual Learning from Bandit Feedback," Cowles Foundation Discussion Papers 2155, Cowles Foundation for Research in Economics, Yale University.
    27. John D. Storey, 2002. "A direct approach to false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 479-498, August.
    28. Efron, Bradley, 2004. "Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 96-104, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carl Lieberman, 2020. "Variation in Racial Disparities in Police Use of Force," Working Papers 639, Princeton University, Department of Economics, Industrial Relations Section..
    2. Peter Christensen & Ignacio Sarmiento-Barbieri & Christopher Timmins, 2020. "Housing Discrimination and the Toxics Exposure Gap in the United States: Evidence from the Rental Market," NBER Working Papers 26805, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hinnerich, Björn Tyrefors & Vlachos, Jonas, 2017. "The impact of upper-secondary voucher school attendance on student achievement. Swedish evidence using external and internal evaluations," Labour Economics, Elsevier, vol. 47(C), pages 1-14.
    2. Andrew McEachin & Allison Atteberry, 2017. "The Impact of Summer Learning Loss on Measures of School Performance," Education Finance and Policy, MIT Press, vol. 12(4), pages 468-491, Fall.
    3. Bertrand, Marianne & Duflo, Esther, 2016. "Field Experiments on Discrimination," CEPR Discussion Papers 11123, C.E.P.R. Discussion Papers.
    4. Baert, Stijn, 2015. "Hiring a Homosexual, Taking a Risk? A Lab Experiment on Employment Discrimination and Risk Aversion," IZA Discussion Papers 9536, Institute of Labor Economics (IZA).
    5. Naven, Matthew, 2019. "Human-Capital Formation During Childhood and Adolescence: Evidence from School Quality and Postsecondary Success in California," MPRA Paper 97716, University Library of Munich, Germany.
    6. Jesse Rothstein & Till von Wachter, 2016. "Social Experiments in the Labor Market," NBER Working Papers 22585, National Bureau of Economic Research, Inc.
    7. Gaddis, S. Michael, 2018. "An Introduction to Audit Studies in the Social Sciences," SocArXiv e5hfc, Center for Open Science.
    8. Naven, Matthew, 2020. "Within-School Heterogeneity in Quality: Do Schools Provide Equal Value Added to All Students?," MPRA Paper 100123, University Library of Munich, Germany.
    9. Drydakis, Nick, 2017. "Measuring labour differences between natives, non-natives, and natives with an ethnic-minority background," Economics Letters, Elsevier, vol. 161(C), pages 27-30.
    10. Chowdhury, Shyamal & Ooi, Evarn & Slonim, Robert, 2017. "Racial discrimination and white first name adoption: a field experiment in the Australian labour market," Working Papers 2017-15, University of Sydney, School of Economics.
    11. Button, Patrick & Walker, Brigham, 2020. "Employment discrimination against Indigenous Peoples in the United States: Evidence from a field experiment," Labour Economics, Elsevier, vol. 65(C).
    12. Morten Størling Hedegaard & Jean-Robert Tyran, 2018. "The Price of Prejudice," American Economic Journal: Applied Economics, American Economic Association, vol. 10(1), pages 40-63, January.
    13. Ingo E. Isphording & Ulf Zölitz, 2020. "The value of a peer," ECON - Working Papers 342, Department of Economics - University of Zurich.
    14. Silvia Angerer & Christian Waibel & Harald Stummer, 2019. "Discrimination in Health Care: A Field Experiment on the Impact of Patients’ Socioeconomic Status on Access to Care," American Journal of Health Economics, MIT Press, vol. 5(4), pages 407-427, Fall.
    15. Siddique, Abu & Vlassopoulos, Michael & Zenou, Yves, 2020. "Market Competition and Discrimination," IZA Discussion Papers 13269, Institute of Labor Economics (IZA).
    16. Adam J. Kapor & Christopher A. Neilson & Seth D. Zimmerman, 2020. "Heterogeneous Beliefs and School Choice Mechanisms," American Economic Review, American Economic Association, vol. 110(5), pages 1274-1315, May.
    17. Corrado Giulietti & Mirco Tonin & Michael Vlassopoulos, 2015. "Racial Discrimination in Local Public Services: A Field Experiment in the US," CESifo Working Paper Series 5537, CESifo.
    18. Aedin Doris & Donal O'Neill & Olive Sweetman, 2019. "Good Schools or Good Students? The Importance of Selectivity for School Rankings," Economics, Finance and Accounting Department Working Paper Series n293-19.pdf, Department of Economics, Finance and Accounting, National University of Ireland - Maynooth.
    19. Patrick M. Kline & Christopher R. Walters, 2020. "Reasonable Doubt: Experimental Detection of Job-Level Employment Discrimination," NBER Working Papers 26861, National Bureau of Economic Research, Inc.
    20. Guido Imbens, 2019. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," NBER Working Papers 26104, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Social and Behavioral Sciences; Gender and Race; Inequality; Public Policy;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cdl:indrel:qt3z72m9kn. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Lisa Schiff). General contact details of provider: http://edirc.repec.org/data/irucbus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.