IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/29267.html
   My bibliography  Save this paper

Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System

Author

Listed:
  • Jens Ludwig
  • Sendhil Mullainathan

Abstract

Algorithms (in some form) are already widely used in the criminal justice system. We draw lessons from this experience for what is to come for the rest of society as machine learning diffuses. We find economists and other social scientists have a key role to play in shaping the impact of algorithms, in part through improving the tools used to build them.

Suggested Citation

  • Jens Ludwig & Sendhil Mullainathan, 2021. "Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System," NBER Working Papers 29267, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:29267
    Note: EH LE LS PE PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w29267.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    2. Mark Hoekstra & CarlyWill Sloan, 2022. "Does Race Matter for Police Use of Force? Evidence from 911 Calls," American Economic Review, American Economic Association, vol. 112(3), pages 827-860, March.
    3. Kerwin Kofi Charles & Jonathan Guryan, 2011. "Studying Discrimination: Fundamental Challenges and Recent Progress," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 479-511, September.
    4. Stephen L. Ross & John Yinger, 2002. "The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262182289, December.
    5. Richard A. Berk & Susan B. Sorenson & Geoffrey Barnes, 2016. "Forecasting Domestic Violence: A Machine Learning Approach to Help Inform Arraignment Decisions," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 13(1), pages 94-115, March.
    6. Richard Berk, 2019. "Accuracy and Fairness for Juvenile Justice Risk Assessments," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(1), pages 175-194, March.
    7. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    8. Anthony Heyes & Soodeh Saberian, 2019. "Temperature and Decisions: Evidence from 207,000 Court Cases," American Economic Journal: Applied Economics, American Economic Association, vol. 11(2), pages 238-265, April.
    9. Robert Bartlett & Adair Morse & Richard Stanton & Nancy Wallace, 2019. "Consumer-Lending Discrimination in the FinTech Era," NBER Working Papers 25943, National Bureau of Economic Research, Inc.
    10. Ilyana Kuziemko, 2013. "How should inmates be released from prison? An assessment of parole versus fixed-sentence regimes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(1), pages 371-424.
    11. Chen, Daniel L. & Moskowitz, Tobias J. & Shue, Kelly, 2016. "Decision-Making Under the Gambler’s Fallacy: Evidence From Asylum Courts, Loan Officers, and Baseball Umpires," IAST Working Papers 16-43, Institute for Advanced Study in Toulouse (IAST).
    12. Shamena Anwar & Hanming Fang, 2015. "Testing for Racial Prejudice in the Parole Board Release Process: Theory and Evidence," The Journal of Legal Studies, University of Chicago Press, vol. 44(1), pages 1-37.
    13. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    14. G. O. Mohler & M. B. Short & Sean Malinowski & Mark Johnson & G. E. Tita & Andrea L. Bertozzi & P. J. Brantingham, 2015. "Randomized Controlled Field Trials of Predictive Policing," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1399-1411, December.
    15. Stéphane Mechoulan & Nicolas Sahuguet, 2015. "Assessing Racial Disparities in Parole Release," The Journal of Legal Studies, University of Chicago Press, vol. 44(1), pages 39-74.
    16. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    17. Roland G. Fryer, 2020. "An Empirical Analysis of Racial Differences in Police Use of Force: A Response," Journal of Political Economy, University of Chicago Press, vol. 128(10), pages 4003-4008.
    18. Daniel L. Chen & Tobias J. Moskowitz & Kelly Shue, 2016. "Decision Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1181-1242.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Douglas Kiarelly Godoy de Araujo, 2023. "gingado: a machine learning library focused on economics and finance," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: applications and tools, volume 59, Bank for International Settlements.
    2. Runshan Fu & Ginger Zhe Jin & Meng Liu, 2022. "Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate," NBER Working Papers 29880, National Bureau of Economic Research, Inc.
    3. Brendan O'Flaherty & Rajiv Sethi & Morgan Williams, 2024. "The nature, detection, and avoidance of harmful discrimination in criminal justice," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 43(1), pages 289-320, January.
    4. S. Mills & S. Costa & C. R. Sunstein, 2023. "AI, Behavioural Science, and Consumer Welfare," Journal of Consumer Policy, Springer, vol. 46(3), pages 387-400, September.
    5. Brendon McConnell & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "How do Parole Boards Respond to Large, Societal Shocks? Evidence from the 9/11 Terrorist Attacks," Working Papers 2023-010, Human Capital and Economic Opportunity Working Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yulin Liu & Luyao Zhang, 2022. "Cryptocurrency Valuation: An Explainable AI Approach," Papers 2201.12893, arXiv.org, revised Jul 2023.
    2. Meier, Armando N. & Levav, Jonathan & Meier, Stephan, 2020. "Early Release and Recidivism," IZA Discussion Papers 13035, Institute of Labor Economics (IZA).
    3. Julia Godfrey & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "The Effect of Parole Board Racial Composition on Prisoner Outcomes," Working Papers 2023-011, Human Capital and Economic Opportunity Working Group.
    4. Eric Fesselmeyer, 2021. "The impact of temperature on labor quality: Umpire accuracy in Major League Baseball," Southern Economic Journal, John Wiley & Sons, vol. 88(2), pages 545-567, October.
    5. Andreas Fuster & Paul Goldsmith‐Pinkham & Tarun Ramadorai & Ansgar Walther, 2022. "Predictably Unequal? The Effects of Machine Learning on Credit Markets," Journal of Finance, American Finance Association, vol. 77(1), pages 5-47, February.
    6. Jonas Radbruch & Amelie Schiprowski, 2020. "Interview Sequences and the Formation of Subjective Assessments," ECONtribute Discussion Papers Series 045, University of Bonn and University of Cologne, Germany.
    7. Brendon McConnell & Kegon Teng Kok Tan & Mariyana Zapryanova, 2023. "How do Parole Boards Respond to Large, Societal Shocks? Evidence from the 9/11 Terrorist Attacks," Working Papers 2023-010, Human Capital and Economic Opportunity Working Group.
    8. Jessica LaVoice & Domonkos F. Vamossy, 2019. "Racial Disparities in Debt Collection," Papers 1910.02570, arXiv.org, revised Jun 2023.
    9. Miao Liu, 2022. "Assessing Human Information Processing in Lending Decisions: A Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 60(2), pages 607-651, May.
    10. Reio Tanji, 2022. "Pitch Call Discrimination in Major League Baseball: The Effect on the Observed Performance and the Salaries," Discussion Papers in Economics and Business 22-02, Osaka University, Graduate School of Economics.
    11. Guido de Blasio & Alessio D'Ignazio & Marco Letta, 2020. "Predicting Corruption Crimes with Machine Learning. A Study for the Italian Municipalities," Working Papers 16/20, Sapienza University of Rome, DISS.
    12. Ewens, Michael, 2022. "Race and Gender in Entrepreneurial Finance," SocArXiv djf8z, Center for Open Science.
    13. Chen, Daniel L. & Philippe, Arnaud, 2023. "Clash of norms judicial leniency on defendant birthdays," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 324-344.
    14. Anthony Edo & Nicolas Jacquemet & Constantine Yannelis, 2019. "Language skills and homophilous hiring discrimination: Evidence from gender and racially differentiated applications," Review of Economics of the Household, Springer, vol. 17(1), pages 349-376, March.
    15. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    16. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2017. "Econom\'etrie et Machine Learning," Papers 1708.06992, arXiv.org, revised Mar 2018.
    17. Erik Heilmann & Janosch Henze & Heike Wetzel, 2021. "Machine learning in energy forecasts with an application to high frequency electricity consumption data," MAGKS Papers on Economics 202135, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    18. repec:wyi:journl:002164 is not listed on IDEAS
    19. Maria R. Ibanez & Michael W. Toffel, 2020. "How Scheduling Can Bias Quality Assessment: Evidence from Food-Safety Inspections," Management Science, INFORMS, vol. 66(6), pages 2396-2416, June.
    20. Manuel J. García Rodríguez & Vicente Rodríguez Montequín & Francisco Ortega Fernández & Joaquín M. Villanueva Balsera, 2019. "Public Procurement Announcements in Spain: Regulations, Data Analysis, and Award Price Estimator Using Machine Learning," Complexity, Hindawi, vol. 2019, pages 1-20, November.
    21. Galdo, Virgilio & Li, Yue & Rama, Martin, 2021. "Identifying urban areas by combining human judgment and machine learning: An application to India," Journal of Urban Economics, Elsevier, vol. 125(C).

    More about this item

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • H0 - Public Economics - - General
    • K0 - Law and Economics - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:29267. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.