IDEAS home Printed from
   My bibliography  Save this paper

Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability


  • Jon Kleinberg
  • Sendhil Mullainathan


Algorithms are increasingly used to aid, or in some cases supplant, human decision-making, particularly for decisions that hinge on predictions. As a result, two additional features in addition to prediction quality have generated interest: (i) to facilitate human interaction and understanding with these algorithms, we desire prediction functions that are in some fashion simple or interpretable; and (ii) because they influence consequential decisions, we also want them to produce equitable allocations. We develop a formal model to explore the relationship between the demands of simplicity and equity. Although the two concepts appear to be motivated by qualitatively distinct goals, we show a fundamental inconsistency between them. Specifically, we formalize a general framework for producing simple prediction functions, and in this framework we establish two basic results. First, every simple prediction function is strictly improvable: there exists a more complex prediction function that is both strictly more efficient and also strictly more equitable. Put another way, using a simple prediction function both reduces utility for disadvantaged groups and reduces overall welfare relative to other options. Second, we show that simple prediction functions necessarily create incentives to use information about individuals' membership in a disadvantaged group—incentives that weren't present before simplification, and that work against these individuals. Thus, simplicity transforms disadvantage into bias against the disadvantaged group. Our results are not only about algorithms but about any process that produces simple models, and as such they connect to the psychology of stereotypes and to an earlier economics literature on statistical discrimination.

Suggested Citation

  • Jon Kleinberg & Sendhil Mullainathan, 2019. "Simplicity Creates Inequity: Implications for Fairness, Stereotypes, and Interpretability," NBER Working Papers 25854, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:25854
    Note: LE LS PE

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Sendhil Mullainathan & Joshua Schwartzstein & Andrei Shleifer, 2008. "Coarse Thinking and Persuasion," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(2), pages 577-619.
    2. Paul R. Milgrom, 1981. "Good News and Bad News: Representation Theorems and Applications," Bell Journal of Economics, The RAND Corporation, vol. 12(2), pages 380-391, Autumn.
    3. Ed Hopkins & Tatiana Kornienko, 2003. "Ratio Orderings and Comparative Statics," Edinburgh School of Economics Discussion Paper Series 91, Edinburgh School of Economics, University of Edinburgh.
    4. Susan Athey, 2002. "Monotone Comparative Statics under Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(1), pages 187-223.
    5. Amanda Agan & Sonja Starr, 2018. "Ban the Box, Criminal Records, and Racial Discrimination: A Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 191-235.
    6. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    7. Jiaming Zeng & Berk Ustun & Cynthia Rudin, 2017. "Interpretable classification models for recidivism prediction," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(3), pages 689-722, June.
    8. Jonah E. Rockoff & Brian A. Jacob & Thomas J. Kane & Douglas O. Staiger, 2011. "Can You Recognize an Effective Teacher When You Recruit One?," Education Finance and Policy, MIT Press, vol. 6(1), pages 43-74, January.
    9. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    10. Phelps, Edmund S, 1972. "The Statistical Theory of Racism and Sexism," American Economic Review, American Economic Association, vol. 62(4), pages 659-661, September.
    11. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ashesh Rambachan, 2018. "Algorithmic Fairness," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 22-27, May.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Runshan Fu & Manmohan Aseri & Param Vir Singh & Kannan Srinivasan, 2022. "“Un”Fair Machine Learning Algorithms," Management Science, INFORMS, vol. 68(6), pages 4173-4195, June.
    2. John W. Patty & Elizabeth Maggie Penn, 2022. "Algorithmic Fairness and Statistical Discrimination," Papers 2208.08341,
    3. Elizabeth Maggie Penn & John W. Patty, 2023. "Algorithms, Incentives, and Democracy," Papers 2307.02319,
    4. Malamud, Semyon & Cieslak, Anna & Schrimpf, Paul, 2021. "Optimal Transport of Information," CEPR Discussion Papers 15859, C.E.P.R. Discussion Papers.
    5. Heng Xu & Nan Zhang, 2022. "Implications of Data Anonymization on the Statistical Evidence of Disparity," Management Science, INFORMS, vol. 68(4), pages 2600-2618, April.
    6. Semyon Malamud & Andreas Schrimpf, 2021. "Persuasion by Dimension Reduction," Swiss Finance Institute Research Paper Series 21-69, Swiss Finance Institute.
    7. Tengyuan Liang & Pragya Sur, 2020. "A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-L1-Norm Interpolated Classifiers," Working Papers 2020-152, Becker Friedman Institute for Research In Economics.
    8. Ashesh Rambachan & Jon Kleinberg & Sendhil Mullainathan & Jens Ludwig, 2020. "An Economic Approach to Regulating Algorithms," NBER Working Papers 27111, National Bureau of Economic Research, Inc.
    9. Claire Lazar Reich, 2021. "The Disparate Impact of Uncertainty: Affirmative Action vs. Affirmative Information," Papers 2102.10019,, revised Feb 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andini, Monica & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Salvestrini, Viola, 2018. "Targeting with machine learning: An application to a tax rebate program in Italy," Journal of Economic Behavior & Organization, Elsevier, vol. 156(C), pages 86-102.
    2. Battiston, Pietro & Gamba, Simona & Santoro, Alessandro, 2024. "Machine learning and the optimization of prediction-based policies," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    3. Andini, Monica & Boldrini, Michela & Ciani, Emanuele & de Blasio, Guido & D'Ignazio, Alessio & Paladini, Andrea, 2022. "Machine learning in the service of policy targeting: The case of public credit guarantees," Journal of Economic Behavior & Organization, Elsevier, vol. 198(C), pages 434-475.
    4. de Blasio, Guido & D'Ignazio, Alessio & Letta, Marco, 2022. "Gotham city. Predicting ‘corrupted’ municipalities with machine learning," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    5. Monica Andini & Emanuele Ciani & Guido de Blasio & Alessio D'Ignazio & Viola Salvestrini, 2017. "Targeting policy-compliers with machine learning: an application to a tax rebate programme in Italy," Temi di discussione (Economic working papers) 1158, Bank of Italy, Economic Research and International Relations Area.
    6. Elliott Ash & Sergio Galletta & Tommaso Giommoni, 2021. "A Machine Learning Approach to Analyze and Support Anti-Corruption Policy," CESifo Working Paper Series 9015, CESifo.
    7. Guido de Blasio & Alessio D'Ignazio & Marco Letta, 2020. "Predicting Corruption Crimes with Machine Learning. A Study for the Italian Municipalities," Working Papers 16/20, Sapienza University of Rome, DISS.
    8. Dessein, Wouter & Frankel, Alexander & Kartik, Navin, 2023. "Test-Optional Admissions," CEPR Discussion Papers 18090, C.E.P.R. Discussion Papers.
    9. Klumpp, Tilman & Su, Xuejuan, 2013. "Second-order statistical discrimination," Journal of Public Economics, Elsevier, vol. 97(C), pages 108-116.
    10. Pierre Fleckinger & Matthieu Glachant & Gabrielle Moineville, 2017. "Incentives for Quality in Friendly and Hostile Informational Environments," American Economic Journal: Microeconomics, American Economic Association, vol. 9(1), pages 242-274, February.
    11. Eduardo Perez & Delphine Prady, 2012. "Complicating to Persuade?," Working Papers hal-03583827, HAL.
    12. Haisken-DeNew, John & Hasan, Syed & Jha, Nikhil & Sinning, Mathias, 2018. "Unawareness and selective disclosure: The effect of school quality information on property prices," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 449-464.
    13. Anthony M. Marino, 2020. "Banning information in hiring decisions," Journal of Regulatory Economics, Springer, vol. 58(1), pages 33-58, August.
    14. Philippe Jehiel, 2022. "Analogy-Based Expectation Equilibrium and Related Concepts:Theory, Applications, and Beyond," PSE Working Papers halshs-03735680, HAL.
    15. Shan Huang & Michael Allan Ribers & Hannes Ullrich, 2021. "The Value of Data for Prediction Policy Problems: Evidence from Antibiotic Prescribing," Discussion Papers of DIW Berlin 1939, DIW Berlin, German Institute for Economic Research.
    16. Delogu, Marco & Lagravinese, Raffaele & Paolini, Dimitri & Resce, Giuliano, 2024. "Predicting dropout from higher education: Evidence from Italy," Economic Modelling, Elsevier, vol. 130(C).
    17. Emir Kamenica & Matthew Gentzkow, 2011. "Bayesian Persuasion," American Economic Review, American Economic Association, vol. 101(6), pages 2590-2615, October.
    18. Eliaz, Kfir & Spiegler, Ran & Thysen, Heidi C., 2021. "Persuasion with endogenous misspecified beliefs," LSE Research Online Documents on Economics 109842, London School of Economics and Political Science, LSE Library.
    19. Jeanne Hagenbach & Frédéric Koessler & Eduardo Perez‐Richet, 2014. "Certifiable Pre‐Play Communication: Full Disclosure," Econometrica, Econometric Society, vol. 82(3), pages 1093-1131, May.
    20. Arnaud Costinot, 2009. "An Elementary Theory of Comparative Advantage," Econometrica, Econometric Society, vol. 77(4), pages 1165-1192, July.

    More about this item

    JEL classification:

    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • D8 - Microeconomics - - Information, Knowledge, and Uncertainty
    • I30 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - General
    • J7 - Labor and Demographic Economics - - Labor Discrimination
    • K00 - Law and Economics - - General - - - General (including Data Sources and Description)

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25854. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.