IDEAS home Printed from https://ideas.repec.org/p/qed/wpaper/1510.html
   My bibliography  Save this paper

Can Unbiased Predictive AI Amplify Bias?

Author

Listed:
  • Tanvir Ahmed Khan

Abstract

Predictive AI is increasingly used to guide decisions on agents. I show that even a bias-neutral predictive AI can potentially amplify exogenous (human) bias in settings where the predictive AI represents a cost-adjusted precision gain to unbiased predictions, and the final judgments are made by biased human evaluators. In the absence of perfect and instantaneous belief updating, expected victims of bias become less likely to be saved by randomness under more precise predictions. An increase in aggregate discrimination is possible if this effect dominates. Not accounting for this mechanism may result in AI being unduly blamed for creating bias.

Suggested Citation

  • Tanvir Ahmed Khan, 2023. "Can Unbiased Predictive AI Amplify Bias?," Working Paper 1510, Economics Department, Queen's University.
    • Handle: RePEc:qed:wpaper:1510
    as

    Download full text from publisher

    File URL: https://www.econ.queensu.ca/sites/econ.queensu.ca/files/wpaper/qed_wp_1510.pdf
    File Function: First version 2023
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Arnold & Will Dobbie & Crystal S Yang, 2018. "Racial Bias in Bail Decisions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1885-1932.
    2. Bartlett, Robert & Morse, Adair & Stanton, Richard & Wallace, Nancy, 2022. "Consumer-lending discrimination in the FinTech Era," Journal of Financial Economics, Elsevier, vol. 143(1), pages 30-56.
    3. Lepage, Louis Pierre, 2020. "Endogenous learning and the persistence of employer biases in the labor market," CLEF Working Paper Series 24, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    4. Erik Brynjolfsson & Tom Mitchell & Daniel Rock, 2018. "What Can Machines Learn, and What Does It Mean for Occupations and the Economy?," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 43-47, May.
    5. Mallory Avery & Andreas Leibbrandt & Joseph Vecci, 2023. "Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech," Monash Economics Working Papers 2023-09, Monash University, Department of Economics.
    6. Bo Cowgill, 2019. "Bias and Productivity in Humans and Machines," Upjohn Working Papers 19-309, W.E. Upjohn Institute for Employment Research.
    7. Bohren, Aislinn & Hauser, Daniel, 2017. "Learning with Heterogeneous Misspecified Models: Characterization and Robustness," CEPR Discussion Papers 12036, C.E.P.R. Discussion Papers.
    8. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    9. Bo Cowgill & Fabrizio Dell'Acqua & Samuel Deng & Daniel Hsu & Nakul Verma & Augustin Chaintreau, 2020. "Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics," Papers 2012.02394, arXiv.org.
    10. Anton Korinek, 2023. "Language Models and Cognitive Automation for Economic Research," NBER Working Papers 30957, National Bureau of Economic Research, Inc.
    11. J. Aislinn Bohren & Daniel N. Hauser, 2021. "Learning With Heterogeneous Misspecified Models: Characterization and Robustness," Econometrica, Econometric Society, vol. 89(6), pages 3025-3077, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gagnon-Bartsch, Tristan & Rosato, Antonio, 2022. "Quality is in the eye of the beholder: taste projection in markets with observational learning," MPRA Paper 115426, University Library of Munich, Germany.
    2. Chen, S. & Doerr, S. & Frost, J. & Gambacorta, L. & Shin, H.S., 2023. "The fintech gender gap," Journal of Financial Intermediation, Elsevier, vol. 54(C).
    3. Ing-Haw Cheng & Alice Hsiaw, 2023. "Bayesian Doublespeak," Working Papers 135, Brandeis University, Department of Economics and International Business School.
    4. Gagnon-Bartsch, Tristan & Bushong, Benjamin, 2022. "Learning with misattribution of reference dependence," Journal of Economic Theory, Elsevier, vol. 203(C).
    5. Bharti, Nitin Kumar & Roy, Sutanuka, 2023. "The early origins of judicial stringency in bail decisions: Evidence from early childhood exposure to Hindu-Muslim riots in India," Journal of Public Economics, Elsevier, vol. 221(C).
    6. Fudenberg, Drew & Lanzani, Giacomo, 2023. "Which misspecifications persist?," Theoretical Economics, Econometric Society, vol. 18(3), July.
    7. Mira Frick & Ryota Iijima & Yuhta Ishii, 2018. "Dispersed Behavior and Perceptions in Assortative Societies," Cowles Foundation Discussion Papers 2128, Cowles Foundation for Research in Economics, Yale University.
    8. Jehiel, Philippe & Mohlin, Erik, 2021. "Cycling and Categorical Learning in Decentralized Adverse Selection Economies," Working Papers 2021:11, Lund University, Department of Economics.
    9. Jia, Chengcheng & Wu, Jing Cynthia, 2023. "Average inflation targeting: Time inconsistency and ambiguous communication," Journal of Monetary Economics, Elsevier, vol. 138(C), pages 69-86.
    10. Mira Frick & Ryota Iijima & Yuhta Ishii, 2018. "Dispersed Behavior and Perceptions in Assortative Societies," Cowles Foundation Discussion Papers 2128R3, Cowles Foundation for Research in Economics, Yale University, revised Jun 2022.
    11. Abhijit Banerjee & Olivier Compte, 2022. "Consensus and Disagreement: Information Aggregation under (not so) Naive Learning," NBER Working Papers 29897, National Bureau of Economic Research, Inc.
    12. Tyna Eloundou & Sam Manning & Pamela Mishkin & Daniel Rock, 2023. "GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models," Papers 2303.10130, arXiv.org, revised Aug 2023.
    13. Philippe Jehiel & Erik Mohlin, 2023. "Categorization in Games: A Bias-Variance Perspective," Working Papers halshs-04154272, HAL.
    14. Fudenberg, Drew & Gao, Ying & Pei, Harry, 2022. "A reputation for honesty," Journal of Economic Theory, Elsevier, vol. 204(C).
    15. Manav Raj & Robert Seamans, 2019. "Primer on artificial intelligence and robotics," Journal of Organization Design, Springer;Organizational Design Community, vol. 8(1), pages 1-14, December.
    16. Le Yaouanq, Yves, 2023. "A model of voting with motivated beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 213(C), pages 394-408.
    17. Mira Frick & Ryota Iijima & Yuhta Ishii, 2020. "Belief Convergence under Misspecified Learning: A Martingale Approach," Cowles Foundation Discussion Papers 2235R, Cowles Foundation for Research in Economics, Yale University, revised Mar 2021.
    18. Simon Board & Moritz Meyer‐ter‐Vehn, 2021. "Learning Dynamics in Social Networks," Econometrica, Econometric Society, vol. 89(6), pages 2601-2635, November.
    19. Mira Frick & Ryota Iijima & Yuhta Ishii, 2020. "Belief Convergence under Misspecified Learning: A Martingale Approach," Cowles Foundation Discussion Papers 2235R3, Cowles Foundation for Research in Economics, Yale University, revised Apr 2022.
    20. Ivan A. Canay & Magne Mogstad & Jack Mountjoy, 2020. "On the Use of Outcome Tests for Detecting Bias in Decision Making," NBER Working Papers 27802, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    artificial intelligence; AI; algorithm; human-machine interactions; discrimination; bias; algorithmic bias; financial institutions;
    All these keywords.

    JEL classification:

    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes
    • J15 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Minorities, Races, Indigenous Peoples, and Immigrants; Non-labor Discrimination
    • G2 - Financial Economics - - Financial Institutions and Services

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qed:wpaper:1510. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Babcock (email available below). General contact details of provider: https://edirc.repec.org/data/qedquca.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.