IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2405.00910.html
   My bibliography  Save this paper

De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data

Author

Listed:
  • Nicholas Tenev

Abstract

Prediction models can improve efficiency by automating decisions such as the approval of loan applications. However, they may inherit bias against protected groups from the data they are trained on. This paper adds counterfactual (simulated) ethnic bias to real data on mortgage application decisions, and shows that this bias is replicated by a machine learning model (XGBoost) even when ethnicity is not used as a predictive variable. Next, several other de-biasing methods are compared: averaging over prohibited variables, taking the most favorable prediction over prohibited variables (a novel method), and jointly minimizing errors as well as the association between predictions and prohibited variables. De-biasing can recover some of the original decisions, but the results are sensitive to whether the bias is effected through a proxy.

Suggested Citation

  • Nicholas Tenev, 2024. "De-Biasing Models of Biased Decisions: A Comparison of Methods Using Mortgage Application Data," Papers 2405.00910, arXiv.org.
    • Handle: RePEc:arx:papers:2405.00910
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2405.00910
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. J. Aislinn Bohren & Kareem Haggag & Alex Imas & Devin G. Pope, 2019. "Inaccurate Statistical Discrimination: An Identification Problem," PIER Working Paper Archive 19-010, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 17 Jul 2020.
    2. Daniel Aaronson & Daniel Hartley & Bhashkar Mazumder, 2021. "The Effects of the 1930s HOLC "Redlining" Maps," American Economic Journal: Economic Policy, American Economic Association, vol. 13(4), pages 355-392, November.
    3. Laura Blattner & Scott Nelson, 2021. "How Costly is Noise? Data and Disparities in Consumer Credit," Papers 2105.07554, arXiv.org.
    4. Richard Berk & Hoda Heidari & Shahin Jabbari & Michael Kearns & Aaron Roth, 2021. "Fairness in Criminal Justice Risk Assessments: The State of the Art," Sociological Methods & Research, , vol. 50(1), pages 3-44, February.
    5. Paul S. Willen & David Hao Zhang, 2020. "Do Lenders Still Discriminate? A Robust Approach for Assessing Differences in Menus," Working Papers 20-19, Federal Reserve Bank of Boston.
    6. Blattner, Laura & Nelson, Scott, 2021. "How Costly Is Noise? Data and Disparities in Consumer Credit," Research Papers 3978, Stanford University, Graduate School of Business.
    7. Helen F. Ladd, 1998. "Evidence on Discrimination in Mortgage Lending," Journal of Economic Perspectives, American Economic Association, vol. 12(2), pages 41-62, Spring.
    8. Stephen L. Ross & John Yinger, 2002. "The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262182289, April.
    9. Marianne Bertrand & Sendhil Mullainathan, 2004. "Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination," American Economic Review, American Economic Association, vol. 94(4), pages 991-1013, September.
    10. Peter Hull, 2021. "What Marginal Outcome Tests Can Tell Us About Racially Biased Decision-Making," NBER Working Papers 28503, National Bureau of Economic Research, Inc.
    11. Tenev, Nicholas H, 2020. "Social Connections and Racial Wage Inequality," SocArXiv vm82w, Center for Open Science.
    12. Nicolas Grau & Damian Vergara, 2021. "An Observational Implementation of the Outcome Test with an Application to Ethnic Prejudice in Pretrial Detentions," Working Papers wp514, University of Chile, Department of Economics.
    13. Emily Blank & Padma Venkatachalam & Lawrence McNeil & Rodney Green, 2005. "Racial discrimination in mortgage lending in Washington, D.C.: A mixed methods approach," The Review of Black Political Economy, Springer;National Economic Association, vol. 33(2), pages 9-30, December.
    14. Devin G. Pope & Justin R. Sydnor, 2011. "Implementing Anti-discrimination Policies in Statistical Profiling Models," American Economic Journal: Economic Policy, American Economic Association, vol. 3(3), pages 206-231, August.
    15. Neil Bhutta & Aurel Hizmo & Daniel R. Ringo, 2022. "How Much Does Racial Bias Affect Mortgage Lending? Evidence from Human and Algorithmic Credit Decisions," Finance and Economics Discussion Series 2022-067, Board of Governors of the Federal Reserve System (U.S.).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bertrand, Jérémie & Weill, Laurent, 2021. "Do algorithms discriminate against African Americans in lending?," Economic Modelling, Elsevier, vol. 104(C).
    2. Ghent, Andra C. & Hernández-Murillo, Rubén & Owyang, Michael T., 2014. "Differences in subprime loan pricing across races and neighborhoods," Regional Science and Urban Economics, Elsevier, vol. 48(C), pages 199-215.
    3. David Arnold & Will Dobbie & Peter Hull, 2022. "Measuring Racial Discrimination in Bail Decisions," American Economic Review, American Economic Association, vol. 112(9), pages 2992-3038, September.
    4. Hurtado, Agustin & Sakong, Jung, 2022. "The effect of minority bank ownership on minority credit," Working Papers 325, The University of Chicago Booth School of Business, George J. Stigler Center for the Study of the Economy and the State.
    5. Hanson, Andrew & Hawley, Zackary & Martin, Hal & Liu, Bo, 2016. "Discrimination in mortgage lending: Evidence from a correspondence experiment," Journal of Urban Economics, Elsevier, vol. 92(C), pages 48-65.
    6. Sabrina T. Howell & Theresa Kuchler & David Snitkof & Johannes Stroebel & Jun Wong, 2021. "Lender Automation and Racial Disparities in Credit Access," NBER Working Papers 29364, National Bureau of Economic Research, Inc.
    7. Sabrina T. Howell & Theresa Kuchler & David Snitkof & Johannes Stroebel & Jun Wong, 2021. "Racial Disparities in Access to Small Business Credit: Evidence from the Paycheck Protection Program," CESifo Working Paper Series 9345, CESifo.
    8. Andreas Fuster & Paul Goldsmith‐Pinkham & Tarun Ramadorai & Ansgar Walther, 2022. "Predictably Unequal? The Effects of Machine Learning on Credit Markets," Journal of Finance, American Finance Association, vol. 77(1), pages 5-47, February.
    9. Sean Lewis-Faupel & Nicholas Tenev, 2024. "Racial and Ethnic Disparities in Mortgage Lending: New Evidence from Expanded HMDA Data," Papers 2405.00895, arXiv.org.
    10. John J. Donohue III, 2005. "The Law and Economics of Antidiscrimination Law," NBER Working Papers 11631, National Bureau of Economic Research, Inc.
    11. Greenwald, Daniel L. & Howell, Sabrina T. & Li, Cangyuan & Yimfor, Emmanuel, 2024. "Regulatory arbitrage or random errors? Implications of race prediction algorithms in fair lending analysis," Journal of Financial Economics, Elsevier, vol. 157(C).
    12. Ashesh Rambachan, 2022. "Identifying Prediction Mistakes in Observational Data," NBER Chapters, in: Economics of Artificial Intelligence, National Bureau of Economic Research, Inc.
    13. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    14. Manthos D. Delis & Panagiotis Papadopoulos, 2019. "Mortgage Lending Discrimination Across the U.S.: New Methodology and New Evidence," Journal of Financial Services Research, Springer;Western Finance Association, vol. 56(3), pages 341-368, December.
    15. Beck, T.H.L. & Behr, P. & Madestam, A., 2011. "Sex and Credit : Is There a Gender Bias in Microfinance?," Other publications TiSEM 65849ab0-04f2-4dc9-9824-3, Tilburg University, School of Economics and Management.
    16. Vitaly Meursault & Daniel Moulton & Larry Santucci & Nathan Schor, 2022. "One Threshold Doesn’t Fit All: Tailoring Machine Learning Predictions of Consumer Default for Lower-Income Areas," Working Papers 22-39, Federal Reserve Bank of Philadelphia.
    17. Jens Ludwig & Sendhil Mullainathan, 2021. "Fragile Algorithms and Fallible Decision-Makers: Lessons from the Justice System," Journal of Economic Perspectives, American Economic Association, vol. 35(4), pages 71-96, Fall.
    18. Daniel Martin & Philip Marx, 2022. "A Robust Test of Prejudice for Discrimination Experiments," Management Science, INFORMS, vol. 68(6), pages 4527-4536, June.
    19. Barron, Kai & Ditlmann, Ruth & Gehrig, Stefan & Schweighofer-Kodritsch, Sebastian, 2020. "Explicit and implicit belief-based gender discrimination: A hiring experiment," Discussion Papers, Research Unit: Economics of Change SP II 2020-306, WZB Berlin Social Science Center.
    20. Patricia Funjika & Rachel M. Gisselquist, 2020. "Social mobility and inequality between groups," WIDER Working Paper Series wp2020-12, World Institute for Development Economic Research (UNU-WIDER).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2405.00910. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.