IDEAS home Printed from https://ideas.repec.org/a/eee/phsmap/v680y2025ics037843712500682x.html

Explainability, fairness and the Simpson’s paradox in credit lending

Author

Listed:
  • Babaei, Golnoosh
  • Giudici, Paolo
  • Neelakantan, Parvati

Abstract

Fairness is a key requirement for artificial intelligence applications. The assessment of fairness is typically based on group based measures, such as statistical parity, which compares the machine learning output for different protected population groups, such as male and females. Although intuitive and simple, statistical parity may be affected by the presence of explanatory variables correlated with the protected variable. To remove this effect, we propose to replace statistical parity with Shapley values, which measures the difference in output specifically due to the protected variable. This allows to check for the presence of Simpson’s paradox, for which a fair model may become unfair when conditioning on the explanatory variables. We apply our proposal to a real-world database that concerns credit lending in the state of New York, containing 157,269 personal lending decisions. The empirical findings show that both logistic regression and random forest models are fair, when all loan applications are considered; but become unfair, when the requested loan amount is high.

Suggested Citation

  • Babaei, Golnoosh & Giudici, Paolo & Neelakantan, Parvati, 2025. "Explainability, fairness and the Simpson’s paradox in credit lending," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 680(C).
    • Handle: RePEc:eee:phsmap:v:680:y:2025:i:c:s037843712500682x
    DOI: 10.1016/j.physa.2025.131030
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S037843712500682X
    Download Restriction: Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000
    File URL: https://libkey.io/10.1016/j.physa.2025.131030?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Violet Xinying Chen & J. N. Hooker, 2023. "A guide to formulating fairness in an optimization model," Annals of Operations Research, Springer, vol. 326(1), pages 581-619, July.
    2. Agarwal, Shivam & Muckley, Cal B. & Neelakantan, Parvati, 2023. "Countering racial discrimination in algorithmic lending: A case for model-agnostic interpretation methods," Economics Letters, Elsevier, vol. 226(C).
    3. Kozodoi, Nikita & Jacob, Johannes & Lessmann, Stefan, 2022. "Fairness in credit scoring: Assessment, implementation and profit implications," European Journal of Operational Research, Elsevier, vol. 297(3), pages 1083-1094.
    4. Nikita Kozodoi & Johannes Jacob & Stefan Lessmann, 2021. "Fairness in Credit Scoring: Assessment, Implementation and Profit Implications," Papers 2103.01907, arXiv.org, revised Jun 2022.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jie Shi & Arno P. J. M. Siebes & Siamak Mehrkanoon, 2023. "TransCORALNet: A Two-Stream Transformer CORAL Networks for Supply Chain Credit Assessment Cold Start," Papers 2311.18749, arXiv.org.
    2. Silvana M. Pesenti & Pietro Millossovich & Andreas Tsanakas, 2023. "Differential Quantile-Based Sensitivity in Discontinuous Models," Papers 2310.06151, arXiv.org, revised Oct 2024.
    3. Topuz, Kazim & Urban, Timothy L. & Yildirim, Mehmet B., 2024. "A Markovian score model for evaluating provider performance for continuity of care—An explainable analytics approach," European Journal of Operational Research, Elsevier, vol. 317(2), pages 341-351.
    4. Anna Langenberg & Shih-Chi Ma & Tatiana Ermakova & Benjamin Fabian, 2023. "Formal Group Fairness and Accuracy in Automated Decision Making," Mathematics, MDPI, vol. 11(8), pages 1-25, April.
    5. Henry Penikas, 2023. "Unaccounted model risk for Basel IRB models deemed acceptable by conventional validation criteria," Risk Management, Palgrave Macmillan, vol. 25(4), pages 1-25, December.
    6. Bogdan Mirea & Giani-Ionel Gradinaru, 2026. "Ethics and bias in AI: a potential challenge to fair economic progress," Romanian Journal of Economics, Institute of National Economy, vol. 62(1(71)), pages 99-110, June.
    7. Xia, Yufei & Han, Zhiyin & Li, Yawen & He, Lingyun, 2025. "Credit scoring model for fintech lending: An integration of large language models and FocalPoly loss," International Journal of Forecasting, Elsevier, vol. 41(3), pages 894-919.
    8. Baesens, Bart & Smedts, Kristien, 2025. "Boosting credit risk models," The British Accounting Review, Elsevier, vol. 57(4).
    9. Lu, Xuefei & Calabrese, Raffaella, 2023. "The Cohort Shapley value to measure fairness in financing small and medium enterprises in the UK," Finance Research Letters, Elsevier, vol. 58(PC).
    10. Li, Zhe & Liang, Shuguang & Pan, Xianyou & Pang, Meng, 2024. "Credit risk prediction based on loan profit: Evidence from Chinese SMEs," Research in International Business and Finance, Elsevier, vol. 67(PA).
    11. Kazim Topuz & Akhilesh Bajaj & Kristof Coussement & Timothy L. Urban, 2025. "Interpretable machine learning and explainable artificial intelligence," Annals of Operations Research, Springer, vol. 347(2), pages 775-782, April.
    12. Maarouf, Abdurahman & Feuerriegel, Stefan & Pröllochs, Nicolas, 2025. "A fused large language model for predicting startup success," European Journal of Operational Research, Elsevier, vol. 322(1), pages 198-214.
    13. Piccialli, Veronica & Romero Morales, Dolores & Salvatore, Cecilia, 2024. "Supervised feature compression based on counterfactual analysis," European Journal of Operational Research, Elsevier, vol. 317(2), pages 273-285.
    14. Zilong Liu & Hongyan Liang, 2025. "Do Fintech Lenders Align Pricing with Risk? Evidence from a Model-Based Assessment of Conforming Mortgages," FinTech, MDPI, vol. 4(2), pages 1-16, June.
    15. Zha, Yong & Wang, Yuting & Li, Quan & Yao, Wenying, 2022. "Credit offering strategy and dynamic pricing in the presence of consumer strategic behavior," European Journal of Operational Research, Elsevier, vol. 303(2), pages 753-766.
    16. Jos'e Pombal & Andr'e F. Cruz & Jo~ao Bravo & Pedro Saleiro & M'ario A. T. Figueiredo & Pedro Bizarro, 2022. "Understanding Unfairness in Fraud Detection through Model and Data Bias Interactions," Papers 2207.06273, arXiv.org.
    17. Dimitrios Nikolaidis & Michalis Doumpos, 2022. "Credit Scoring with Drift Adaptation Using Local Regions of Competence," SN Operations Research Forum, Springer, vol. 3(4), pages 1-28, December.
    18. Sultan Amed & Tanmay Sen & Sayantan Banerjee, 2026. "FSL-BDP: Federated Survival Learning with Bayesian Differential Privacy for Credit Risk Modeling," Papers 2601.11134, arXiv.org.
    19. Schwab, Brandon & Kriebel, Johannes, 2026. "Mitigating adversarial attacks on transformer models in credit scoring," European Journal of Operational Research, Elsevier, vol. 328(1), pages 309-323.
    20. Sullivan Hué, 2022. "GAM(L)A: An econometric model for interpretable machine learning," French Stata Users' Group Meetings 2022 19, Stata Users Group.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:phsmap:v:680:y:2025:i:c:s037843712500682x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/physica-a-statistical-mechpplications/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.