IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-44906-y.html
   My bibliography  Save this article

Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population

Author

Listed:
  • Siqiong Yao

    (Shanghai Jiao Tong University
    AI Institute Shanghai Jiao Tong University)

  • Fang Dai

    (Shanghai Jiao Tong University)

  • Peng Sun

    (Shanghai Jiao Tong University)

  • Weituo Zhang

    (Shanghai Jiao Tong University School of Medicine)

  • Biyun Qian

    (Shanghai Jiao Tong University School of Medicine)

  • Hui Lu

    (Shanghai Jiao Tong University
    AI Institute Shanghai Jiao Tong University
    NHC Key Laboratory of Medical Embryogenesis and Developmental Molecular Biology & Shanghai Key Laboratory of Embryo and Reproduction Engineering)

Abstract

Artificial Intelligence (AI) models for medical diagnosis often face challenges of generalizability and fairness. We highlighted the algorithmic unfairness in a large thyroid ultrasound dataset with significant diagnostic performance disparities across subgroups linked causally to sample size imbalances. To address this, we introduced the Quasi-Pareto Improvement (QPI) approach and a deep learning implementation (QP-Net) combining multi-task learning and domain adaptation to improve model performance among disadvantaged subgroups without compromising overall population performance. On the thyroid ultrasound dataset, our method significantly mitigated the area under curve (AUC) disparity for three less-prevalent subgroups by 0.213, 0.112, and 0.173 while maintaining the AUC for dominant subgroups; we also further confirmed the generalizability of our approach on two public datasets: the ISIC2019 skin disease dataset and the CheXpert chest radiograph dataset. Here we show the QPI approach to be widely applicable in promoting AI for equitable healthcare outcomes.

Suggested Citation

  • Siqiong Yao & Fang Dai & Peng Sun & Weituo Zhang & Biyun Qian & Hui Lu, 2024. "Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-44906-y
    DOI: 10.1038/s41467-024-44906-y
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-44906-y
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-024-44906-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yan Gao & Yan Cui, 2020. "Deep transfer learning for reducing health care disparities arising from biomedical data inequality," Nature Communications, Nature, vol. 11(1), pages 1-8, December.
    2. Yan Gao & Yan Cui, 2020. "Author Correction: Deep transfer learning for reducing health care disparities arising from biomedical data inequality," Nature Communications, Nature, vol. 11(1), pages 1-1, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hu, Chenxi & Zhang, Jun & Yuan, Hongxia & Gao, Tianlu & Jiang, Huaiguang & Yan, Jing & Wenzhong Gao, David & Wang, Fei-Yue, 2022. "Black swan event small-sample transfer learning (BEST-L) and its case study on electrical power prediction in COVID-19," Applied Energy, Elsevier, vol. 309(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-44906-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.