IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v45y2025i6p690-702.html

The Impact of Machine Learning Mortality Risk Prediction on Clinician Prognostic Accuracy and Decision Support: A Randomized Vignette Study

Author

Listed:
  • Ravi B. Parikh

    (Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA, USA
    Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA, USA)

  • William J. Ferrell

    (Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA, USA)

  • Anthony Girard

    (Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA)

  • Jenna White

    (Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA, USA)

  • Sophia Fang

    (Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ, USA)

  • Justin E. Bekelman

    (Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA, USA)

  • Marilyn M. Schapira

    (Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
    Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA, USA
    Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA, USA)

Abstract

Background Machine learning (ML) algorithms may improve the prognosis for serious illnesses such as cancer, identifying patients who may benefit from earlier palliative care (PC) or advance care planning (ACP). We evaluated the impact of various presentation strategies of a hypothetical ML algorithm on clinician prognostic accuracy and decision making. Methods This was a randomized clinical vignette survey study among medical oncologists who treat metastatic non-small-cell lung cancer (mNSCLC). Between March and June 2023, clinicians were shown 3 vignettes of patients presenting with mNSCLC. The vignettes varied by prognostic risk, as defined from the Lung Cancer Prognostic Index (LCPI). Clinicians estimated life expectancy in months and made recommendations about PC and ACP. Clinicians were then shown the same vignette with a hypothetical survival estimate from a black-box ML algorithm; clinicians were randomized to receive the ML prediction using absolute and/or reference-dependent prognostic estimates. The primary outcome was prognostic accuracy relative to the LCPI. Results Among 51 clinicians with complete responses, the median years in practice was 7 (interquartile range 3.5–19), 14 (27.5%) were female, 23 (45.1%) practiced in a community oncology setting, and baseline accuracy was 54.9% (95% confidence interval [CI] 47.0–62.8) across all vignettes. ML presentation improved accuracy (mean change relative to baseline 20.9%, 95% CI 13.9–27.9, P  

Suggested Citation

  • Ravi B. Parikh & William J. Ferrell & Anthony Girard & Jenna White & Sophia Fang & Justin E. Bekelman & Marilyn M. Schapira, 2025. "The Impact of Machine Learning Mortality Risk Prediction on Clinician Prognostic Accuracy and Decision Support: A Randomized Vignette Study," Medical Decision Making, , vol. 45(6), pages 690-702, August.
    • Handle: RePEc:sae:medema:v:45:y:2025:i:6:p:690-702
    DOI: 10.1177/0272989X251349489
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X251349489
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X251349489?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Riccardo Zanardelli, 2025. "Navigating the safe harbor paradox in human-machine systems," Papers 2509.14057, arXiv.org, revised Jan 2026.
    2. Daniele Caliari & Valentino Dardanoni & Carla Guerriero & Paola Manzini & Marco Mariotti, 2026. "Exploring Choice Errors in Children," Bristol Economics Discussion Papers 26/823, School of Economics, University of Bristol, UK.
    3. Jingyi Cui & Gabriel Dias & Justin Ye, 2025. "Signaling in the Age of AI: Evidence from Cover Letters," Papers 2509.25054, arXiv.org, revised Nov 2025.
    4. Martin Baily & David Byrne & Aidan Kane & Paul Soto, 2025. "Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?," Papers 2505.14588, arXiv.org, revised Sep 2025.
    5. Obidzinski, Marie & Oytana, Yves, 2024. "Artificial intelligence, inattention and liability rules," International Review of Law and Economics, Elsevier, vol. 79(C).
    6. Marie Obidzinski & Yves Oytana, 2025. "Advisory algorithms, automation bias and liability rules," Working Papers hal-05486321, HAL.
    7. Yuhao Fu & Nobuyuki Hanaki, 2024. "Do people rely on ChatGPT more than their peers to detect deepfake news?," ISER Discussion Paper 1233r, Institute of Social and Economic Research, The University of Osaka, revised Dec 2024.
    8. Lingxiao Huang & Wenyang Xiao & Nisheeth K. Vishnoi, 2026. "Delegation and Verification Under AI," Cowles Foundation Discussion Papers 2500, Cowles Foundation for Research in Economics, Yale University.
    9. Lagziel, David & Tsodikovich, Yevgeny, 2025. "Working with AI: An analysis for rational integration," Games and Economic Behavior, Elsevier, vol. 153(C), pages 254-267.
    10. Joshua S. Gans, 2026. "Optimal Use of Preferences in Artificial Intelligence Algorithms," NBER Working Papers 34780, National Bureau of Economic Research, Inc.
    11. Saurabh Amin & Amine Bennouna & Daniel Huttenlocher & Dingwen Kong & Liang Lyu & Asuman Ozdaglar, 2026. "A Bayesian Framework for Human-AI Collaboration: Complementarity and Correlation Neglect," Papers 2602.14331, arXiv.org.
    12. Grimon, Marie-Pascale & Mills, Christopher, 2025. "Better Together? A Field Experiment on Human-Algorithm Interaction in Child Protection," SOFI Working Papers in Labour Economics 2/2025, Stockholm University, Swedish Institute for Social Research.
    13. Andrei Iakovlev & Annie Liang, 2024. "The Value of Context: Human versus Black Box Evaluators," Papers 2402.11157, arXiv.org, revised Jun 2024.
    14. L. Elisa Celis & Lingxiao Huang & Nisheeth K. Vishnoi, 2025. "A Mathematical Framework for AI-Human Integration in Work," Papers 2505.23432, arXiv.org, revised May 2025.
    15. Bouacida, Elias & Foucart, Renaud & Jalloul, Maya, 2025. "When expert advice fails to reduce the productivity gap: Experimental evidence from chess players," Journal of Economic Behavior & Organization, Elsevier, vol. 236(C).
    16. Hanzhe Li & Jin Li & Ye Luo & Xiaowei Zhang, 2024. "AI Persuasion, Bayesian Attribution, and Career Concerns of Decision-Makers," Papers 2410.01114, arXiv.org, revised Feb 2026.
    17. David Almog & Romain Gauriot & Lionel Page & Daniel Martin, 2024. "AI Oversight and Human Mistakes: Evidence from Centre Court," Papers 2401.16754, arXiv.org, revised Feb 2025.
    18. Chen, Qin & Ge, Jinfeng & Xie, Huaqing & Xu, Xingcheng & Yang, Yanqing, 2025. "Large language models at work in China’s labor market," China Economic Review, Elsevier, vol. 92(C).
    19. Felix Chopra & Ingar Haaland, 2023. "Conducting qualitative interviews with AI," CEBI working paper series 23-06, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    20. Foucart, Renaud & Zeng, Fanqi & Wang, Shidong, 2025. "The Social Importance of Being Stubborn When an Organization Meets AI," SocArXiv nfgy3_v1, Center for Open Science.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:45:y:2025:i:6:p:690-702. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.