IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/31422.html
   My bibliography  Save this paper

Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology

Author

Listed:
  • Nikhil Agarwal
  • Alex Moehring
  • Pranav Rajpurkar
  • Tobias Salz

Abstract

Full automation using Artificial Intelligence (AI) predictions may not be optimal if humans can access contextual information. We study human-AI collaboration using an information experiment with professional radiologists. Results show that providing (i) AI predictions does not always improve performance, whereas (ii) contextual information does. Radiologists do not realize the gains from AI assistance because of errors in belief updating – they underweight AI predictions and treat their own information and AI predictions as statistically independent. Unless these mistakes can be corrected, the optimal human-AI collaboration design delegates cases either to humans or to AI, but rarely to AI assisted humans.

Suggested Citation

  • Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:31422
    Note: EH IO PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w31422.pdf
    Download Restriction: Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Felix Chopra & Ingar Haaland, 2023. "Conducting qualitative interviews with AI," CEBI working paper series 23-06, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    2. Marie Obidzinski & Yves Oytana, 2024. "Artificial intelligence, inattention and liability rules," Working Papers 2024-08, CRESE.
    3. Yuhao Fu & Nobuyuki Hanaki, 2024. "Do people rely on ChatGPT more than their peers to detect fake news?," ISER Discussion Paper 1233, Institute of Social and Economic Research, Osaka University.
    4. Andrei Iakovlev & Annie Liang, 2024. "The Value of Context: Human versus Black Box Evaluators," Papers 2402.11157, arXiv.org.
    5. David Almog & Romain Gauriot & Lionel Page & Daniel Martin, 2024. "AI Oversight and Human Mistakes: Evidence from Centre Court," Papers 2401.16754, arXiv.org, revised Feb 2024.
    6. Huelden, Tobias & Jascisens, Vitalijs & Roemheld, Lars & Werner, Tobias, 2024. "Human-machine interactions in pricing: Evidence from two large-scale field experiments," DICE Discussion Papers 412, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    7. Yang, Nanyin & Palma, Marco & Drichoutis, Andreas C., 2023. "Humanization of Virtual Assistants and Delegation Choices," MPRA Paper 119275, University Library of Munich, Germany.
    8. Hannes Ullrich & Michael Allan Ribers, 2023. "Machine predictions and human decisions with variation in payoffs and skill: the case of antibiotic prescribing," Berlin School of Economics Discussion Papers 0027, Berlin School of Economics.

    More about this item

    JEL classification:

    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • D47 - Microeconomics - - Market Structure, Pricing, and Design - - - Market Design
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:31422. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.