IDEAS home Printed from https://ideas.repec.org/p/dpr/wpaper/1233.html
   My bibliography  Save this paper

Do people rely on ChatGPT more than their peers to detect fake news?

Author

Listed:
  • Yuhao Fu
  • Nobuyuki Hanaki

Abstract

In the era of rapidly advancing artificial intelligence (AI), understanding to what extent people rely on generative AI products (AI tools), such as ChatGPT, is crucial. This study experimentally investigates whether people rely more on AI tools than their human peers in assessing the authenticity of misinformation. We quantify participants’ degree of reliance using the weight of reference (WOR) and decompose it into two stages using the activation-integration model. Our results indicate that participants exhibit a higher reliance on ChatGPT than their peers, influenced significantly by the quality of the reference and their prior beliefs. The proportion of real parts did not impact the WOR. In addition, we found that the reference source affects both the activation and integration stages, but the quality of reference only influences the second stage.

Suggested Citation

  • Yuhao Fu & Nobuyuki Hanaki, 2024. "Do people rely on ChatGPT more than their peers to detect fake news?," ISER Discussion Paper 1233, Institute of Social and Economic Research, Osaka University.
    • Handle: RePEc:dpr:wpaper:1233
    as

    Download full text from publisher

    File URL: https://www.iser.osaka-u.ac.jp/library/dp/2024/DP1233.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    2. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    3. Korinek, Anton, 2023. "Language Models and Cognitive Automation for Economic Research," CEPR Discussion Papers 17923, C.E.P.R. Discussion Papers.
    4. Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    5. Harvey, Nigel & Harries, Clare & Fischer, Ilan, 2000. "Using Advice and Assessing Its Quality," Organizational Behavior and Human Decision Processes, Elsevier, vol. 81(2), pages 252-273, March.
    6. Logg, Jennifer M. & Minson, Julia A. & Moore, Don A., 2019. "Algorithm appreciation: People prefer algorithmic to human judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 151(C), pages 90-103.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    2. Daniel Woods & Mustafa Abdallah & Saurabh Bagchi & Shreyas Sundaram & Timothy Cason, 2022. "Network defense and behavioral biases: an experimental study," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 254-286, February.
    3. Bryce McLaughlin & Jann Spiess, 2022. "Algorithmic Assistance with Recommendation-Dependent Preferences," Papers 2208.07626, arXiv.org, revised Jan 2024.
    4. Markus Jung & Mischa Seiter, 2021. "Towards a better understanding on mitigating algorithm aversion in forecasting: an experimental study," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 32(4), pages 495-516, December.
    5. Ekaterina Jussupow & Kai Spohrer & Armin Heinzl & Joshua Gawlitza, 2021. "Augmenting Medical Diagnosis Decisions? An Investigation into Physicians’ Decision-Making Process with Artificial Intelligence," Information Systems Research, INFORMS, vol. 32(3), pages 713-735, September.
    6. repec:cup:judgdm:v:15:y:2020:i:3:p:449-451 is not listed on IDEAS
    7. Kevin Bauer & Andrej Gill, 2024. "Mirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies," Information Systems Research, INFORMS, vol. 35(1), pages 226-248, March.
    8. Mahmud, Hasan & Islam, A.K.M. Najmul & Mitra, Ranjan Kumar, 2023. "What drives managers towards algorithm aversion and how to overcome it? Mitigating the impact of innovation resistance through technology readiness," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    9. Hemesath, Sebastian & Tepe, Markus, 2023. "Framing the approval to test self-driving cars on public roads. The effect of safety and competitiveness on citizens' agreement," Technology in Society, Elsevier, vol. 72(C).
    10. Said Kaawach & Oskar Kowalewski & Oleksandr Talavera, 2023. "Automatic vs Manual Investing: Role of Past Performance," Discussion Papers 23-04, Department of Economics, University of Birmingham.
    11. Awtrey, Eli & Thornley, Nico & Dannals, Jennifer E. & Barnes, Christopher M. & Uhlmann, Eric Luis, 2021. "Distribution neglect in performance evaluations," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 213-227.
    12. Mallory Avery & Andreas Leibbrandt & Joseph Vecci, 2023. "Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech," Monash Economics Working Papers 2023-09, Monash University, Department of Economics.
    13. Martin Adam & Konstantin Roethke & Alexander Benlian, 2023. "Human vs. Automated Sales Agents: How and Why Customer Responses Shift Across Sales Stages," Information Systems Research, INFORMS, vol. 34(3), pages 1148-1168, September.
    14. Wang, Cuicui & Li, Yiyang & Fu, Weizhong & Jin, Jia, 2023. "Whether to trust chatbots: Applying the event-related approach to understand consumers’ emotional experiences in interactions with chatbots in e-commerce," Journal of Retailing and Consumer Services, Elsevier, vol. 73(C).
    15. Ivanov, Dmitry, 2023. "Intelligent digital twin (iDT) for supply chain stress-testing, resilience, and viability," International Journal of Production Economics, Elsevier, vol. 263(C).
    16. Ivanova-Stenzel, Radosveta & Tolksdorf, Michel, 2023. "Measuring Preferences for Algorithms - Are people really algorithm averse after seeing the algorithm perform?," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277692, Verein für Socialpolitik / German Economic Association.
    17. Kevin Bauer & Moritz von Zahn & Oliver Hinz, 2023. "Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing," Information Systems Research, INFORMS, vol. 34(4), pages 1582-1602, December.
    18. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    19. Newman, David T. & Fast, Nathanael J. & Harmon, Derek J., 2020. "When eliminating bias isn’t fair: Algorithmic reductionism and procedural justice in human resource decisions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 160(C), pages 149-167.
    20. Bauer, Kevin & von Zahn, Moritz & Hinz, Oliver, 2022. "Expl(AI)ned: The impact of explainable Artificial Intelligence on cognitive processes," SAFE Working Paper Series 315, Leibniz Institute for Financial Research SAFE, revised 2022.
    21. Mahmud, Hasan & Islam, A.K.M. Najmul & Ahmed, Syed Ishtiaque & Smolander, Kari, 2022. "What influences algorithmic decision-making? A systematic literature review on algorithm aversion," Technological Forecasting and Social Change, Elsevier, vol. 175(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:dpr:wpaper:1233. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/isosujp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.