IDEAS home Printed from https://ideas.repec.org/p/dpr/wpaper/1233r.html
   My bibliography  Save this paper

Do people rely on ChatGPT more than their peers to detect deepfake news?

Author

Listed:
  • Yuhao Fu
  • Nobuyuki Hanaki

Abstract

This experimental study investigates whether people rely more on ChatGPT (GPT-4) than on their human peers when detecting AI-generated fake news (deepfake news). In multiple rounds of deepfake detection tasks conducted in a laboratory setting, student participants exhibited a greater reliance on ChatGPT compared to their peers. We explored this over-reliance on AI from two perspectives: the weight of advice (WOA) and the decomposition of reliance (DOR) into two stages. Our analysis indicates that reliance on external advice is primarily influenced by the source and quality of the advice, as well as the subjects’ prior beliefs, knowledge, and experience, while the type of news and time spent on tasks have no effect. Additionally, our study indicates a potential sequential mechanism of advice utilization, wherein the advice source affects reliance in both stages—activation and integration—whereas the quality of the advice, along with knowledge and experience, influences only the second stage. Our findings suggest that relying on AI to detect AI may not be detrimental and could, in fact, contribute to a deeper understanding of human-AI interaction and support advancements in AI development during the Generative Artificial Intelligence (GAI) era.

Suggested Citation

  • Yuhao Fu & Nobuyuki Hanaki, 2024. "Do people rely on ChatGPT more than their peers to detect deepfake news?," ISER Discussion Paper 1233r, Institute of Social and Economic Research, The University of Osaka, revised Dec 2024.
  • Handle: RePEc:dpr:wpaper:1233r
    as

    Download full text from publisher

    File URL: https://www.iser.osaka-u.ac.jp/static/resources/docs/dp/DP1233R.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    2. Christopher Whyte, 2020. "Deepfake news: AI-enabled disinformation as a multi-level public policy challenge," Journal of Cyber Policy, Taylor & Francis Journals, vol. 5(2), pages 199-217, May.
    3. Tse, Tiffany Tsz Kwan & Hanaki, Nobuyuki & Mao, Bolin, 2024. "Beware the performance of an algorithm before relying on it: Evidence from a stock price forecasting experiment," Journal of Economic Psychology, Elsevier, vol. 102(C).
    4. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    5. Mesbah, Neda & Tauchert, Christoph & Buxmann, Peter, 2021. "Whose Advice Counts More – Man or Machine? An Experimental Investigation of AI-based Advice Utilization," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 124796, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    6. Korinek, Anton, 2023. "Language Models and Cognitive Automation for Economic Research," CEPR Discussion Papers 17923, C.E.P.R. Discussion Papers.
    7. Nikhil Agarwal & Alex Moehring & Pranav Rajpurkar & Tobias Salz, 2023. "Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology," NBER Working Papers 31422, National Bureau of Economic Research, Inc.
    8. Margarita Leib & Nils Köbis & Rainer Michael Rilke & Marloes Hagens & Bernd Irlenbusch, 2024. "Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty," The Economic Journal, Royal Economic Society, vol. 134(658), pages 766-784.
    9. Harvey, Nigel & Harries, Clare & Fischer, Ilan, 2000. "Using Advice and Assessing Its Quality," Organizational Behavior and Human Decision Processes, Elsevier, vol. 81(2), pages 252-273, March.
    10. Logg, Jennifer M. & Minson, Julia A. & Moore, Don A., 2019. "Algorithm appreciation: People prefer algorithmic to human judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 151(C), pages 90-103.
    11. Maggioni, Mario A. & Rossignoli, Domenico, 2023. "If it looks like a human and speaks like a human ... Communication and cooperation in strategic Human–Robot interactions," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 104(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mathieu Chevrier, 2025. "Social Reputation as one of the Key Driver of AI Over-Reliance: An Experimental Test with ChatGPT-3.5," GREDEG Working Papers 2025-12, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chacon, Alvaro & Kausel, Edgar E. & Reyes, Tomas & Trautmann, Stefan, 2025. "Preventing algorithm aversion: People are willing to use algorithms with a learning label," Journal of Business Research, Elsevier, vol. 187(C).
    2. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    3. Alexia GAUDEUL & Caterina GIANNETTI, 2023. "Trade-offs in the design of financial algorithms," Discussion Papers 2023/288, Dipartimento di Economia e Management (DEM), University of Pisa, Pisa, Italy.
    4. Ivanova-Stenzel, Radosveta & Tolksdorf, Michel, 2024. "Measuring preferences for algorithms — How willing are people to cede control to algorithms?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 112(C).
    5. Daniel Woods & Mustafa Abdallah & Saurabh Bagchi & Shreyas Sundaram & Timothy Cason, 2022. "Network defense and behavioral biases: an experimental study," Experimental Economics, Springer;Economic Science Association, vol. 25(1), pages 254-286, February.
    6. Bryce McLaughlin & Jann Spiess, 2022. "Algorithmic Assistance with Recommendation-Dependent Preferences," Papers 2208.07626, arXiv.org, revised Jan 2024.
    7. Markus Jung & Mischa Seiter, 2021. "Towards a better understanding on mitigating algorithm aversion in forecasting: an experimental study," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 32(4), pages 495-516, December.
    8. Tse, Tiffany Tsz Kwan & Hanaki, Nobuyuki & Mao, Bolin, 2024. "Beware the performance of an algorithm before relying on it: Evidence from a stock price forecasting experiment," Journal of Economic Psychology, Elsevier, vol. 102(C).
    9. Ekaterina Jussupow & Kai Spohrer & Armin Heinzl & Joshua Gawlitza, 2021. "Augmenting Medical Diagnosis Decisions? An Investigation into Physicians’ Decision-Making Process with Artificial Intelligence," Information Systems Research, INFORMS, vol. 32(3), pages 713-735, September.
    10. repec:cup:judgdm:v:15:y:2020:i:3:p:449-451 is not listed on IDEAS
    11. Wang, Xun & Rodrigues, Vasco Sanchez & Demir, Emrah & Sarkis, Joseph, 2024. "Algorithm aversion during disruptions: The case of safety stock," International Journal of Production Economics, Elsevier, vol. 278(C).
    12. Kevin Bauer & Andrej Gill, 2024. "Mirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies," Information Systems Research, INFORMS, vol. 35(1), pages 226-248, March.
    13. Mahmud, Hasan & Islam, A.K.M. Najmul & Mitra, Ranjan Kumar, 2023. "What drives managers towards algorithm aversion and how to overcome it? Mitigating the impact of innovation resistance through technology readiness," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    14. Ammeling, Jonas & Aubreville, Marc & Fritz, Alexis & Kießig, Angelika & Krügel, Sebastian & Uhl, Matthias, 2025. "An interdisciplinary perspective on AI-supported decision making in medicine," Technology in Society, Elsevier, vol. 81(C).
    15. Hemesath, Sebastian & Tepe, Markus, 2023. "Framing the approval to test self-driving cars on public roads. The effect of safety and competitiveness on citizens' agreement," Technology in Society, Elsevier, vol. 72(C).
    16. Said Kaawach & Oskar Kowalewski & Oleksandr Talavera, 2023. "Automatic vs Manual Investing: Role of Past Performance," Discussion Papers 23-04, Department of Economics, University of Birmingham.
    17. Awtrey, Eli & Thornley, Nico & Dannals, Jennifer E. & Barnes, Christopher M. & Uhlmann, Eric Luis, 2021. "Distribution neglect in performance evaluations," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 213-227.
    18. Mallory Avery & Andreas Leibbrandt & Joseph Vecci, 2023. "Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech," Monash Economics Working Papers 2023-09, Monash University, Department of Economics.
    19. Martin Adam & Konstantin Roethke & Alexander Benlian, 2023. "Human vs. Automated Sales Agents: How and Why Customer Responses Shift Across Sales Stages," Information Systems Research, INFORMS, vol. 34(3), pages 1148-1168, September.
    20. Wang, Cuicui & Li, Yiyang & Fu, Weizhong & Jin, Jia, 2023. "Whether to trust chatbots: Applying the event-related approach to understand consumers’ emotional experiences in interactions with chatbots in e-commerce," Journal of Retailing and Consumer Services, Elsevier, vol. 73(C).
    21. Ivanov, Dmitry, 2023. "Intelligent digital twin (iDT) for supply chain stress-testing, resilience, and viability," International Journal of Production Economics, Elsevier, vol. 263(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:dpr:wpaper:1233r. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/isosujp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.