IDEAS home Printed from https://ideas.repec.org/p/gre/wpaper/2024-03.html

Algorithm Credulity: Human and Algorithmic Advice in Prediction Experiments

Author

Listed:
  • Mathieu Chevrier

    (Université Côte d'Azur, CNRS, GREDEG, France)

  • Brice Corgnet

    (Emlyon Business School, GATE UMR 5824, France)

  • Eric Guerci

    (Université Côte d'Azur, CNRS, GREDEG, France)

  • Julie Rosaz

    (CEREN EA 7477, Burgundy School of Business, Université Bourgogne Franche-Comté, Dijon, France)

Abstract

This study examines algorithm credulity by which people rely on faulty algorithmic advice without critical evaluation. Using a prediction task comparing human and algorithm advisors, we find that participants are more likely to follow the same deficient advice when issued by an algorithm than by a human. We show that algorithm credulity reduces expected earnings by 13%. To explain this finding, we posit that people are more likely to perceive as credible an unpredictable and deficient piece of advice when produced by an algorithm than by a human. Overall, our results imply that humans might be particularly susceptible to the influence of deficient algorithmic advice.

Suggested Citation

  • Mathieu Chevrier & Brice Corgnet & Eric Guerci & Julie Rosaz, 2024. "Algorithm Credulity: Human and Algorithmic Advice in Prediction Experiments," GREDEG Working Papers 2024-03, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France, revised Dec 2024.
  • Handle: RePEc:gre:wpaper:2024-03
    as

    Download full text from publisher

    File URL: http://195.220.190.85/GREDEG-WP-2024-03.pdf
    File Function: Revised version, 2024-12
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dargnies, Marie-Pierre & Hakimov, Rustamdjan & Kübler, Dorothea, 2022. "Aversion to hiring algorithms: Transparency, gender profiling, and self-confidence," Discussion Papers, Research Unit: Market Behavior SP II 2022-202, WZB Berlin Social Science Center.
    2. Dohmen, Thomas & Falk, Armin & Huffman, David & Marklein, Felix & Sunde, Uwe, 2009. "Biased probability judgment: Evidence of incidence and relationship to economic outcomes from a representative sample," Journal of Economic Behavior & Organization, Elsevier, vol. 72(3), pages 903-915, December.
    3. Cedric A. Lehmann & Christiane B. Haubitz & Andreas Fügener & Ulrich W. Thonemann, 2022. "The risk of algorithm transparency: How algorithm complexity drives the effects on the use of advice," Production and Operations Management, Production and Operations Management Society, vol. 31(9), pages 3419-3434, September.
    4. Wu, Kewen & Zhao, Yuxiang & Zhu, Qinghua & Tan, Xiaojie & Zheng, Hua, 2011. "A meta-analysis of the impact of trust on technology acceptance model: Investigation of moderating influence of subject and context type," International Journal of Information Management, Elsevier, vol. 31(6), pages 572-581.
    5. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    6. Edward T. Cokely & Colleen M. Kelley, 2009. "Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 4(1), pages 20-33, February.
    7. Highhouse, Scott, 2008. "Stubborn Reliance on Intuition and Subjectivity in Employee Selection," Industrial and Organizational Psychology, Cambridge University Press, vol. 1(3), pages 333-342, September.
    8. Rachel Croson & James Sundali, 2005. "The Gambler’s Fallacy and the Hot Hand: Empirical Data from Casinos," Journal of Risk and Uncertainty, Springer, vol. 30(3), pages 195-209, May.
    9. Benjamin Haibe-Kains & George Alexandru Adam & Ahmed Hosny & Farnoosh Khodakarami & Levi Waldron & Bo Wang & Chris McIntosh & Anna Goldenberg & Anshul Kundaje & Casey S. Greene & Tamara Broderick & Mi, 2020. "Transparency and reproducibility in artificial intelligence," Nature, Nature, vol. 586(7829), pages 14-16, October.
    10. Corgnet, Brice & Hernán-González, Roberto & Mateo, Ricardo, 2023. "Peer effects in an automated world," Labour Economics, Elsevier, vol. 85(C).
    11. Charles T. Clotfelter & Philip J. Cook, 1993. "Notes: The "Gambler's Fallacy" in Lottery Play," Management Science, INFORMS, vol. 39(12), pages 1521-1525, December.
    12. Guillermo Campitelli & Martin Labollita, 2010. "Correlations of cognitive reflection with judgments and choices," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(3), pages 182-191, June.
    13. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    14. Liang, Annie, 2019. "Inference of preference heterogeneity from choice data," Journal of Economic Theory, Elsevier, vol. 179(C), pages 275-311.
    15. Brice Corgnet & Mark Desantis & David Porter, 2018. "What Makes a Good Trader? On the Role of Intuition and Reflection on Trader Performance," Journal of Finance, American Finance Association, vol. 73(3), pages 1113-1137, June.
    16. Matthew Rabin, 2002. "Inference by Believers in the Law of Small Numbers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 775-816.
    17. repec:hal:journl:hal-02312062 is not listed on IDEAS
    18. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    19. Hey, John D., 1998. "Experimental economics and deception: A comment," Journal of Economic Psychology, Elsevier, vol. 19(3), pages 397-401, June.
    20. Oechssler, Jörg & Roider, Andreas & Schmitz, Patrick W., 2009. "Cognitive abilities and behavioral biases," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 147-152, October.
    21. Gary Charness & Anya Samek & Jeroen Ven, 2022. "What is considered deception in experimental economics?," Experimental Economics, Springer;Economic Science Association, vol. 25(2), pages 385-412, April.
    22. Brice Corgnet, 2023. "An Experimental Test of Algorithmic Dismissals," Working Papers 23-02, Chapman University, Economic Science Institute.
    23. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    24. Fumagalli, Elena & Rezaei, Sarah & Salomons, Anna, 2022. "OK computer: Worker perceptions of algorithmic recruitment," Research Policy, Elsevier, vol. 51(2).
    25. Jürgen Huber & Michael Kirchler & Thomas Stöckl, 2010. "The hot hand belief and the gambler’s fallacy in investment decisions under risk," Theory and Decision, Springer, vol. 68(4), pages 445-462, April.
    26. Viswanath Venkatesh & Fred D. Davis, 2000. "A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies," Management Science, INFORMS, vol. 46(2), pages 186-204, February.
    27. Andrew Prahl & Lyn Van Swol, 2017. "Understanding algorithm aversion: When is advice from automation discounted?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(6), pages 691-702, September.
    28. Mahmud, Hasan & Islam, A.K.M. Najmul & Ahmed, Syed Ishtiaque & Smolander, Kari, 2022. "What influences algorithmic decision-making? A systematic literature review on algorithm aversion," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    29. Aleksandr Alekseev, 2020. "The Economics of Babysitting a Robot," Working Papers 20-29, Chapman University, Economic Science Institute.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicolas Eber & Patrick Roger & Tristan Roger, 2024. "Finance and intelligence: An overview of the literature," Journal of Economic Surveys, Wiley Blackwell, vol. 38(2), pages 503-554, April.
    2. Corgnet, Brice & DeSantis, Mark & Porter, David, 2020. "The distribution of information and the price efficiency of markets," Journal of Economic Dynamics and Control, Elsevier, vol. 110(C).
    3. Corgnet, Brice & DeSantis, Mark & Porter, David, 2021. "Information aggregation and the cognitive make-up of market participants," European Economic Review, Elsevier, vol. 133(C).
    4. Taylor, Matthew P., 2020. "Heterogeneous motivation and cognitive ability in the lab," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    5. Amador-Hidalgo, Luis & Brañas-Garza, Pablo & Espín, Antonio M. & García-Muñoz, Teresa & Hernández-Román, Ana, 2021. "Cognitive abilities and risk-taking: Errors, not preferences," European Economic Review, Elsevier, vol. 134(C).
    6. Taylor, Matthew P., 2016. "Are high-ability individuals really more tolerant of risk? A test of the relationship between risk aversion and cognitive ability," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 63(C), pages 136-147.
    7. Gerardo Sabater-Grande & Nikolaos Georgantzís & Noemí Herranz-Zarzoso, 2023. "Goals and guesses as reference points: a field experiment on student performance," Theory and Decision, Springer, vol. 94(2), pages 249-274, February.
    8. Amador, Luis & Brañas-Garza, Pablo & Espín, Antonio M. & Garcia, Teresa & Hernández, Ana, 2019. "Consistent and inconsistent choices under uncertainty: The role of cognitive abilities," MPRA Paper 95178, University Library of Munich, Germany.
    9. Brice Corgnet & Mark DeSantis & David Porter, 2020. "Information Aggregation and the Cognitive Make-up of Traders," Working Papers 20-18, Chapman University, Economic Science Institute.
    10. Markus Jung & Mischa Seiter, 2021. "Towards a better understanding on mitigating algorithm aversion in forecasting: an experimental study," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 32(4), pages 495-516, December.
    11. Antoni Bosch-Domènech & Pablo Brañas-Garza & Antonio M. Espín, 2013. "Fetal testosterone (2D:4D) as predictor of cognitive reflection," Economics Working Papers 1371, Department of Economics and Business, Universitat Pompeu Fabra.
    12. Ciril Bosch-Rosa & Thomas Meissner & Antoni Bosch-Domènech, 2018. "Cognitive bubbles," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 132-153, March.
    13. Maximilian Späth & Daniel Goller, 2023. "Gender differences in investment reactions to irrelevant information," CEPA Discussion Papers 67, Center for Economic Policy Analysis.
    14. Jia, Z. Tingting & McMahon, Matthew J., 2020. "Being watched in an investment game setting: Behavioral changes when making risky decisions," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 88(C).
    15. Joshua Zonca & Giorgio Coricelli & Luca Polonio, 2020. "Gaze patterns disclose the link between cognitive reflection and sophistication in strategic interaction," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(2), pages 230-245, March.
    16. Zonca, Joshua & Coricelli, Giorgio & Polonio, Luca, 2020. "Gaze patterns disclose the link between cognitive reflection and sophistication in strategic interaction," Judgment and Decision Making, Cambridge University Press, vol. 15(2), pages 230-245, March.
    17. Brice Corgnet & Mark Desantis & David Porter, 2018. "What Makes a Good Trader? On the Role of Intuition and Reflection on Trader Performance," Journal of Finance, American Finance Association, vol. 73(3), pages 1113-1137, June.
    18. Dustin P. Calvillo & Jessica N. Burgeno, 2015. "Cognitive reflection predicts the acceptance of unfair ultimatum game offers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(4), pages 332-341, July.
    19. Duttle, Kai, 2015. "Disentangling two causes of biased probability judgment: Cognitive skills and perception of randomness," Ruhr Economic Papers 568, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    20. Bao, Te & Corgnet, Brice & Hanaki, Nobuyuki & Okada, Katsuhiko & Riyanto, Yohanes E. & Zhu, Jiahua, 2025. "Financial forecasting in the lab and the field: Qualified professionals vs. smart students," Journal of Behavioral and Experimental Finance, Elsevier, vol. 46(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gre:wpaper:2024-03. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Patrice Bougette (email available below). General contact details of provider: https://edirc.repec.org/data/credcfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.