IDEAS home Printed from https://ideas.repec.org/p/wiw/wus055/60237853.html

Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study

Author

Listed:
  • Greiner, Ben
  • Grünwald, Philipp
  • Lindner, Thomas
  • Lintner, Georg
  • Wiernsperger, Martin

Abstract

Managerial decision-makers are increasingly supported by advanced data analytics and other AI-based technologies, but are often found to be hesitant to follow the algorithmic advice. We examine how compensation contract design and framing of an AI algorithm influence decision-makers’ reliance on algorithmic advice and performance in a price estimation task. Based on a large sample of almost 1,500 participants, we find that compared to a fixed compensation, both compensation contracts based on individual performance and tournament contracts lead to an increase in effort duration and to more reliance on algorithmic advice. We further find that using an AI algorithm that is framed as incorporating also human expertise has positive effects on advice utilization, especially for decision-makers with fixed pay contracts. By showing how widely used control practices such as incentives and task framing influence the interaction of human decision-makers with AI algorithms, our findings have direct implications for managerial practice.

Suggested Citation

  • Greiner, Ben & Grünwald, Philipp & Lindner, Thomas & Lintner, Georg & Wiernsperger, Martin, 2024. "Incentives, Framing, and Reliance on Algorithmic Advice: An Experimental Study," Department for Strategy and Innovation Working Paper Series 01/2024, WU Vienna University of Economics and Business.
    • Handle: RePEc:wiw:wus055:60237853
    as

    Download full text from publisher

    File URL: https://research.wu.ac.at/en/publications/137467e7-9d47-4a6c-87dc-673ff4f58009
    File Function: original version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ottaviani, Marco & Sorensen, Peter Norman, 2006. "The strategy of professional forecasting," Journal of Financial Economics, Elsevier, vol. 81(2), pages 441-466, August.
    2. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    3. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    4. Andrew Schotter & Isabel Trevino, 2014. "Belief Elicitation in the Laboratory," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 103-128, August.
    5. Kevin Bauer & Moritz von Zahn & Oliver Hinz, 2023. "Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing," Information Systems Research, INFORMS, vol. 34(4), pages 1582-1602, December.
    6. Charness, Gary & Gneezy, Uri & Halladay, Brianna, 2016. "Experimental methods: Pay one or pay all," Journal of Economic Behavior & Organization, Elsevier, vol. 131(PA), pages 141-150.
    7. Tanjim Hossain & Ryo Okui, 2013. "The Binarized Scoring Rule," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 80(3), pages 984-1001.
    8. Bonner, Sarah E. & Sprinkle, Geoffrey B., 2002. "The effects of monetary incentives on effort and task performance: theories, evidence, and a framework for research," Accounting, Organizations and Society, Elsevier, vol. 27(4-5), pages 303-345.
    9. Arkes, Hal R. & Dawes, Robyn M. & Christensen, Caryn, 1986. "Factors influencing the use of a decision rule in a probabilistic task," Organizational Behavior and Human Decision Processes, Elsevier, vol. 37(1), pages 93-110, February.
    10. Charness, Gary & Gneezy, Uri & Rasocha, Vlastimil, 2021. "Experimental methods: Eliciting beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 234-256.
    11. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    12. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    13. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    14. Weiguang Wang & Guodong (Gordon) Gao & Ritu Agarwal, 2024. "Friend or Foe? Teaming Between Artificial Intelligence and Workers with Variation in Experience," Management Science, INFORMS, vol. 70(9), pages 5753-5775, September.
    15. Benjamin P. Commerford & Sean A. Dennis & Jennifer R. Joe & Jenny W. Ulla, 2022. "Man Versus Machine: Complex Estimates and Auditor Reliance on Artificial Intelligence," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 60(1), pages 171-201, March.
    16. Schotter,Andrew, 2023. "Advice, Social Learning and the Evolution of Conventions," Cambridge Books, Cambridge University Press, number 9781316518076, Enero-Abr.
    17. Fabian Gaessler & Henning Piezunka, 2023. "Training with AI: Evidence from chess computers," Strategic Management Journal, Wiley Blackwell, vol. 44(11), pages 2724-2750, November.
    18. Joshua Tasoff & Wenjie Zhang, 2022. "The Performance of Time-Preference and Risk-Preference Measures in Surveys," Management Science, INFORMS, vol. 68(2), pages 1149-1173, February.
    19. Ryan Allen & Prithwiraj (Raj) Choudhury, 2022. "Algorithm-Augmented Work and Domain Experience: The Countervailing Forces of Ability and Aversion," Organization Science, INFORMS, vol. 33(1), pages 149-169, January.
    20. Costello, Anna M. & Down, Andrea K. & Mehta, Mihir N., 2020. "Machine + man: A field experiment on the role of discretion in augmenting AI-based lending models," Journal of Accounting and Economics, Elsevier, vol. 70(2).
    21. Shujun Ding & Philip Beaulieu, 2011. "The Role of Financial Incentives in Balanced Scorecard‐Based Performance Evaluations: Correcting Mood Congruency Biases," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 49(5), pages 1223-1247, December.
    22. Prithwiraj Choudhury & Evan Starr & Rajshree Agarwal, 2020. "Machine learning and human capital complementarities: Experimental evidence on bias mitigation," Strategic Management Journal, Wiley Blackwell, vol. 41(8), pages 1381-1411, August.
    23. Andrew Prahl & Lyn Van Swol, 2017. "Understanding algorithm aversion: When is advice from automation discounted?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(6), pages 691-702, September.
    24. Ashton, Rh, 1990. "Pressure And Performance In Accounting Decision Settings - Paradoxical Effects Of Incentives, Feedback, And Justification," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 28, pages 148-180.
    25. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    26. Sebastian Krakowski & Johannes Luger & Sebastian Raisch, 2023. "Artificial intelligence and the changing sources of competitive advantage," Strategic Management Journal, Wiley Blackwell, vol. 44(6), pages 1425-1452, June.
    27. Cassandra Estep & Emily E. Griffith & Nikki L. MacKenzie, 2024. "How do financial executives respond to the use of artificial intelligence in financial reporting and auditing?," Review of Accounting Studies, Springer, vol. 29(3), pages 2798-2831, September.
    28. Libby, Robert & Luft, Joan, 1993. "Determinants of judgment performance in accounting settings: Ability, knowledge, motivation, and environment," Accounting, Organizations and Society, Elsevier, vol. 18(5), pages 425-450, July.
    29. Libby, R & Lipe, Mg, 1992. "Incentives, Effort, And The Cognitive-Processes Involved In Accounting-Related Judgments," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 30(2), pages 249-273.
    30. Logg, Jennifer M. & Minson, Julia A. & Moore, Don A., 2019. "Algorithm appreciation: People prefer algorithmic to human judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 151(C), pages 90-103.
    31. Yaron Azrieli & Christopher P. Chambers & Paul J. Healy, 2018. "Incentives in Experiments: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1472-1503.
    32. Kevin Bauer & Oliver Hinz & Wil Aalst & Christof Weinhardt, 2021. "Expl(AI)n It to Me – Explainable AI and Information Systems Research," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(2), pages 79-82, April.
    33. Labro, Eva & Lang, Mark & Omartian, James D., 2023. "Predictive analytics and centralization of authority," Journal of Accounting and Economics, Elsevier, vol. 75(1).
    34. Frechette, Guillaume R. & Schotter, Andrew (ed.), 2015. "Handbook of Experimental Economic Methodology," OUP Catalogue, Oxford University Press, number 9780195328325.
    35. Harvey, Nigel & Fischer, Ilan, 1997. "Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(2), pages 117-133, May.
    36. Efendić, Emir & Van de Calseyde, Philippe P.F.M. & Evans, Anthony M., 2020. "Slow response times undermine trust in algorithmic (but not human) predictions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 157(C), pages 103-114.
    37. Miao Liu, 2022. "Assessing Human Information Processing in Lending Decisions: A Machine Learning Approach," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 60(2), pages 607-651, May.
    38. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    39. Christoph March & Anthony Ziegelmeyer & Ben Greiner & René Cyranek, 2016. "Pay Few Subjects but Pay Them Well: Cost-Effectiveness of Random Incentive Systems," CESifo Working Paper Series 5988, CESifo.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marie-Pierre Dargnies & Rustamdjan Hakimov & Dorothea Kübler, 2026. "Aversion to Hiring Algorithms: Transparency, Gender Profiling, and Self-Confidence," Management Science, INFORMS, vol. 72(1), pages 285-301, January.
    2. Bohren, Noah & Hakimov, Rustamdjan & Lalive, Rafael, 2024. "Creative and Strategic Capabilities of Generative AI: Evidence from Large-Scale Experiments," IZA Discussion Papers 17302, IZA Network @ LISER.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian Krakowski & Darek Haftor & Johannes Luger & Natallia Pashkevich & Sebastian Raisch, 2026. "Human-Centered Artificial Intelligence: A Field Experiment," Management Science, INFORMS, vol. 72(1), pages 57-72, January.
    2. Robert M. Gillenkirch & Julia Ortner & Sebastian Robert & Louis Velthuis, 2023. "Designing incentives and performance measurement for advisors: How to make decision-makers listen to advice," Working Papers 2304, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    3. Marius Protte & Behnud Mir Djawadi, 2025. "Human vs. Algorithmic Auditors: The Impact of Entity Type and Ambiguity on Human Dishonesty," Papers 2507.15439, arXiv.org.
    4. Aycinena, Diego & Bogliacino, Francesco & Kimbrough, Erik O., 2024. "Measuring norms: Assessing the threat of social desirability bias to the Bicchieri and Xiao elicitation method," Journal of Economic Behavior & Organization, Elsevier, vol. 222(C), pages 225-239.
    5. Mahmud, Hasan & Islam, A.K.M. Najmul & Ahmed, Syed Ishtiaque & Smolander, Kari, 2022. "What influences algorithmic decision-making? A systematic literature review on algorithm aversion," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    6. Markus Jung & Mischa Seiter, 2021. "Towards a better understanding on mitigating algorithm aversion in forecasting: an experimental study," Journal of Management Control: Zeitschrift für Planung und Unternehmenssteuerung, Springer, vol. 32(4), pages 495-516, December.
    7. Stephen L. Cheung & Lachlan Johnstone, 2025. "True overconfidence, revealed through actions: An experiment," Journal of Risk and Uncertainty, Springer, vol. 70(2), pages 171-199, April.
    8. Keding, Christoph & Meissner, Philip, 2021. "Managerial overreliance on AI-augmented decision-making processes: How the use of AI-based advisory systems shapes choice behavior in R&D investment decisions," Technological Forecasting and Social Change, Elsevier, vol. 171(C).
    9. Hassan, Reda & Nguyen, Nhien & Finserås, Stine Rasdal & Adde, Lars & Strümke, Inga & Støen, Ragnhild, 2025. "Unlocking the black box: Enhancing human-AI collaboration in high-stakes healthcare scenarios through explainable AI," Technological Forecasting and Social Change, Elsevier, vol. 219(C).
    10. Burdea, Valeria & Woon, Jonathan, 2022. "Online belief elicitation methods," Journal of Economic Psychology, Elsevier, vol. 90(C).
    11. Bauer, Kevin & von Zahn, Moritz & Hinz, Oliver, 2022. "Expl(AI)ned: The impact of explainable Artificial Intelligence on cognitive processes," SAFE Working Paper Series 315, Leibniz Institute for Financial Research SAFE, revised 2022.
    12. Ersoy, Fulya Y., 2025. "Do incentives matter in elicitation of beliefs?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 119(C).
    13. Samuel N. Kirshner, 2025. "Psychological Distance and Algorithm Aversion: Congruency and Advisor Confidence," Service Science, INFORMS, vol. 17(2-3), pages 74-91, June.
    14. Merle, Aurélie & St-Onge, Anik & Sénécal, Sylvain, 2022. "Does it pay to be honest? The effect of retailer-provided negative feedback on consumers’ product choice and shopping experience," Journal of Business Research, Elsevier, vol. 147(C), pages 532-543.
    15. Chen, Yan & He, YingHua, 2021. "Information acquisition and provision in school choice: An experimental study," Journal of Economic Theory, Elsevier, vol. 197(C).
    16. Kevin Bauer & Andrej Gill, 2024. "Mirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies," Information Systems Research, INFORMS, vol. 35(1), pages 226-248, March.
    17. Dominik Bauer & Irenaeus Wolff, 2018. "Biases in Beliefs: Experimental Evidence," TWI Research Paper Series 109, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    18. Gorny, Paul M. & Schäfer, Louis, 2025. "Adaptivity and Revealed Robot Aversion in Human-Robot Collaboration: A Field-in-the-Lab Experiment," MPRA Paper 126663, University Library of Munich, Germany.
    19. Rafkin, Charlie & Shreekumar, Advik & Vautrey, Pierre-Luc, 2021. "When guidance changes: Government stances and public beliefs," Journal of Public Economics, Elsevier, vol. 196(C).
    20. Zulia Gubaydullina & Jan René Judek & Marco Lorenz & Markus Spiwoks, 2022. "Comparing Different Kinds of Influence on an Algorithm in Its Forecasting Process and Their Impact on Algorithm Aversion," Businesses, MDPI, vol. 2(4), pages 1-23, October.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wiw:wus055:60237853. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: WU Library (email available below). General contact details of provider: https://research.wu.ac.at/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.