IDEAS home Printed from https://ideas.repec.org/p/hal/wpaper/hal-05273501.html
   My bibliography  Save this paper

Delegation to Artificial Intelligence can increase dishonest behaviour

Author

Listed:
  • Nils Köbis

    (Universität Duisburg-Essen = University of Duisburg-Essen [Essen], Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Zoe Rahwan

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Raluca Rilla

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Bramantyo Ibrahim Supriyatno

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Clara Bersch

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Tamer Ajaj

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

  • Jean-François Bonnefon

    (TSE-R - Toulouse School of Economics - UT Capitole - Université Toulouse Capitole - Comue de Toulouse - Communauté d'universités et établissements de Toulouse - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique - INRAE - Institut National de Recherche pour l’Agriculture, l’Alimentation et l’Environnement)

  • Iyad Rahwan

    (Max Planck Institute for Human Development - Max-Planck-Gesellschaft)

Abstract

While Artificial Intelligence enables productivity gains from delegating tasks to machines, it may facilitate the delegation of unethical behaviour. This risk is highly relevant amid the rapid rise of 'agentic' AI systems. Here we demonstrate this risk by having human principals instruct machine agents to perform tasks with incentives to cheat. Requests for cheating increased when principals could induce machine dishonesty without telling the machine precisely what to do, through supervised learning or high-level goal-setting. These effects held whether delegation was voluntary or mandatory. We also examined delegation via natural language to Large Language Models. While principals' cheating requests were not always higher for machine agents, compliance diverged sharply: Machines were far more likely than human agents to carry out fully unethical instructions. This compliance could be curbed, but usually not eliminated, with the injection of prohibitive, task-specific guardrails. Our results highlight ethical risks in the context of increasingly accessible and powerful machine delegation, and suggest design and policy strategies to mitigate them.

Suggested Citation

  • Nils Köbis & Zoe Rahwan & Raluca Rilla & Bramantyo Ibrahim Supriyatno & Clara Bersch & Tamer Ajaj & Jean-François Bonnefon & Iyad Rahwan, 2025. "Delegation to Artificial Intelligence can increase dishonest behaviour," Working Papers hal-05273501, HAL.
  • Handle: RePEc:hal:wpaper:hal-05273501
    Note: View the original document on HAL open archive server: https://hal.science/hal-05273501v1
    as

    Download full text from publisher

    File URL: https://hal.science/hal-05273501v1/document
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nils Köbis & Jean-François Bonnefon & Iyad Rahwan, 2021. "Bad machines corrupt good morals," Nature Human Behaviour, Nature, vol. 5(6), pages 679-685, June.
    2. Terrence Hendershott & Charles M. Jones & Albert J. Menkveld, 2011. "Does Algorithmic Trading Improve Liquidity?," Journal of Finance, American Finance Association, vol. 66(1), pages 1-33, February.
    3. Gogoll, Jan & Uhl, Matthias, 2018. "Rage against the machine: Automation in the moral domain," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 74(C), pages 97-103.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Foucart, Renaud & Zeng, Fanqi & Wang, Shidong, 2025. "The Social Importance of Being Stubborn When an Organization Meets AI," SocArXiv nfgy3_v1, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marius Protte & Behnud Mir Djawadi, 2025. "Human vs. Algorithmic Auditors: The Impact of Entity Type and Ambiguity on Human Dishonesty," Papers 2507.15439, arXiv.org.
    2. repec:osf:osfxxx:dnjgz_v1 is not listed on IDEAS
    3. Chugunova, Marina & Sele, Daniela, 2022. "We and It: An interdisciplinary review of the experimental evidence on how humans interact with machines," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 99(C).
    4. March, Christoph, 2021. "Strategic interactions between humans and artificial intelligence: Lessons from experiments with computer players," Journal of Economic Psychology, Elsevier, vol. 87(C).
    5. Köbis, Nils & Rahwan, Zoe & Bersch, Clara & Ajaj, Tamer & Bonnefon, Jean-François & Rahwan, Iyad, 2024. "Experimental evidence that delegating to intelligent machines can increase dishonest behaviour," OSF Preprints dnjgz, Center for Open Science.
    6. Evangelos Benos & Satchit Sagade, 2012. "High-frequency trading behaviour and its impact on market quality: evidence from the UK equity market," Bank of England working papers 469, Bank of England.
    7. Bellia, Mario & Christensen, Kim & Kolokolov, Aleksey & Pelizzon, Loriana & Renò, Roberto, 2022. "Do designated market makers provide liquidity during a flash crash?," SAFE Working Paper Series 270, Leibniz Institute for Financial Research SAFE, revised 2022.
    8. Bruno Biais & Fany Declerck & Sophie Moinas, 2016. "Who supplies liquidity, how and when?," BIS Working Papers 563, Bank for International Settlements.
    9. Indriawan, Ivan & Martinez, Valeria & Tse, Yiuman, 2021. "The impact of the change in USDA announcement release procedures on agricultural commodity futures," Journal of Commodity Markets, Elsevier, vol. 23(C).
    10. Álvaro Cartea & José Penalva, 2012. "Where is the Value in High Frequency Trading?," Quarterly Journal of Finance (QJF), World Scientific Publishing Co. Pte. Ltd., vol. 2(03), pages 1-46.
    11. Ahmed Baig & Nasim Sabah & Drew Winters, 2019. "Have Stock Prices become more Uniformly Distributed?," Economics Bulletin, AccessEcon, vol. 39(2), pages 1242-1250.
    12. Robert J. Kauffman & Yuzhou Hu & Dan Ma, 2015. "Will high-frequency trading practices transform the financial markets in the Asia Pacific Region?," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 1(1), pages 1-27, December.
    13. Uctum, Remzi & Renou-Maissant, Patricia & Prat, Georges & Lecarpentier-Moyal, Sylvie, 2017. "Persistence of announcement effects on the intraday volatility of stock returns: Evidence from individual data," Review of Financial Economics, Elsevier, vol. 35(C), pages 43-56.
    14. George Jiang & Ingrid Lo & Giorgio Valente, 2014. "High-Frequency Trading around Macroeconomic News Announcements: Evidence from the U.S. Treasury Market," Staff Working Papers 14-56, Bank of Canada.
    15. Bank, Matthias & Baumann, Ralf H., 2016. "Price formation, market quality and the effects of reduced latency in the very short run," Research in International Business and Finance, Elsevier, vol. 37(C), pages 629-645.
    16. Linnenluecke, Martina K. & Chen, Xiaoyan & Ling, Xin & Smith, Tom & Zhu, Yushu, 2017. "Research in finance: A review of influential publications and a research agenda," Pacific-Basin Finance Journal, Elsevier, vol. 43(C), pages 188-199.
    17. Gousgounis, Eleni & Onur, Esen, 2018. "The effect of pit closure on futures trading," Journal of Commodity Markets, Elsevier, vol. 10(C), pages 69-90.
    18. Bruno Breyer Caldas & João Frois Caldeira & Guilherme Vale Moura, 2016. "Is Pairs Trading Performance Sensitive To The Methodologies?: A Comparison," Anais do XLII Encontro Nacional de Economia [Proceedings of the 42nd Brazilian Economics Meeting] 130, ANPEC - Associação Nacional dos Centros de Pós-Graduação em Economia [Brazilian Association of Graduate Programs in Economics].
    19. Benjamin Myers & Austin Gerig, 2013. "Simulating the Synchronizing Behavior of High-Frequency Trading in Multiple Markets," Papers 1311.4160, arXiv.org.
    20. Chiarella, Carl & Ladley, Daniel, 2016. "Chasing trends at the micro-level: The effect of technical trading on order book dynamics," Journal of Banking & Finance, Elsevier, vol. 72(S), pages 119-131.
    21. Vinay Patel, 2015. "Price Discovery in US and Australian Stock and Options Markets," PhD Thesis, Finance Discipline Group, UTS Business School, University of Technology, Sydney, number 27, July-Dece.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:hal-05273501. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.