IDEAS home Printed from https://ideas.repec.org/p/tse/wpaper/130935.html
   My bibliography  Save this paper

Delegation to Artificial Intelligence can increase dishonest behaviour

Author

Listed:
  • Köbis, Nils
  • Rahwan, Zoe
  • Rilla, Raluca
  • Supriyatno, Bramantyo Ibrahim
  • Bersch, Clara
  • Ajaj, Tamer
  • Bonnefon, Jean-François
  • Rahwan, Iyad

Abstract

Although artificial intelligence enables productivity gains from delegating tasks to machines1, it may facilitate the delegation of unethical behaviour2. This risk is highly relevant amid the rapid rise of ‘agentic' artificial intelligence systems3,4. Here we demonstrate this risk by having human principals instruct machine agents to perform tasks with incentives to cheat. Requests for cheating increased when principals could induce machine dishonesty without telling the machine precisely what to do, through supervised learning or high-level goal setting. These effects held whether delegation was voluntary or mandatory. We also examined delegation via natural language to large language models5. Although the cheating requests by principals were not always higher for machine agents than for human agents, compliance diverged sharply: machines were far more likely than human agents to carry out fully unethical instructions. This compliance could be curbed, but usually not eliminated, with the injection of prohibitive, task-specific guardrails. Our results highlight ethical risks in the context of increasingly accessible and powerful machine delegation, and suggest design and policy strategies to mitigate them.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Köbis, Nils & Rahwan, Zoe & Rilla, Raluca & Supriyatno, Bramantyo Ibrahim & Bersch, Clara & Ajaj, Tamer & Bonnefon, Jean-François & Rahwan, Iyad, 2025. "Delegation to Artificial Intelligence can increase dishonest behaviour," TSE Working Papers 25-1663, Toulouse School of Economics (TSE).
  • Handle: RePEc:tse:wpaper:130935
    as

    Download full text from publisher

    File URL: https://www.tse-fr.eu/sites/default/files/TSE/documents/doc/wp/2025/wp_tse_1663.pdf
    File Function: Full Text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nils Köbis & Jean-François Bonnefon & Iyad Rahwan, 2021. "Bad machines corrupt good morals," Nature Human Behaviour, Nature, vol. 5(6), pages 679-685, June.
    2. Terrence Hendershott & Charles M. Jones & Albert J. Menkveld, 2011. "Does Algorithmic Trading Improve Liquidity?," Journal of Finance, American Finance Association, vol. 66(1), pages 1-33, February.
    3. Gogoll, Jan & Uhl, Matthias, 2018. "Rage against the machine: Automation in the moral domain," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 74(C), pages 97-103.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Foucart, Renaud & Zeng, Fanqi & Wang, Shidong, 2025. "The Social Importance of Being Stubborn When an Organization Meets AI," SocArXiv nfgy3_v1, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chugunova, Marina & Sele, Daniela, 2022. "We and It: An interdisciplinary review of the experimental evidence on how humans interact with machines," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 99(C).
    2. March, Christoph, 2021. "Strategic interactions between humans and artificial intelligence: Lessons from experiments with computer players," Journal of Economic Psychology, Elsevier, vol. 87(C).
    3. Marius Protte & Behnud Mir Djawadi, 2025. "Human vs. Algorithmic Auditors: The Impact of Entity Type and Ambiguity on Human Dishonesty," Papers 2507.15439, arXiv.org.
    4. Köbis, Nils & Rahwan, Zoe & Bersch, Clara & Ajaj, Tamer & Bonnefon, Jean-François & Rahwan, Iyad, 2024. "Experimental evidence that delegating to intelligent machines can increase dishonest behaviour," OSF Preprints dnjgz, Center for Open Science.
    5. repec:osf:osfxxx:dnjgz_v1 is not listed on IDEAS
    6. Evangelos Benos & Satchit Sagade, 2012. "High-frequency trading behaviour and its impact on market quality: evidence from the UK equity market," Bank of England working papers 469, Bank of England.
    7. Bellia, Mario & Christensen, Kim & Kolokolov, Aleksey & Pelizzon, Loriana & Renò, Roberto, 2022. "Do designated market makers provide liquidity during a flash crash?," SAFE Working Paper Series 270, Leibniz Institute for Financial Research SAFE, revised 2022.
    8. Tamer Khraisha & Keren Arthur, 2018. "Can we have a general theory of financial innovation processes? A conceptual review," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 4(1), pages 1-27, December.
    9. Nils Köbis & Jean-François Bonnefon & Iyad Rahwan, 2021. "Bad machines corrupt good morals," Nature Human Behaviour, Nature, vol. 5(6), pages 679-685, June.
    10. Bruno Biais & Fany Declerck & Sophie Moinas, 2016. "Who supplies liquidity, how and when?," BIS Working Papers 563, Bank for International Settlements.
    11. Indriawan, Ivan & Martinez, Valeria & Tse, Yiuman, 2021. "The impact of the change in USDA announcement release procedures on agricultural commodity futures," Journal of Commodity Markets, Elsevier, vol. 23(C).
    12. Álvaro Cartea & José Penalva, 2012. "Where is the Value in High Frequency Trading?," Quarterly Journal of Finance (QJF), World Scientific Publishing Co. Pte. Ltd., vol. 2(03), pages 1-46.
    13. NIdhi Aggarwal & Venkatesh Panchapagesan & Susan Thomas, 2022. "When is the Order to Trade Ratio fee effective?," Working Papers 8, xKDR.
    14. Kang, Jongho & Kang, Jangkoo & Kwon, Kyung Yoon, 2022. "Market versus limit orders of speculative high-frequency traders and price discovery," Research in International Business and Finance, Elsevier, vol. 63(C).
    15. Ahmed Baig & Nasim Sabah & Drew Winters, 2019. "Have Stock Prices become more Uniformly Distributed?," Economics Bulletin, AccessEcon, vol. 39(2), pages 1242-1250.
    16. Robert J. Kauffman & Yuzhou Hu & Dan Ma, 2015. "Will high-frequency trading practices transform the financial markets in the Asia Pacific Region?," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 1(1), pages 1-27, December.
    17. Uctum, Remzi & Renou-Maissant, Patricia & Prat, Georges & Lecarpentier-Moyal, Sylvie, 2017. "Persistence of announcement effects on the intraday volatility of stock returns: Evidence from individual data," Review of Financial Economics, Elsevier, vol. 35(C), pages 43-56.
    18. George Jiang & Ingrid Lo & Giorgio Valente, 2014. "High-Frequency Trading around Macroeconomic News Announcements: Evidence from the U.S. Treasury Market," Staff Working Papers 14-56, Bank of Canada.
    19. Aggarwal, Nidhi & Panchapagesan, Venkatesh & Thomas, Susan, 2023. "When is the order-to-trade ratio fee effective?," Journal of Financial Markets, Elsevier, vol. 62(C).
    20. Bank, Matthias & Baumann, Ralf H., 2016. "Price formation, market quality and the effects of reduced latency in the very short run," Research in International Business and Finance, Elsevier, vol. 37(C), pages 629-645.
    21. Jing Nie & Juliana Malagon & Julian Williams, 2022. "The impact of high speed quoting on execution risk dynamics: Evidence from interest rate futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 42(8), pages 1434-1465, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tse:wpaper:130935. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/tsetofr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.