IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.01147.html

Memoryless Algorithmic Collusion: Sure to Fail, Slow to Fall

Author

Listed:
  • Zhang Xu
  • Wei Zhao

Abstract

This paper shows that, in a class of Bertrand-style competition games, memoryless Q-learning algorithms should adapt to Nash Equilibrium given sufficient explorations in the long-run. This is also verified through accelerating simulations, while the convergence time grows super-exponentially for high discount factors. The resilience of collusive outcomes in the short-run is due to exploration of unprofitable actions, making the system resemble random walk. A structural model is proposed to estimate the convergence time, which not only explains the role of discount factor, but also unveils the non-monotonic relation in learning rate, which is overlooked in the literature.

Suggested Citation

  • Zhang Xu & Wei Zhao, 2024. "Memoryless Algorithmic Collusion: Sure to Fail, Slow to Fall," Papers 2409.01147, arXiv.org, revised Aug 2026.
    • Handle: RePEc:arx:papers:2409.01147
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2409.01147
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Calvano, Emilio & Calzolari, Giacomo & Denicolò, Vincenzo & Pastorello, Sergio, 2023. "Algorithmic collusion: Genuine or spurious?," International Journal of Industrial Organization, Elsevier, vol. 90(C).
    2. Glenn Ellison, 2000. "Basins of Attraction, Long-Run Stochastic Stability, and the Speed of Step-by-Step Evolution," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 67(1), pages 17-45.
    3. Calvano, Emilio & Calzolari, Giacomo & Denicoló, Vincenzo & Pastorello, Sergio, 2021. "Algorithmic collusion with imperfect monitoring," International Journal of Industrial Organization, Elsevier, vol. 79(C).
    4. Dolgopolov, Arthur, 2024. "Reinforcement learning in a prisoner's dilemma," Games and Economic Behavior, Elsevier, vol. 144(C), pages 84-103.
    5. Kandori, Michihiro & Mailath, George J & Rob, Rafael, 1993. "Learning, Mutation, and Long Run Equilibria in Games," Econometrica, Econometric Society, vol. 61(1), pages 29-56, January.
    6. repec:esx:essedp:747 is not listed on IDEAS
    7. Werner, Tobias, 2023. "Algorithmic and Human Collusion," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277573, Verein für Socialpolitik / German Economic Association.
    8. Chaim Fershtman & Ariel Pakes, 2012. "Dynamic Games with Asymmetric Information: A Framework for Empirical Work," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(4), pages 1611-1661.
    9. Justin P. Johnson & Andrew Rhodes & Matthijs Wildenbeest, 2023. "Platform Design When Sellers Use Pricing Algorithms," Econometrica, Econometric Society, vol. 91(5), pages 1841-1879, September.
    10. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    11. Waltman, Ludo & Kaymak, Uzay, 2008. "Q-learning agents in a Cournot oligopoly model," Journal of Economic Dynamics and Control, Elsevier, vol. 32(10), pages 3275-3293, October.
    12. Jackson, Matthew O. & Watts, Alison, 2002. "The Evolution of Social and Economic Networks," Journal of Economic Theory, Elsevier, vol. 106(2), pages 265-295, October.
    13. Cui, Zhiwei & Zhai, Jian, 2010. "Escape dynamics and equilibria selection by iterative cycle decomposition," Journal of Mathematical Economics, Elsevier, vol. 46(6), pages 1015-1029, November.
    14. Staudigl, Mathias & Weidenholzer, Simon, 2014. "Constrained interactions and social coordination," Journal of Economic Theory, Elsevier, vol. 152(C), pages 41-63.
    15. Levine, David Knudsen & Modica, Salvatore, 2016. "Dynamics in stochastic evolutionary models," Theoretical Economics, Econometric Society, vol. 11(1), January.
    16. Joseph E Harrington, 2018. "Developing Competition Law For Collusion By Autonomous Artificial Agents," Journal of Competition Law and Economics, Oxford University Press, vol. 14(3), pages 331-363.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zexin Ye, 2025. "Algorithmic Collusion under Observed Demand Shocks," Papers 2502.15084, arXiv.org, revised Dec 2025.
    2. Christos Spyridon Koulouris & Carlo Campajola, 2026. "Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution," Papers 2605.20348, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jonathan Newton, 2018. "Evolutionary Game Theory: A Renaissance," Games, MDPI, vol. 9(2), pages 1-67, May.
    2. Dolgopolov, Arthur, 2024. "Reinforcement learning in a prisoner's dilemma," Games and Economic Behavior, Elsevier, vol. 144(C), pages 84-103.
    3. J. Manuel Sanchez-Cartas & Evangelos Katsamakas, 2025. "AI pricing algorithms under platform competition," Electronic Commerce Research, Springer, vol. 25(6), pages 4343-4370, December.
    4. Epivent, Andréa & Lambin, Xavier, 2024. "On algorithmic collusion and reward–punishment schemes," Economics Letters, Elsevier, vol. 237(C).
    5. John Asker & Chaim Fershtman & Ariel Pakes, 2024. "The impact of artificial intelligence design on pricing," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 33(2), pages 276-304, March.
    6. Cui, Zhiwei, 2023. "Linking friction, social coordination and the speed of evolution," Games and Economic Behavior, Elsevier, vol. 140(C), pages 410-430.
    7. Zexin Ye, 2025. "Algorithmic Collusion under Observed Demand Shocks," Papers 2502.15084, arXiv.org, revised Dec 2025.
    8. Levine, David K., 2025. "The evolution of resilience," Journal of Economic Theory, Elsevier, vol. 230(C).
    9. Simon Martin & Alexander Rasch, 2022. "Collusion by Algorithm: The Role of Unobserved Actions," CESifo Working Paper Series 9629, CESifo.
    10. Leonardo Boncinelli & Alessio Muscillo & Paolo Pin, 2022. "Correction to: Efficiency and Stability in a Process of Teams Formation," Dynamic Games and Applications, Springer, vol. 12(4), pages 1130-1130, December.
    11. Chengcheng Wang & Zexin Ye, 2026. "Strategic Information Disclosure in Algorithmic Pricing," Papers 2607.04345, arXiv.org.
    12. Soria, Jorge & Moya, Jorge & Mohazab, Amin, 2023. "Optimal mining in proof-of-work blockchain protocols," Finance Research Letters, Elsevier, vol. 53(C).
    13. Lucila Porto, 2022. "Q-Learning algorithms in a Hotelling model," Asociación Argentina de Economía Política: Working Papers 4587, Asociación Argentina de Economía Política.
    14. Bharat Bhole & Sunita Surana, 2025. "Tacit collusion by pricing algorithms," Economic Inquiry, Western Economic Association International, vol. 63(4), pages 1036-1065, October.
    15. Levine, David Knudsen & Modica, Salvatore, 2016. "Dynamics in stochastic evolutionary models," Theoretical Economics, Econometric Society, vol. 11(1), January.
    16. Newton, Jonathan & Angus, Simon D., 2015. "Coalitions, tipping points and the speed of evolution," Journal of Economic Theory, Elsevier, vol. 157(C), pages 172-187.
    17. Cui, Zhiwei & Liu, Jinhua, 2024. "Homophily in network formation and social coordination," Economics Letters, Elsevier, vol. 238(C).
    18. Levine, David K. & Modica, Salvatore, 2022. "Survival of the Weakest: Why the West Rules," Journal of Economic Behavior & Organization, Elsevier, vol. 204(C), pages 394-421.
    19. Marcel Wieting & Geza Sapi, 2021. "Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing in E-Commerce," Working Papers 21-06, NET Institute.
    20. Pin, Paolo & Weidenholzer, Elke & Weidenholzer, Simon, 2017. "Constrained mobility and the evolution of efficient outcomes," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 165-175.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.01147. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.