IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2312.02644.html
   My bibliography  Save this paper

Spontaneous Coupling of Q-Learning Algorithms in Equilibrium

Author

Listed:
  • Ivan Conjeaud

Abstract

Most contributions in the algorithmic collusion literature only consider symmetric algorithms interacting with each other. We study a simple model of algorithmic collusion in which Q-learning algorithms repeatedly play a prisoner's dilemma and allow players to choose different exploration policies. We characterize behavior of such algorithms with asymmetric policies for extreme values and prove that any Nash equilibrium features some cooperative behavior. We further investigate the dynamics for general profiles of exploration policy by running extensive numerical simulations which indicate symmetry of equilibria, and give insight for their distribution.

Suggested Citation

  • Ivan Conjeaud, 2023. "Spontaneous Coupling of Q-Learning Algorithms in Equilibrium," Papers 2312.02644, arXiv.org.
    • Handle: RePEc:arx:papers:2312.02644
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2312.02644
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    2. Stephanie Assad & Robert Clark & Daniel Ershov & Lei Xu, 2020. "Algorithmic Pricing and Competition: Empirical Evidence from the German Retail Gasoline Market," CESifo Working Paper Series 8521, CESifo.
    3. Calzolari, Giacomo & Calvano, Emilio & Denicolo, Vincenzo & Pastorello, Sergio, 2021. "Algorithmic collusion with imperfect monitoring," CEPR Discussion Papers 15738, C.E.P.R. Discussion Papers.
    4. Martino Banchio & Andrzej Skrzypacz, 2022. "Artificial Intelligence and Auction Design," NBER Chapters, in: Economics of Artificial Intelligence, National Bureau of Economic Research, Inc.
    5. Matthias Hettich, 2021. "Algorithmic Collusion: Insights from Deep Learning," CQE Working Papers 9421, Center for Quantitative Economics (CQE), University of Muenster.
    6. Maskin, Eric & Tirole, Jean, 1988. "A Theory of Dynamic Oligopoly, II: Price Competition, Kinked Demand Curves, and Edgeworth Cycles," Econometrica, Econometric Society, vol. 56(3), pages 571-599, May.
    7. Johnson, Justin Pappas & Rhodes, Andrew & Wildenbeest, Matthijs, 2020. "Platform Design when Sellers Use Pricing Algorithms," TSE Working Papers 20-1146, Toulouse School of Economics (TSE).
    8. Harrington, Joseph E. , Jr., 2017. "The Theory of Collusion and Competition Policy," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262036932, December.
    9. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2019. "Algorithmic Pricing What Implications for Competition Policy?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 55(1), pages 155-171, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Martin, Simon & Rasch, Alexander, 2022. "Collusion by algorithm: The role of unobserved actions," DICE Discussion Papers 382, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    2. Gonzalo Ballestero, 2021. "Collusion and Artificial Intelligence: A computational experiment with sequential pricing algorithms under stochastic costs," Young Researchers Working Papers 1, Universidad de San Andres, Departamento de Economia, revised Oct 2022.
    3. Simon Martin & Alexander Rasch, 2022. "Collusion by Algorithm: The Role of Unobserved Actions," CESifo Working Paper Series 9629, CESifo.
    4. Gonzalo Ballestero, 2022. "Collusion and Artificial Intelligence: A Computational Experiment with Sequential Pricing Algorithms under Stochastic Costs," Working Papers 118, Red Nacional de Investigadores en Economía (RedNIE).
    5. Marcel Wieting & Geza Sapi, 2021. "Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing in E-Commerce," Working Papers 21-06, NET Institute.
    6. Norman, Thomas W.L., 2023. "Pigouvian algorithmic platform design," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 322-332.
    7. Werner, Tobias, 2023. "Algorithmic and Human Collusion," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277573, Verein für Socialpolitik / German Economic Association.
    8. Timo Klein, 2021. "Autonomous algorithmic collusion: Q‐learning under sequential pricing," RAND Journal of Economics, RAND Corporation, vol. 52(3), pages 538-558, September.
    9. Yiquan Gu & Leonardo Madio & Carlo Reggiani, 2019. "Exclusive Data, Price Manipulation and Market Leadership," CESifo Working Paper Series 7853, CESifo.
    10. Werner, Tobias, 2021. "Algorithmic and human collusion," DICE Discussion Papers 372, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    11. Calvano, Emilio & Calzolari, Giacomo & Denicoló, Vincenzo & Pastorello, Sergio, 2021. "Algorithmic collusion with imperfect monitoring," International Journal of Industrial Organization, Elsevier, vol. 79(C).
    12. Normann, Hans-Theo & Sternberg, Martin, 2023. "Human-algorithm interaction: Algorithmic pricing in hybrid laboratory markets," European Economic Review, Elsevier, vol. 152(C).
    13. W. Benedikt Schmal, 2022. "From Rules to Regs: A Structural Topic Model of Collusion Research," Papers 2210.02957, arXiv.org.
    14. Aleksandar B. Todorov, 2022. "Algorithmic pricing and concerted behaviour – competitive challenges?," Economic Thought journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 1, pages 90-107.
    15. Gonzalo Ballestero, 2021. "Collusion and Artificial Intelligence: A computational experiment with sequential pricing algorithms under stochastic costs," Asociación Argentina de Economía Política: Working Papers 4433, Asociación Argentina de Economía Política.
    16. Inkoo Cho & Noah Williams, 2024. "Collusive Outcomes Without Collusion," Papers 2403.07177, arXiv.org.
    17. Bingyan Han, 2021. "Understanding algorithmic collusion with experience replay," Papers 2102.09139, arXiv.org, revised Mar 2021.
    18. Martino Banchio & Giacomo Mantegazza, 2022. "Artificial Intelligence and Spontaneous Collusion," Papers 2202.05946, arXiv.org, revised Sep 2023.
    19. Drolsbach, Chiara Patricia & Gail, Maximilian Maurice & Klotz, Phil-Adrian, 2023. "Pass-through of Temporary Fuel Tax Reductions: Evidence from Europe," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277655, Verein für Socialpolitik / German Economic Association.
    20. Mark J. Tremblay, 2020. "The Limits of Marketplace Fee Discrimination," Working Papers 20-10, NET Institute.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2312.02644. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.