IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1704.05276.html
   My bibliography  Save this paper

Best reply structure and equilibrium convergence in generic games

Author

Listed:
  • Marco Pangallo
  • Torsten Heinrich
  • J Doyne Farmer

Abstract

Game theory is widely used as a behavioral model for strategic interactions in biology and social science. It is common practice to assume that players quickly converge to an equilibrium, e.g. a Nash equilibrium. This can be studied in terms of best reply dynamics, in which each player myopically uses the best response to her opponent's last move. Existing research shows that convergence can be problematic when there are best reply cycles. Here we calculate how typical this is by studying the space of all possible two-player normal form games and counting the frequency of best reply cycles. The two key parameters are the number of moves, which defines how complicated the game is, and the anti-correlation of the payoffs, which determines how competitive it is. We find that as games get more complicated and more competitive, best reply cycles become dominant. The existence of best reply cycles predicts non-convergence of six different learning algorithms that have support from human experiments. Our results imply that for complicated and competitive games equilibrium is typically an unrealistic assumption. Alternatively, if for some reason "real" games are special and do not possess cycles, we raise the interesting question of why this should be so.

Suggested Citation

  • Marco Pangallo & Torsten Heinrich & J Doyne Farmer, 2017. "Best reply structure and equilibrium convergence in generic games," Papers 1704.05276, arXiv.org, revised Sep 2018.
  • Handle: RePEc:arx:papers:1704.05276
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1704.05276
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    2. Crawford, Vincent P, 1974. "Learning the Optimal Strategy in a Zero-Sum Game," Econometrica, Econometric Society, vol. 42(5), pages 885-891, September.
    3. Borgers, Tilman & Sarin, Rajiv, 1997. "Learning Through Reinforcement and Replicator Dynamics," Journal of Economic Theory, Elsevier, vol. 77(1), pages 1-14, November.
    4. Colin Camerer & Teck-Hua Ho, 1999. "Experience-weighted Attraction Learning in Normal Form Games," Econometrica, Econometric Society, vol. 67(4), pages 827-874, July.
    5. Conlisk, John, 1993. "Adaptation in games : Two solutions to the Crawford puzzle," Journal of Economic Behavior & Organization, Elsevier, vol. 22(1), pages 25-50, September.
    6. Marco Pangallo & James Sanders & Tobias Galla & Doyne Farmer, 2017. "Towards a taxonomy of learning dynamics in 2 x 2 games," Papers 1701.09043, arXiv.org, revised Sep 2021.
    7. Nagel, Rosemarie, 1995. "Unraveling in Guessing Games: An Experimental Study," American Economic Review, American Economic Association, vol. 85(5), pages 1313-1326, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Collins, Sean M. & James, Duncan & Servátka, Maroš & Vadovič, Radovan, 2021. "Attainment of equilibrium via Marshallian path adjustment: Queueing and buyer determinism," Games and Economic Behavior, Elsevier, vol. 125(C), pages 94-106.
    2. Ben Amiet & Andrea Collevecchio & Marco Scarsini & Ziwen Zhong, 2019. "Pure Nash Equilibria and Best-Response Dynamics in Random Games," Papers 1905.10758, arXiv.org, revised Jun 2020.
    3. Samuel C. Wiese & Torsten Heinrich, 2020. "The Frequency of Convergent Games under Best-Response Dynamics," Papers 2011.01052, arXiv.org.
    4. Collins, Sean M. & James, Duncan & Servátka, Maroš & Vadovič, Radovan, 2020. "Attainment of Equilibrium: Marshallian Path Adjustment and Buyer Determinism," MPRA Paper 104103, University Library of Munich, Germany.
    5. Pangallo, Marco & Heinrich, Torsten & Jang, Yoojin & Scott, Alex & Tarbush, Bassel & Wiese, Samuel & Mungo, Luca, 2021. "Best-Response Dynamics, Playing Sequences, And Convergence To Equilibrium In Random Games," INET Oxford Working Papers 2021-23, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    6. Torsten Heinrich & Yoojin Jang & Luca Mungo & Marco Pangallo & Alex Scott & Bassel Tarbush & Samuel Wiese, 2021. "Best-response dynamics, playing sequences, and convergence to equilibrium in random games," Papers 2101.04222, arXiv.org, revised Nov 2021.
    7. Heinrich, Torsten & Wiese, Samuel, 2020. "The Frequency of Convergent Games under Best-Response Dynamics," INET Oxford Working Papers 2020-24, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    8. Jakub Bielawski & Thiparat Chotibut & Fryderyk Falniowski & Michal Misiurewicz & Georgios Piliouras, 2022. "Unpredictable dynamics in congestion games: memory loss can prevent chaos," Papers 2201.10992, arXiv.org, revised Jan 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Garcia-Pola, Bernardo & Iriberri, Nagore, 2019. "Naivete and Sophistication in Initial and Repeated Play in Games," CEPR Discussion Papers 14088, C.E.P.R. Discussion Papers.
    2. Pangallo, Marco & Farmer, J. Doyne & Sanders, James & Galla, Tobias, 2017. "A taxonomy of learning dynamics in 2 × 2 games," INET Oxford Working Papers 2017-06, Institute for New Economic Thinking at the Oxford Martin School, University of Oxford.
    3. Antonio Cabrales & Rosemarie Nagel & Roc Armenter, 2007. "Equilibrium selection through incomplete information in coordination games: an experimental study," Experimental Economics, Springer;Economic Science Association, vol. 10(3), pages 221-234, September.
    4. Atanasios Mitropoulos, 2001. "Learning Under Little Information: An Experiment on Mutual Fate Control," Game Theory and Information 0110003, University Library of Munich, Germany.
    5. Ianni, A., 2002. "Reinforcement learning and the power law of practice: some analytical results," Discussion Paper Series In Economics And Econometrics 203, Economics Division, School of Social Sciences, University of Southampton.
    6. Chernov, G. & Susin, I., 2019. "Models of learning in games: An overview," Journal of the New Economic Association, New Economic Association, vol. 44(4), pages 77-125.
    7. Mariano Runco, 2013. "Estimating depth of reasoning in a repeated guessing game with no feedback," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 402-413, September.
    8. Marchiori, Davide & Di Guida, Sibilla & Polonio, Luca, 2021. "Plasticity of strategic sophistication in interactive decision-making," Journal of Economic Theory, Elsevier, vol. 196(C).
    9. Jacob K. Goeree & Charles A. Holt, 2001. "Ten Little Treasures of Game Theory and Ten Intuitive Contradictions," American Economic Review, American Economic Association, vol. 91(5), pages 1402-1422, December.
    10. Breitmoser, Yves, 2019. "Knowing me, imagining you: Projection and overbidding in auctions," Games and Economic Behavior, Elsevier, vol. 113(C), pages 423-447.
    11. Trabelsi, Emna & Hichri, Walid, 2021. "Central Bank Transparency with (semi-)public Information: Laboratory Experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 90(C).
    12. Sarin, Rajiv & Vahid, Farshid, 2001. "Predicting How People Play Games: A Simple Dynamic Model of Choice," Games and Economic Behavior, Elsevier, vol. 34(1), pages 104-122, January.
    13. Giovanna Devetag, 2000. "Coordination in "Critical Mass" Games: An Experimental Study," LEM Papers Series 2000/03, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    14. Mohlin, Erik & Östling, Robert & Wang, Joseph Tao-yi, 2020. "Learning by similarity-weighted imitation in winner-takes-all games," Games and Economic Behavior, Elsevier, vol. 120(C), pages 225-245.
    15. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.
    16. Daskalova, Vessela & Vriend, Nicolaas J., 2021. "Learning frames," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 78-96.
    17. Torsten Heinrich & Yoojin Jang & Luca Mungo & Marco Pangallo & Alex Scott & Bassel Tarbush & Samuel Wiese, 2021. "Best-response dynamics, playing sequences, and convergence to equilibrium in random games," Papers 2101.04222, arXiv.org, revised Nov 2021.
    18. Hopkins, Ed & Posch, Martin, 2005. "Attainability of boundary points under reinforcement learning," Games and Economic Behavior, Elsevier, vol. 53(1), pages 110-125, October.
    19. Cardella, Eric, 2012. "Learning to make better strategic decisions," Journal of Economic Behavior & Organization, Elsevier, vol. 84(1), pages 382-392.
    20. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.

    More about this item

    JEL classification:

    • C62 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Existence and Stability Conditions of Equilibrium
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1704.05276. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.