IDEAS home Printed from https://ideas.repec.org/a/eee/jetheo/v177y2018icp315-364.html
   My bibliography  Save this article

Riemannian game dynamics

Author

Listed:
  • Mertikopoulos, Panayotis
  • Sandholm, William H.

Abstract

We study a class of evolutionary game dynamics defined by balancing a gain determined by the game's payoffs against a cost of motion that captures the difficulty with which the population moves between states. Costs of motion are represented by a Riemannian metric, i.e., a state-dependent inner product on the set of population states. The replicator dynamics and the (Euclidean) projection dynamics are the archetypal examples of the class we study. Like these representative dynamics, all Riemannian game dynamics satisfy certain basic desiderata, including positive correlation, local stability of interior ESSs, and global convergence in potential games. When the underlying Riemannian metric satisfies a Hessian integrability condition, the resulting dynamics preserve many further properties of the replicator and projection dynamics. We examine the close connections between Hessian game dynamics and reinforcement learning in normal form games, extending and elucidating a well-known link between the replicator dynamics and exponential reinforcement learning.

Suggested Citation

  • Mertikopoulos, Panayotis & Sandholm, William H., 2018. "Riemannian game dynamics," Journal of Economic Theory, Elsevier, vol. 177(C), pages 315-364.
  • Handle: RePEc:eee:jetheo:v:177:y:2018:i:c:p:315-364
    DOI: 10.1016/j.jet.2018.06.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0022053118302679
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jet.2018.06.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fudenberg, Drew & Levine, David, 1998. "Learning in games," European Economic Review, Elsevier, vol. 42(3-5), pages 631-639, May.
    2. Terry L. Friesz & David Bernstein & Nihal J. Mehta & Roger L. Tobin & Saiid Ganjalizadeh, 1994. "Day-To-Day Dynamic Network Disequilibria and Idealized Traveler Information Systems," Operations Research, INFORMS, vol. 42(6), pages 1120-1136, December.
    3. repec:dau:papers:123456789/1014 is not listed on IDEAS
    4. Sergiu Hart & Andreu Mas-Colell, 2013. "A General Class Of Adaptive Strategies," World Scientific Book Chapters, in: Simple Adaptive Strategies From Regret-Matching to Uncoupled Dynamics, chapter 3, pages 47-76, World Scientific Publishing Co. Pte. Ltd..
    5. Sandholm, William H. & DokumacI, Emin & Lahkar, Ratul, 2008. "The projection dynamic and the replicator dynamic," Games and Economic Behavior, Elsevier, vol. 64(2), pages 666-683, November.
    6. Sandholm, William H., 2015. "Population Games and Deterministic Evolutionary Dynamics," Handbook of Game Theory with Economic Applications,, Elsevier.
    7. Martin Posch, 1997. "Cycling in a stochastic learning algorithm for normal form games," Journal of Evolutionary Economics, Springer, vol. 7(2), pages 193-207.
    8. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.
    9. Josef Hofbauer & William H. Sandholm, 2002. "On the Global Convergence of Stochastic Fictitious Play," Econometrica, Econometric Society, vol. 70(6), pages 2265-2294, November.
    10. Michel BenaÔm & J–rgen W. Weibull, 2003. "Deterministic Approximation of Stochastic Evolution in Games," Econometrica, Econometric Society, vol. 71(3), pages 873-903, May.
    11. Borgers, Tilman & Sarin, Rajiv, 1997. "Learning Through Reinforcement and Replicator Dynamics," Journal of Economic Theory, Elsevier, vol. 77(1), pages 1-14, November.
    12. Anna Nagurney & Ding Zhang, 1997. "Projected Dynamical Systems in the Formulation, Stability Analysis, and Computation of Fixed-Demand Traffic Network Equilibria," Transportation Science, INFORMS, vol. 31(2), pages 147-158, May.
    13. Tsakas, Elias & Voorneveld, Mark, 2009. "The target projection dynamic," Games and Economic Behavior, Elsevier, vol. 67(2), pages 708-719, November.
    14. Bravo, Mario & Mertikopoulos, Panayotis, 2017. "On the robustness of learning in games with stochastically perturbed payoff observations," Games and Economic Behavior, Elsevier, vol. 103(C), pages 41-66.
    15. William Sandholm, 2014. "Probabilistic Interpretations of Integrability for Game Dynamics," Dynamic Games and Applications, Springer, vol. 4(1), pages 95-106, March.
    16. Sandholm, William H., 2001. "Potential Games with Continuous Player Sets," Journal of Economic Theory, Elsevier, vol. 97(1), pages 81-108, March.
    17. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945, December.
    18. Swinkels Jeroen M., 1993. "Adjustment Dynamics and Rational Play in Games," Games and Economic Behavior, Elsevier, vol. 5(3), pages 455-484, July.
    19. Hofbauer, Josef & Sandholm, William H., 2007. "Evolution in games with randomly disturbed payoffs," Journal of Economic Theory, Elsevier, vol. 132(1), pages 47-69, January.
    20. Sandholm, William H., 2005. "Excess payoff dynamics and other well-behaved evolutionary dynamics," Journal of Economic Theory, Elsevier, vol. 124(2), pages 149-170, October.
    21. Hopkins, Ed, 1999. "A Note on Best Response Dynamics," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 138-150, October.
    22. , H., 2010. "Local stability under evolutionary game dynamics," Theoretical Economics, Econometric Society, vol. 5(1), January.
    23. Rustichini, Aldo, 1999. "Optimal Properties of Stimulus--Response Learning Models," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 244-273, October.
    24. Schlag, Karl H., 1998. "Why Imitate, and If So, How?, : A Boundedly Rational Approach to Multi-armed Bandits," Journal of Economic Theory, Elsevier, vol. 78(1), pages 130-156, January.
    25. Pierre Coucheney & Bruno Gaujal & Panayotis Mertikopoulos, 2015. "Penalty-Regulated Dynamics and Robust Learning Procedures in Games," Mathematics of Operations Research, INFORMS, vol. 40(3), pages 611-633, March.
    26. Friedman, Daniel, 1991. "Evolutionary Games in Economics," Econometrica, Econometric Society, vol. 59(3), pages 637-666, May.
    27. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    28. Josef Hofbauer & Sylvain Sorin & Yannick Viossat, 2009. "Time Average Replicator and Best-Reply Dynamics," Mathematics of Operations Research, INFORMS, vol. 34(2), pages 263-269, May.
    29. Dai Zusai, 2018. "Net gains in evolutionary dynamics: A unifying and intuitive approach to dynamic stability," Papers 1805.04898, arXiv.org, revised Oct 2023.
    30. Lahkar, Ratul & Sandholm, William H., 2008. "The projection dynamic and the geometry of population games," Games and Economic Behavior, Elsevier, vol. 64(2), pages 565-590, November.
    31. D. Zhang & A. Nagurney, 1997. "Formulation, Stability, and Computation of Traffic Network Equilibria as Projected Dynamical Systems," Journal of Optimization Theory and Applications, Springer, vol. 93(2), pages 417-444, May.
    32. Demichelis, Stefano & Ritzberger, Klaus, 2003. "From evolutionary to strategic stability," Journal of Economic Theory, Elsevier, vol. 113(1), pages 51-75, November.
    33. Hofbauer, Josef & Sandholm, William H., 2009. "Stable games and their dynamics," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1665-1693.4, July.
    34. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Sandholm, William H., 2019. "An introduction to ABED: Agent-based simulation of evolutionary game dynamics," Games and Economic Behavior, Elsevier, vol. 118(C), pages 434-462.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Srinivas Arigapudi & Omer Edhan & Yuval Heller & Ziv Hellman, 2022. "Mentors and Recombinators: Multi-Dimensional Social Learning," Papers 2205.00278, arXiv.org, revised Nov 2023.
    2. Takashi Akamatsu & Tomoya Mori & Minoru Osawa & Yuki Takayama, 2019. "Multimodal agglomeration in economic geography," Papers 1912.05113, arXiv.org, revised Apr 2023.
    3. Sylvain Sorin, 2023. "Continuous Time Learning Algorithms in Optimization and Game Theory," Dynamic Games and Applications, Springer, vol. 13(1), pages 3-24, March.
    4. Saeed Hadikhanloo & Rida Laraki & Panayotis Mertikopoulos & Sylvain Sorin, 2022. "Learning in nonatomic games, part Ⅰ: Finite action spaces and population games," Post-Print hal-03767995, HAL.
    5. Minoru Osawa & Takashi Akamatsu & Yosuke Kogure, 2020. "Stochastic stability of agglomeration patterns in an urban retail model," Papers 2011.06778, arXiv.org.
    6. Osawa, Minoru & Akamatsu, Takashi, 2020. "Equilibrium refinement for a model of non-monocentric internal structures of cities: A potential game approach," Journal of Economic Theory, Elsevier, vol. 187(C).
    7. Jakub Bielawski & Thiparat Chotibut & Fryderyk Falniowski & Michal Misiurewicz & Georgios Piliouras, 2022. "Unpredictable dynamics in congestion games: memory loss can prevent chaos," Papers 2201.10992, arXiv.org, revised Jan 2022.
    8. Chongyi Zhong & Hui Yang & Zixin Liu & Juanyong Wu, 2020. "Stability of Replicator Dynamics with Bounded Continuously Distributed Time Delay," Mathematics, MDPI, vol. 8(3), pages 1-12, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sandholm, William H., 2015. "Population Games and Deterministic Evolutionary Dynamics," Handbook of Game Theory with Economic Applications,, Elsevier.
    2. Lahkar, Ratul & Sandholm, William H., 2008. "The projection dynamic and the geometry of population games," Games and Economic Behavior, Elsevier, vol. 64(2), pages 565-590, November.
    3. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    4. Jonathan Newton, 2018. "Evolutionary Game Theory: A Renaissance," Games, MDPI, vol. 9(2), pages 1-67, May.
    5. Hofbauer, Josef & Sandholm, William H., 2009. "Stable games and their dynamics," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1665-1693.4, July.
    6. Tsakas, Elias & Voorneveld, Mark, 2009. "The target projection dynamic," Games and Economic Behavior, Elsevier, vol. 67(2), pages 708-719, November.
    7. Sandholm,W.H., 2003. "Excess payoff dynamics, potential dynamics, and stable games," Working papers 5, Wisconsin Madison - Social Systems.
    8. Sylvain Sorin, 2023. "Continuous Time Learning Algorithms in Optimization and Game Theory," Dynamic Games and Applications, Springer, vol. 13(1), pages 3-24, March.
    9. Saeed Hadikhanloo & Rida Laraki & Panayotis Mertikopoulos & Sylvain Sorin, 2022. "Learning in nonatomic games, part Ⅰ: Finite action spaces and population games," Post-Print hal-03767995, HAL.
    10. Reinoud Joosten & Berend Roorda, 2008. "Generalized projection dynamics in evolutionary game theory," Papers on Economics and Evolution 2008-11, Philipps University Marburg, Department of Geography.
    11. Sandholm,W.H., 2002. "Potential dynamics and stable games," Working papers 21, Wisconsin Madison - Social Systems.
    12. Reinoud Joosten, 2009. "Paul Samuelson's critique and equilibrium concepts in evolutionary game theory," Papers on Economics and Evolution 2009-16, Philipps University Marburg, Department of Geography.
    13. Sandholm, William H., 2005. "Excess payoff dynamics and other well-behaved evolutionary dynamics," Journal of Economic Theory, Elsevier, vol. 124(2), pages 149-170, October.
    14. Reinoud Joosten & Berend Roorda, 2011. "On evolutionary ray-projection dynamics," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 74(2), pages 147-161, October.
    15. Sylvain Sorin & Cheng Wan, 2016. "Finite composite games: Equilibria and dynamics," Post-Print hal-02885860, HAL.
    16. William Sandholm, 2014. "Probabilistic Interpretations of Integrability for Game Dynamics," Dynamic Games and Applications, Springer, vol. 4(1), pages 95-106, March.
    17. Ratul Lahkar & Sayan Mukherjee & Souvik Roy, 2022. "A Deterministic Approximation Approach to the Continuum Logit Dynamic with an Application to Supermodular Games," Working Papers 79, Ashoka University, Department of Economics.
    18. Bravo, Mario & Mertikopoulos, Panayotis, 2017. "On the robustness of learning in games with stochastically perturbed payoff observations," Games and Economic Behavior, Elsevier, vol. 103(C), pages 41-66.
    19. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Sandholm, William H., 2019. "An introduction to ABED: Agent-based simulation of evolutionary game dynamics," Games and Economic Behavior, Elsevier, vol. 118(C), pages 434-462.
    20. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.

    More about this item

    Keywords

    Evolutionary game theory; Learning in games; Projection dynamics; Riemannian metrics; Replicator dynamics; Reinforcement learning;
    All these keywords.

    JEL classification:

    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jetheo:v:177:y:2018:i:c:p:315-364. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/622869 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.