IDEAS home Printed from https://ideas.repec.org/a/inm/ormoor/v50y2025i1p459-481.html

Convergence and Stability of Coupled Belief-Strategy Learning Dynamics in Continuous Games

Author

Listed:
  • Manxi Wu

    (School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14850)

  • Saurabh Amin

    (Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Asuman Ozdaglar

    (Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

Abstract

We propose a learning dynamics to model how strategic agents repeatedly play a continuous game while relying on an information platform to learn an unknown payoff-relevant parameter. In each time step, the platform updates a belief estimate of the parameter based on players’ strategies and realized payoffs using Bayes’ rule. Then, players adopt a generic learning rule to adjust their strategies based on the updated belief. We present results on the convergence of beliefs and strategies and the properties of convergent fixed points of the dynamics. We obtain sufficient and necessary conditions for the existence of globally stable fixed points. We also provide sufficient conditions for the local stability of fixed points. These results provide an approach to analyzing the long-term outcomes that arise from the interplay between Bayesian belief learning and strategy learning in games and enable us to characterize conditions under which learning leads to a complete information equilibrium.

Suggested Citation

  • Manxi Wu & Saurabh Amin & Asuman Ozdaglar, 2025. "Convergence and Stability of Coupled Belief-Strategy Learning Dynamics in Continuous Games," Mathematics of Operations Research, INFORMS, vol. 50(1), pages 459-481, February.
    • Handle: RePEc:inm:ormoor:v:50:y:2025:i:1:p:459-481
    DOI: 10.1287/moor.2022.0161
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/moor.2022.0161
    Download Restriction: no
    File URL: https://libkey.io/10.1287/moor.2022.0161?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Wendy W. Moe & Peter S. Fader, 2004. "Dynamic Conversion Behavior at E-Commerce Sites," Management Science, INFORMS, vol. 50(3), pages 326-335, March.
    2. Cominetti, Roberto & Melo, Emerson & Sorin, Sylvain, 2010. "A payoff-based learning procedure and its application to traffic games," Games and Economic Behavior, Elsevier, vol. 70(1), pages 71-83, September.
    3. Gale, Douglas & Kariv, Shachar, 2003. "Bayesian learning in social networks," Games and Economic Behavior, Elsevier, vol. 45(2), pages 329-346, November.
    4. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    5. Manxi Wu & Saurabh Amin & Asuman E. Ozdaglar, 2021. "Value of Information in Bayesian Routing Games," Operations Research, INFORMS, vol. 69(1), pages 148-163, January.
    6. Fudenberg, Drew & Kreps, David M., 1995. "Learning in extensive-form games I. Self-confirming equilibria," Games and Economic Behavior, Elsevier, vol. 8(1), pages 20-55.
    7. Fudenberg, Drew & Levine, David K, 1993. "Self-Confirming Equilibrium," Econometrica, Econometric Society, vol. 61(3), pages 523-545, May.
    8. Blume Lawrence E., 1993. "The Statistical Mechanics of Strategic Interaction," Games and Economic Behavior, Elsevier, vol. 5(3), pages 387-424, July.
    9. Marden, Jason R. & Shamma, Jeff S., 2012. "Revisiting log-linear learning: Asynchrony, completeness and payoff-based implementation," Games and Economic Behavior, Elsevier, vol. 75(2), pages 788-808.
    10. Josef Hofbauer & William H. Sandholm, 2002. "On the Global Convergence of Stochastic Fictitious Play," Econometrica, Econometric Society, vol. 70(6), pages 2265-2294, November.
    11. Milgrom, Paul & Roberts, John, 1990. "Rationalizability, Learning, and Equilibrium in Games with Strategic Complementarities," Econometrica, Econometric Society, vol. 58(6), pages 1255-1277, November.
    12. , P. & , Peyton, 2006. "Regret testing: learning to play Nash equilibrium without knowing you have an opponent," Theoretical Economics, Econometric Society, vol. 1(3), pages 341-367, September.
    13. Fudenberg Drew & Kreps David M., 1993. "Learning Mixed Equilibria," Games and Economic Behavior, Elsevier, vol. 5(3), pages 320-367, July.
    14. Samuelson, Larry & Zhang, Jianbo, 1992. "Evolutionary stability in asymmetric games," Journal of Economic Theory, Elsevier, vol. 57(2), pages 363-391, August.
    15. Alós-Ferrer, Carlos & Netzer, Nick, 2010. "The logit-response dynamics," Games and Economic Behavior, Elsevier, vol. 68(2), pages 413-427, March.
    16. Benaim, Michel & Hirsch, Morris W., 1999. "Mixed Equilibria and Dynamical Systems Arising from Fictitious Play in Perturbed Games," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 36-72, October.
    17. Hofbauer, Josef & Sandholm, William H., 2009. "Stable games and their dynamics," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1665-1693.4, July.
    18. Elchanan Mossel & Allan Sly & Omer Tamuz, 2015. "Strategic Learning and the Topology of Social Networks," Econometrica, Econometric Society, vol. 83(5), pages 1755-1794, September.
    19. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.
    20. Matsui, Akihiko, 1992. "Best response dynamics and socially stable strategies," Journal of Economic Theory, Elsevier, vol. 57(2), pages 343-362, August.
    21. Abhijit V. Banerjee, 1992. "A Simple Model of Herd Behavior," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 797-817.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Manxi Wu & Saurabh Amin & Asuman Ozdaglar, 2021. "Multi-agent Bayesian Learning with Best Response Dynamics: Convergence and Stability," Papers 2109.00719, arXiv.org.
    2. Jonathan Newton, 2018. "Evolutionary Game Theory: A Renaissance," Games, MDPI, vol. 9(2), pages 1-67, May.
    3. Jakub Bielawski & Thiparat Chotibut & Fryderyk Falniowski & Michal Misiurewicz & Georgios Piliouras, 2022. "Unpredictable dynamics in congestion games: memory loss can prevent chaos," Papers 2201.10992, arXiv.org, revised Jan 2022.
    4. Sandholm, William H., 2015. "Population Games and Deterministic Evolutionary Dynamics," Handbook of Game Theory with Economic Applications,, Elsevier.
    5. Sobel, Joel, 2000. "Economists' Models of Learning," Journal of Economic Theory, Elsevier, vol. 94(2), pages 241-261, October.
    6. Naoki Funai, 2019. "Convergence results on stochastic adaptive learning," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 68(4), pages 907-934, November.
    7. Michel Benaïm & Josef Hofbauer & Sylvain Sorin, 2012. "Perturbations of Set-Valued Dynamical Systems, with Applications to Game Theory," Dynamic Games and Applications, Springer, vol. 2(2), pages 195-205, June.
    8. Hofbauer, Josef & Hopkins, Ed, 2005. "Learning in perturbed asymmetric games," Games and Economic Behavior, Elsevier, vol. 52(1), pages 133-152, July.
    9. Funai, Naoki, 2022. "Reinforcement learning with foregone payoff information in normal form games," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 638-660.
    10. Ewerhart, Christian & Valkanova, Kremena, 2020. "Fictitious play in networks," Games and Economic Behavior, Elsevier, vol. 123(C), pages 182-206.
    11. Andriy Zapechelnyuk, 2009. "Limit Behavior of No-regret Dynamics," Discussion Papers 21, Kyiv School of Economics.
    12. Fudenberg, Drew & Takahashi, Satoru, 2011. "Heterogeneous beliefs and local information in stochastic fictitious play," Games and Economic Behavior, Elsevier, vol. 71(1), pages 100-120, January.
    13. Juan I Block & Drew Fudenberg & David K Levine, 2017. "Learning Dynamics Based on Social Comparisons," Levine's Working Paper Archive 786969000000001375, David K. Levine.
    14. Mario Bravo, 2016. "An Adjusted Payoff-Based Procedure for Normal Form Games," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1469-1483, November.
    15. Block, Juan I. & Fudenberg, Drew & Levine, David K., 2019. "Learning dynamics with social comparisons and limited memory," Theoretical Economics, Econometric Society, vol. 14(1), January.
    16. Ignacio Esponda & Demian Pouzo, 2016. "Berk–Nash Equilibrium: A Framework for Modeling Agents With Misspecified Models," Econometrica, Econometric Society, vol. 84, pages 1093-1130, May.
    17. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    18. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.
    19. Dai Zusai, 2023. "Evolutionary dynamics in heterogeneous populations: a general framework for an arbitrary type distribution," International Journal of Game Theory, Springer;Game Theory Society, vol. 52(4), pages 1215-1260, December.
    20. Cason, Timothy N. & Friedman, Daniel & Hopkins, Ed, 2010. "Testing the TASP: An experimental investigation of learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 145(6), pages 2309-2331, November.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormoor:v:50:y:2025:i:1:p:459-481. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.