IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article or follow this journal

Reinforcement learning in population games

  • Lahkar, Ratul
  • Seymour, Robert M.
Registered author(s):

    We study reinforcement learning in a population game. Agents in a population game revise mixed strategies using the Cross rule of reinforcement learning. The population state—the probability distribution over the set of mixed strategies—evolves according to the replicator continuity equation which, in its simplest form, is a partial differential equation. The replicator dynamic is a special case in which the initial population state is homogeneous, i.e. when all agents use the same mixed strategy. We apply the continuity dynamic to various classes of symmetric games. Using 3×3 coordination games, we show that equilibrium selection depends on the variance of the initial strategy distribution, or initial population heterogeneity. We give an example of a 2×2 game in which heterogeneity persists even as the mean population state converges to a mixed equilibrium. Finally, we apply the dynamic to negative definite and doubly symmetric games.

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

    File URL: http://www.sciencedirect.com/science/article/pii/S0899825613000286
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

    Article provided by Elsevier in its journal Games and Economic Behavior.

    Volume (Year): 80 (2013)
    Issue (Month): C ()
    Pages: 10-38

    as
    in new window

    Handle: RePEc:eee:gamebe:v:80:y:2013:i:c:p:10-38
    Contact details of provider: Web page: http://www.elsevier.com/locate/inca/622836

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

    as in new window
    1. Friedman, Daniel & Ostrov, Daniel N., 2010. "Gradient dynamics in population games: Some basic results," Journal of Mathematical Economics, Elsevier, vol. 46(5), pages 691-707, September.
    2. Friedman, Daniel & Ostrov, Daniel N., 2008. "Conspicuous consumption dynamics," Games and Economic Behavior, Elsevier, vol. 64(1), pages 121-145, September.
    3. Ely, Jeffrey C. & Sandholm, William H., 2005. "Evolution in Bayesian games I: Theory," Games and Economic Behavior, Elsevier, vol. 53(1), pages 83-109, October.
    4. Hopkins, E., 1995. "Learning, Matching and Aggregation," G.R.E.Q.A.M. 95a20, Universite Aix-Marseille III.
    5. Tilman Borgers & Antonio Morales & Rajiv Sarin, 2010. "Expedient and Monotone Learning Rules," Levine's Working Paper Archive 625018000000000099, David K. Levine.
    6. T. Borgers & R. Sarin, 2010. "Learning Through Reinforcement and Replicator Dynamics," Levine's Working Paper Archive 380, David K. Levine.
    7. Ramsza, Michal & Seymour, Robert M., 2010. "Fictitious play in an evolutionary environment," Games and Economic Behavior, Elsevier, vol. 68(1), pages 303-324, January.
    8. Fudenberg, D. & Kreps, D.M., 1992. "Learning Mixed Equilibria," Working papers 92-13, Massachusetts Institute of Technology (MIT), Department of Economics.
    9. Sandholm, William H., 2001. "Potential Games with Continuous Player Sets," Journal of Economic Theory, Elsevier, vol. 97(1), pages 81-108, March.
    10. Drew Fudenberg & Satoru Takahashi, 2008. "Heterogeneous Beliefs and Local Information in Stochastic Fictitious Play," Levine's Working Paper Archive 122247000000001695, David K. Levine.
    11. Tilman B�rgers & Rajiv Sarin, . "Naive Reinforcement Learning With Endogenous Aspiration," ELSE working papers 037, ESRC Centre on Economics Learning and Social Evolution.
    12. Sergiu Hart & Andreu Mas-Colell, 2000. "A Simple Adaptive Procedure Leading to Correlated Equilibrium," Econometrica, Econometric Society, vol. 68(5), pages 1127-1150, September.
    13. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-81, September.
    14. Hofbauer, Josef & Sandholm, William H., 2009. "Stable games and their dynamics," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1665-1693.e, July.
    15. Josef Hofbauer & Sylvain Sorin & Yannick Viossat, 2009. "Time Average Replicator and Best Reply Dynamics," Post-Print hal-00360767, HAL.
    16. Glenn Ellison & Drew Fudenberg, 1998. "Learning Purified Mixed Equilibria," Harvard Institute of Economic Research Working Papers 1817, Harvard - Institute of Economic Research.
    Full references (including those not matched with items on IDEAS)

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:80:y:2013:i:c:p:10-38. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.