IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this article

Reinforcement learning in population games

  • Lahkar, Ratul
  • Seymour, Robert M.

We study reinforcement learning in a population game. Agents in a population game revise mixed strategies using the Cross rule of reinforcement learning. The population state—the probability distribution over the set of mixed strategies—evolves according to the replicator continuity equation which, in its simplest form, is a partial differential equation. The replicator dynamic is a special case in which the initial population state is homogeneous, i.e. when all agents use the same mixed strategy. We apply the continuity dynamic to various classes of symmetric games. Using 3×3 coordination games, we show that equilibrium selection depends on the variance of the initial strategy distribution, or initial population heterogeneity. We give an example of a 2×2 game in which heterogeneity persists even as the mean population state converges to a mixed equilibrium. Finally, we apply the dynamic to negative definite and doubly symmetric games.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/pii/S0899825613000286
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Games and Economic Behavior.

Volume (Year): 80 (2013)
Issue (Month): C ()
Pages: 10-38

as
in new window

Handle: RePEc:eee:gamebe:v:80:y:2013:i:c:p:10-38
DOI: 10.1016/j.geb.2013.02.006
Contact details of provider: Web page: http://www.elsevier.com/locate/inca/622836

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Ed Hopkins, . "Learning, Matching and Aggregation," Department of Economics 1996 : II, Edinburgh School of Economics, University of Edinburgh.
  2. Fudenberg Drew & Kreps David M., 1993. "Learning Mixed Equilibria," Games and Economic Behavior, Elsevier, vol. 5(3), pages 320-367, July.
  3. Sandholm, William H., 2001. "Potential Games with Continuous Player Sets," Journal of Economic Theory, Elsevier, vol. 97(1), pages 81-108, March.
  4. Ramsza, Michal & Seymour, Robert M., 2010. "Fictitious play in an evolutionary environment," Games and Economic Behavior, Elsevier, vol. 68(1), pages 303-324, January.
  5. Tilman Borgers & Antonio Morales & Rajiv Sarin, 2010. "Expedient and Monotone Learning Rules," Levine's Working Paper Archive 625018000000000099, David K. Levine.
  6. Borgers, Tilman & Sarin, Rajiv, 1997. "Learning Through Reinforcement and Replicator Dynamics," Journal of Economic Theory, Elsevier, vol. 77(1), pages 1-14, November.
  7. S. Hart & A. Mas-Collel, 2010. "A Simple Adaptive Procedure Leading to Correlated Equilibrium," Levine's Working Paper Archive 572, David K. Levine.
  8. Friedman, Daniel & Ostrov, Daniel N., 2010. "Gradient dynamics in population games: Some basic results," Journal of Mathematical Economics, Elsevier, vol. 46(5), pages 691-707, September.
  9. Borgers, Tilman & Sarin, Rajiv, 2000. "Naive Reinforcement Learning with Endogenous Aspirations," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 41(4), pages 921-50, November.
  10. Fudenberg, Drew & Takahashi, Satoru, 2011. "Heterogeneous beliefs and local information in stochastic fictitious play," Games and Economic Behavior, Elsevier, vol. 71(1), pages 100-120, January.
  11. Ely, Jeffrey C. & Sandholm, William H., 2005. "Evolution in Bayesian games I: Theory," Games and Economic Behavior, Elsevier, vol. 53(1), pages 83-109, October.
  12. Takahashi, Satoru & Fudenberg, Drew, 2011. "Heterogeneous beliefs and local information in stochastic fictitious play," Scholarly Articles 27755310, Harvard University Department of Economics.
  13. Friedman, Daniel & Ostrov, Daniel N., 2008. "Conspicuous consumption dynamics," Games and Economic Behavior, Elsevier, vol. 64(1), pages 121-145, September.
  14. Glenn Ellison & Drew Fudenberg, 1998. "Learning Purified Mixed Equilibria," Harvard Institute of Economic Research Working Papers 1817, Harvard - Institute of Economic Research.
  15. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-81, September.
  16. repec:dau:papers:123456789/1014 is not listed on IDEAS
  17. Josef Hofbauer & Sylvain Sorin & Yannick Viossat, 2009. "Time Average Replicator and Best Reply Dynamics," Post-Print hal-00360767, HAL.
  18. Hofbauer, Josef & Sandholm, William H., 2009. "Stable games and their dynamics," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1665-1693.e, July.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:80:y:2013:i:c:p:10-38. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Shamier, Wendy)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.