Advanced Search
MyIDEAS: Login

Attainability of boundary points under reinforcement learning

Contents:

Author Info

  • Hopkins, Ed
  • Posch, Martin

Abstract

This paper investigates the properties of the most common form of reinforcement learning (the "basic model" of Erev and Roth, American Economic Review, 88, 848-881, 1998). Stochastic approximation theory has been used to analyse the local stability of fixed points under this learning process. However, as we show, when such points are on the boundary of the state space, for example, pure strategy equilibria, standard results from the theory of stochastic approximation do not apply. We offer what we believe to be the correct treatment of boundary points, and provide a new and more general result: this model of learning converges with zero probability to fixed points which are unstable under the Maynard Smith or adjusted version of the evolutionary replicator dynamics. For two player games these are the fixed points that are linearly unstable under the standard replicator dynamics.

(This abstract was borrowed from another version of this item.)

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://www.sciencedirect.com/science/article/B6WFW-4DR1NP7-2/2/bde8a30a34051952920a1d2f08d1fe01
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal Games and Economic Behavior.

Volume (Year): 53 (2005)
Issue (Month): 1 (October)
Pages: 110-125

as in new window
Handle: RePEc:eee:gamebe:v:53:y:2005:i:1:p:110-125

Contact details of provider:
Web page: http://www.elsevier.com/locate/inca/622836

Related research

Keywords:

Other versions of this item:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Laslier, J.-F. & Topol, R. & Walliser, B., 1999. "A Behavioral Learning Process in Games," Papers 99-03, Paris X - Nanterre, U.F.R. de Sc. Ec. Gest. Maths Infor..
  2. Colin Camerer & Teck-Hua Ho, 1999. "Experience-weighted Attraction Learning in Normal Form Games," Econometrica, Econometric Society, vol. 67(4), pages 827-874, July.
  3. Sandholm, William H, 2002. "Evolutionary Implementation and Congestion Pricing," Review of Economic Studies, Wiley Blackwell, vol. 69(3), pages 667-89, July.
  4. Tilman B�rgers & Rajiv Sarin, . "Learning Through Reinforcement and Replicator Dynamics," ELSE working papers 051, ESRC Centre on Economics Learning and Social Evolution.
  5. Roth, Alvin E. & Erev, Ido, 1995. "Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term," Games and Economic Behavior, Elsevier, vol. 8(1), pages 164-212.
  6. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-81, September.
  7. Ed Hopkins, 2004. "Two Competing Models of How People Learn in Games," ESE Discussion Papers 51, Edinburgh School of Economics, University of Edinburgh.
  8. Martin Posch, 1997. "Cycling in a stochastic learning algorithm for normal form games," Journal of Evolutionary Economics, Springer, vol. 7(2), pages 193-207.
  9. Rustichini, Aldo, 1999. "Optimal Properties of Stimulus--Response Learning Models," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 244-273, October.
  10. Ellison, Glenn & Fudenberg, Drew, 2000. "Learning Purified Mixed Equilibria," Journal of Economic Theory, Elsevier, vol. 90(1), pages 84-115, January.
  11. Monderer, Dov & Shapley, Lloyd S., 1996. "Potential Games," Games and Economic Behavior, Elsevier, vol. 14(1), pages 124-143, May.
  12. Sarin, Rajiv & Vahid, Farshid, 1999. "Payoff Assessments without Probabilities: A Simple Dynamic Model of Choice," Games and Economic Behavior, Elsevier, vol. 28(2), pages 294-309, August.
  13. Arthur, W Brian, 1993. "On Designing Economic Agents That Behave Like Human Agents," Journal of Evolutionary Economics, Springer, vol. 3(1), pages 1-22, February.
  14. John Duffy & Ed Hopkins, 2004. "Learning, Information and Sorting in Market Entry Games: Theory and Evidence," ESE Discussion Papers 78, Edinburgh School of Economics, University of Edinburgh.
  15. Josef Hofbauer & Ed Hopkins, 2004. "Learning in Perturbed Asymmetric Games," ESE Discussion Papers 53, Edinburgh School of Economics, University of Edinburgh.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Ed Hopkins, 2004. "Adaptive Learning Models of Consumer Behaviour," ESE Discussion Papers 121, Edinburgh School of Economics, University of Edinburgh.
  2. Oyarzun, Carlos & Sarin, Rajiv, 2013. "Learning and risk aversion," Journal of Economic Theory, Elsevier, vol. 148(1), pages 196-225.
  3. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Gotts, Nicholas M. & Polhill, J. Gary, 2007. "Transient and asymptotic dynamics of reinforcement learning in games," Games and Economic Behavior, Elsevier, vol. 61(2), pages 259-276, November.
  4. Ianni, Antonella, 2014. "Learning strict Nash equilibria through reinforcement," Journal of Mathematical Economics, Elsevier, vol. 50(C), pages 148-155.
  5. Josephson, Jens, 2008. "A numerical analysis of the evolutionary stability of learning rules," Journal of Economic Dynamics and Control, Elsevier, vol. 32(5), pages 1569-1599, May.
  6. Duffy, John & Hopkins, Ed, 2005. "Learning, information, and sorting in market entry games: theory and evidence," Games and Economic Behavior, Elsevier, vol. 51(1), pages 31-62, April.
  7. Schuster, Stephan, 2010. "Network Formation with Adaptive Agents," MPRA Paper 27388, University Library of Munich, Germany.
  8. Fudenberg, Drew & Takahashi, Satoru, 2011. "Heterogeneous beliefs and local information in stochastic fictitious play," Games and Economic Behavior, Elsevier, vol. 71(1), pages 100-120, January.
  9. Carlos Oyarzun & Rajiv Sarin, 2012. "Learning and Risk Aversion," Levine's Working Paper Archive 786969000000000572, David K. Levine.
  10. Georgios Chasparis & Jeff Shamma, 2012. "Distributed Dynamic Reinforcement of Efficient Outcomes in Multiagent Coordination and Network Formation," Dynamic Games and Applications, Springer, vol. 2(1), pages 18-50, March.
  11. Conor Mayo-Wilson & Kevin Zollman & David Danks, 2013. "Wisdom of crowds versus groupthink: learning in groups and in isolation," International Journal of Game Theory, Springer, vol. 42(3), pages 695-723, August.
  12. Schuster, Stephan, 2012. "Applications in Agent-Based Computational Economics," MPRA Paper 47201, University Library of Munich, Germany.
  13. Leslie, David S. & Collins, E.J., 2006. "Generalised weakened fictitious play," Games and Economic Behavior, Elsevier, vol. 56(2), pages 285-298, August.
  14. Antonella Ianni, 2007. "Learning Strict Nash Equilibria through Reinforcement," Economics Working Papers ECO2007/21, European University Institute.
  15. Rokhaya Dieye & Habiba Djebbari & Felipe Barrera-Osorio, 2014. "Accounting for Peer Effects in Treatment Response," AMSE Working Papers 1335, Aix-Marseille School of Economics, Marseille, France, revised Jul 2014.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:53:y:2005:i:1:p:110-125. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.