IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-01457301.html
   My bibliography  Save this paper

Reinforcement Learning with Restrictions on the Action Set

Author

Listed:
  • Mario Bravo

    (USACH - Universidad de Santiago de Chile [Santiago])

  • Mathieu Faure

    (GREQAM - Groupement de Recherche en Économie Quantitative d'Aix-Marseille - ECM - École Centrale de Marseille - CNRS - Centre National de la Recherche Scientifique - AMU - Aix Marseille Université - EHESS - École des hautes études en sciences sociales)

Abstract

Consider a two-player normal-form game repeated over time. We introduce an adaptive learning procedure, where the players only observe their own realized payoff at each stage. We assume that agents do not know their own payoff function and have no information on the other player. Furthermore, we assume that they have restrictions on their own actions such that, at each stage, their choice is limited to a subset of their action set. We prove that the empirical distributions of play converge to the set of Nash equilibria for zero-sum and potential games, and games where one player has two actions.

Suggested Citation

  • Mario Bravo & Mathieu Faure, 2015. "Reinforcement Learning with Restrictions on the Action Set," Post-Print hal-01457301, HAL.
  • Handle: RePEc:hal:journl:hal-01457301
    DOI: 10.1137/130936488
    Note: View the original document on HAL open archive server: https://hal-amu.archives-ouvertes.fr/hal-01457301
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Fudenberg Drew & Kreps David M., 1993. "Learning Mixed Equilibria," Games and Economic Behavior, Elsevier, vol. 5(3), pages 320-367, July.
    2. Berger, Ulrich, 2005. "Fictitious play in 2 x n games," Journal of Economic Theory, Elsevier, vol. 120(2), pages 139-154, February.
    3. Erev, Ido & Roth, Alvin E, 1998. "Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria," American Economic Review, American Economic Association, vol. 88(4), pages 848-881, September.
    4. Ed Hopkins, 2002. "Two Competing Models of How People Learn in Games," Econometrica, Econometric Society, vol. 70(6), pages 2141-2166, November.
    5. Hopkins, Ed & Posch, Martin, 2005. "Attainability of boundary points under reinforcement learning," Games and Economic Behavior, Elsevier, vol. 53(1), pages 110-125, October.
    6. Mathieu Faure & Gregory Roth, 2010. "Stochastic Approximations of Set-Valued Dynamical Systems: Convergence with Positive Probability to an Attractor," Mathematics of Operations Research, INFORMS, vol. 35(3), pages 624-640, August.
    7. Michel Benaim & Mathieu Faure, 2010. "Stochastic Approximation, Cooperative Dynamics and Supermodular Games," Levine's Working Paper Archive 814577000000000437, David K. Levine.
    8. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    9. Sergiu Hart & Andreu Mas-Colell, 2013. "A Simple Adaptive Procedure Leading To Correlated Equilibrium," World Scientific Book Chapters, in: Simple Adaptive Strategies From Regret-Matching to Uncoupled Dynamics, chapter 2, pages 17-46, World Scientific Publishing Co. Pte. Ltd..
    10. Martin Posch, 1997. "Cycling in a stochastic learning algorithm for normal form games," Journal of Evolutionary Economics, Springer, vol. 7(2), pages 193-207.
    11. Borgers, Tilman & Sarin, Rajiv, 1997. "Learning Through Reinforcement and Replicator Dynamics," Journal of Economic Theory, Elsevier, vol. 77(1), pages 1-14, November.
    12. Gilboa, Itzhak & Matsui, Akihiko, 1991. "Social Stability and Equilibrium," Econometrica, Econometric Society, vol. 59(3), pages 859-867, May.
    13. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945.
    14. Monderer, Dov & Shapley, Lloyd S., 1996. "Fictitious Play Property for Games with Identical Interests," Journal of Economic Theory, Elsevier, vol. 68(1), pages 258-265, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dai Zusai, 2018. "Gains in evolutionary dynamics: A unifying and intuitive approach to linking static and dynamic stability," Papers 1805.04898, arXiv.org, revised Jun 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leslie, David S. & Collins, E.J., 2006. "Generalised weakened fictitious play," Games and Economic Behavior, Elsevier, vol. 56(2), pages 285-298, August.
    2. Panayotis Mertikopoulos & William H. Sandholm, 2016. "Learning in Games via Reinforcement and Regularization," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1297-1324, November.
    3. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.
    4. Hofbauer, Josef & Hopkins, Ed, 2005. "Learning in perturbed asymmetric games," Games and Economic Behavior, Elsevier, vol. 52(1), pages 133-152, July.
    5. Jacques Durieu & Philippe Solal, 2012. "Models of Adaptive Learning in Game Theory," Chapters, in: Richard Arena & Agnès Festré & Nathalie Lazaric (ed.), Handbook of Knowledge and Economics, chapter 11, Edward Elgar Publishing.
    6. Ianni, Antonella, 2014. "Learning strict Nash equilibria through reinforcement," Journal of Mathematical Economics, Elsevier, vol. 50(C), pages 148-155.
    7. Jonathan Newton, 2018. "Evolutionary Game Theory: A Renaissance," Games, MDPI, Open Access Journal, vol. 9(2), pages 1-67, May.
    8. Sandholm, William H., 2015. "Population Games and Deterministic Evolutionary Dynamics," Handbook of Game Theory with Economic Applications,, Elsevier.
    9. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Gotts, Nicholas M. & Polhill, J. Gary, 2007. "Transient and asymptotic dynamics of reinforcement learning in games," Games and Economic Behavior, Elsevier, vol. 61(2), pages 259-276, November.
    10. Oyarzun, Carlos & Ruf, Johannes, 2014. "Convergence in models with bounded expected relative hazard rates," Journal of Economic Theory, Elsevier, vol. 154(C), pages 229-244.
    11. Beggs, A.W., 2005. "On the convergence of reinforcement learning," Journal of Economic Theory, Elsevier, vol. 122(1), pages 1-36, May.
    12. Hopkins, Ed, 1999. "A Note on Best Response Dynamics," Games and Economic Behavior, Elsevier, vol. 29(1-2), pages 138-150, October.
    13. Naoki Funai, 2019. "Convergence results on stochastic adaptive learning," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 68(4), pages 907-934, November.
    14. Oyarzun, Carlos & Sarin, Rajiv, 2013. "Learning and risk aversion," Journal of Economic Theory, Elsevier, vol. 148(1), pages 196-225.
    15. Alanyali, Murat, 2010. "A note on adjusted replicator dynamics in iterated games," Journal of Mathematical Economics, Elsevier, vol. 46(1), pages 86-98, January.
    16. Duffy, John & Hopkins, Ed, 2005. "Learning, information, and sorting in market entry games: theory and evidence," Games and Economic Behavior, Elsevier, vol. 51(1), pages 31-62, April.
    17. Andriy Zapechelnyuk, 2009. "Limit Behavior of No-regret Dynamics," Discussion Papers 21, Kyiv School of Economics.
    18. Josephson, Jens, 2008. "A numerical analysis of the evolutionary stability of learning rules," Journal of Economic Dynamics and Control, Elsevier, vol. 32(5), pages 1569-1599, May.
    19. Erik Mohlin & Robert Ostling & Joseph Tao-yi Wang, 2014. "Learning by Imitation in Games: Theory, Field, and Laboratory," Economics Series Working Papers 734, University of Oxford, Department of Economics.
    20. Tassos Patokos, 2014. "Introducing Disappointment Dynamics and Comparing Behaviors in Evolutionary Games: Some Simulation Results," Games, MDPI, Open Access Journal, vol. 5(1), pages 1-25, January.

    More about this item

    Keywords

    Economie quantitative;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-01457301. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://hal.archives-ouvertes.fr/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.